Why is Cost Tracking Important?
Arize AX’s cost tracking enables you to:- Monitor LLM spend across use cases, models, and providers.
- Inspect token cost
- Track granular costs by token type (e.g., prompt, completion, audio, cache).
- Customize pricing to reflect discounted rates, fine-tuned models, or non-standard providers.
If there is a default you’d like us to add reach out to support@arize.com
Using Cost Tracking in Arize AX
Once cost tracking is enabled, you can analyze and explore cost data at various levels of detail within the Arize AX UI to better understand and manage your LLM application spend.Filtering and Monitoring
All cost attributes are available throughout the platform and can be used to:- Filter traces or spans where cost exceeds a defined threshold
- Create monitors for high-cost traces or model behavior anomalies
- Build dashboards based on specific token types or cost groupings (e.g., prompt, audio, image)
Trace-Level Visualization
At the trace level, Arize aggregates cost across all LLM spans in the trace. This provides a complete view of how much it cost to serve a given request end-to-end, which is especially useful for understanding the impact of complex flows or multi-stage prompts.
Span-Level Visualization
You can also inspect cost at the individual span level, including a breakdown by token type. This allows you to:- Pinpoint expensive steps in the LLM pipeline
- Analyze the relative contribution of different token categories (e.g., reasoning, cache, image)
Set up Cost Tracking
1. Use a Default (Zero Setup)
If your model and provider match a default, Arize automatically applies the correct pricing — no action needed.2. Customize a Default
To tweak an existing config (e.g., apply discounts):- Go to Settings > Cost Tracking > Configuration
- Click Options > Clone on a default config
- Edit fields like token type cost or provider name
3. Create from Scratch
To define your own model config:- Click Add New
- Enter the model name (required)
- Optionally enter the provider
- Specify cost per 1 million tokens for each token type
- Assign each token type to Prompt or Completion
Cost configs are saved at the organization level.
