Skip to main content
Arize ensures that LLM spans in the platform will contain token count values. We use the logic below to compute input and completion token counts per LLM span, and aggregate the counts up to the trace level. A few useful flows for token counts include:
  • Investigating/sorting traces or sessions by token count to find long running traces or sessions
  • Monitoring or setting custom metrics (i.e. for cost) on token count
  • (soon) Monitoring and debugging prompt variables using token counts
Token count calculation states
In the platform, the 3 possible states for token count calculations are shown below.
Token count calculation states in platform
In the platform, you can see token counts aggregated in the tracing table below. Click into a trace to view token counts at the span level.
Token counts in tracing table