LLM Spans
An LLM span (openinference.span.kind = "LLM") records one call to a large language model, such as a chat
completion or a text completion request to OpenAI, Anthropic, or Cohere. It captures the request sent to the provider
and the response that came back.
Attributes
Requirement levels follow the OpenTelemetry definitions.
| Attribute | Requirement level | Notes |
|---|---|---|
openinference.span.kind |
Required | "LLM" |
llm.system |
Required | The AI system or product, e.g. "openai", "anthropic". See Model Identification |
llm.model_name |
Recommended | The specific model used, e.g. "gpt-4-0613" |
llm.provider |
Recommended | The hosting provider, when it differs from the system, e.g. "azure" |
llm.request.model_name |
Recommended | The model requested by the caller, when it can differ from the model that served the response (e.g. provider-side fallback) |
llm.response.model_name |
Recommended | The model that actually generated the response, when it can differ from the requested model |
llm.invocation_parameters |
Recommended | JSON string of parameters sent to the model |
input.value |
Recommended | The raw input, usually a JSON string |
input.mime_type |
Recommended | Usually "application/json" |
output.value |
Recommended | The raw output, usually a JSON string |
output.mime_type |
Recommended | Usually "application/json" |
llm.input_messages |
Recommended | Flattened list of input messages. See Messages |
llm.output_messages |
Recommended | Flattened list of output messages. See Messages |
llm.tools |
Recommended | Tool definitions advertised to the model. See Tool Calling |
llm.token_count.* |
Recommended | Token usage. See Token Counts |
A completions (non-chat) API call records llm.prompts and llm.choices in place of the message lists; see
Completions API. LLM spans also carry the
context attributes in scope when they are created.
Model Identification
llm.system, llm.provider, and llm.model_name answer different questions:
llm.systemidentifies the AI product or vendor, i.e. the shape of the API the client speaks. It should use a well-known value when one applies (e.g."openai","anthropic","cohere").llm.provideridentifies the hosting provider, i.e. who runs the hardware, when it differs from the system (e.g."azure"for Azure-hosted OpenAI). See the well-known values.llm.model_namecontains the actual model name returned by the API (e.g."gpt-4-0613","claude-3-opus-20240229").
Requested and Response Models
Some providers can route a request to a different model than the one the caller specified (for example,
classifier-triggered fallback). llm.request.model_name and llm.response.model_name let instrumentation record both
values as distinct, queryable attributes when it can tell them apart:
llm.request.model_nameis the model string the caller sent in the request.llm.response.model_nameis the model the provider reports as having generated the response.- Both are optional — only set them when the underlying API response actually distinguishes requested from effective model. Most providers echo the same model back, so these attributes will typically be unset.
llm.model_namekeeps its existing meaning and remains required where applicable: it should equalllm.response.model_namewhen known, falling back tollm.request.model_nameotherwise. This keepsllm.model_namebackward compatible for consumers that don’t yet read the new attributes.
Embedding and decision spans do not use these llm.* identification attributes. See
Embedding Spans and
Decision Spans.
Token Counts
llm.token_count.prompt, llm.token_count.completion, and llm.token_count.total record the token usage the
provider reports. The extended token count attributes break these down:
llm.token_count.prompt_details.*for prompt-related token countsllm.token_count.completion_details.*for completion-related token counts
These enable tracking of multimodal (audio) token usage, monitoring of reasoning token consumption, cache efficiency analysis, and billing reconciliation.
Cache Tokens
llm.token_count.prompt_details.cache_read and llm.token_count.prompt_details.cache_write record cache operations:
cache_readis the number of prompt tokens successfully retrieved from cache (cache hits). For OpenAI, this corresponds tousage.prompt_tokens_details.cached_tokensin Chat Completions API responses andusage.input_tokens_details.cached_tokensin Responses API responses. For Anthropic, when using acache_controlblock, this maps to thecache_read_input_tokensfield in Messages API responses.cache_writeis the number of prompt tokens not found in cache (cache misses) that were subsequently written to cache. For OpenAI, this corresponds to theusage.prompt_tokens_details.cache_write_tokensfield in Chat Completions API responses and theusage.input_tokens_details.cache_write_tokensfield in Responses API responses. For Anthropic, when using acache_controlblock, this maps to thecache_creation_input_tokensfield in Messages API responses.
Sub-Counts Are Inclusive
The prompt_details.* values are sub-counts of llm.token_count.prompt: they are already included in it, so
llm.token_count.prompt is expected to be greater than or equal to their sum. For providers whose reported input
token count excludes cache tokens (e.g. Anthropic’s input_tokens), instrumentations should fold the cache read/write
tokens back into llm.token_count.prompt (and llm.token_count.total) rather than reporting the exclusive value.
Cost
The llm.cost.* attributes record the cost of the call in USD, as floating point values. They mirror the token count
breakdown. Viewed as a JSON-like structure:
{
"prompt": 0.0021, # Cost in USD
"completion": 0.0045, # Cost in USD
"total": 0.0066, # Cost in USD
"completion_details": {
"output": 0.0009, # Cost in USD
"reasoning": 0.0024, # Cost in USD (e.g., 80 tokens * $0.03/1K tokens)
"audio": 0.0012 # Cost in USD (e.g., 40 tokens * $0.03/1K tokens)
},
"prompt_details": {
"input": 0.0003, # Cost in USD
"cache_write": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"cache_read": 0.0003, # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
"cache_input": 0.0006, # Cost in USD (e.g., 20 tokens * $0.03/1K tokens)
"audio": 0.0003 # Cost in USD (e.g., 10 tokens * $0.03/1K tokens)
}
}
Examples
The examples show list attributes in nested JSON form for readability. On the wire they are
flattened, e.g. llm.input_messages.0.message.role. Examples of reasoning
content are in Messages.
Chat Completions
A span for a tool call with OpenAI (shown in logical JSON format for clarity)
{
"name": "ChatCompletion",
"context": {
"trace_id": "409df945-e058-4829-b240-cfbdd2ff4488",
"span_id": "01fa9612-01b8-4358-85d6-e3e067305ec3"
},
"span_kind": "SPAN_KIND_INTERNAL",
"parent_id": "2fe8a793-2cf1-42d7-a1df-bd7d46e017ef",
"start_time": "2024-01-11T16:45:17.982858-07:00",
"end_time": "2024-01-11T16:45:18.517639-07:00",
"status_code": "OK",
"status_message": "",
"attributes": {
"openinference.span.kind": "LLM",
"llm.system": "openai",
"llm.input_messages": [
{
"message.role": "system",
"message.content": "You are a Shakespearean writing assistant who speaks in a Shakespearean style. You help people come up with creative ideas and content like stories, poems, and songs that use Shakespearean style of writing style, including words like \"thou\" and \"hath\u201d.\nHere are some example of Shakespeare's style:\n - Romeo, Romeo! Wherefore art thou Romeo?\n - Love looks not with the eyes, but with the mind; and therefore is winged Cupid painted blind.\n - Shall I compare thee to a summer's day? Thou art more lovely and more temperate.\n"
},
{ "message.role": "user", "message.content": "what is 23 times 87" }
],
"llm.model_name": "gpt-3.5-turbo-0613",
"llm.invocation_parameters": "{\"model\": \"gpt-3.5-turbo-0613\", \"temperature\": 0.1, \"max_tokens\": null}",
"output.value": "{\"tool_calls\": [{\"id\": \"call_Re47Qyh8AggDGEEzlhb4fu7h\", \"function\": {\"arguments\": \"{\\n \\\"a\\\": 23,\\n \\\"b\\\": 87\\n}\", \"name\": \"multiply\"}, \"type\": \"function\"}]}",
"output.mime_type": "application/json",
"llm.output_messages": [
{
"message.role": "assistant",
"message.tool_calls": [
{
"tool_call.function.name": "multiply",
"tool_call.function.arguments": "{\n \"a\": 23,\n \"b\": 87\n}"
}
]
}
],
"llm.token_count.prompt": 229,
"llm.token_count.completion": 21,
"llm.token_count.total": 250
},
"events": []
}
A synthesis call using a function call output
{
"name": "llm",
"context": {
"trace_id": "409df945-e058-4829-b240-cfbdd2ff4488",
"span_id": "f26d1f26-9671-435d-9716-14a87a3f228b"
},
"span_kind": "SPAN_KIND_INTERNAL",
"parent_id": "2fe8a793-2cf1-42d7-a1df-bd7d46e017ef",
"start_time": "2024-01-11T16:45:18.519427-07:00",
"end_time": "2024-01-11T16:45:19.159145-07:00",
"status_code": "OK",
"status_message": "",
"attributes": {
"openinference.span.kind": "LLM",
"llm.system": "openai",
"llm.input_messages": [
{
"message.role": "system",
"message.content": "You are a Shakespearean writing assistant who speaks in a Shakespearean style. You help people come up with creative ideas and content like stories, poems, and songs that use Shakespearean style of writing style, including words like \"thou\" and \"hath\u201d.\nHere are some example of Shakespeare's style:\n - Romeo, Romeo! Wherefore art thou Romeo?\n - Love looks not with the eyes, but with the mind; and therefore is winged Cupid painted blind.\n - Shall I compare thee to a summer's day? Thou art more lovely and more temperate.\n"
},
{
"message.role": "user",
"message.content": "what is 23 times 87"
},
{
"message.role": "assistant",
"message.content": null,
"message.tool_calls": [
{
"tool_call.function.name": "multiply",
"tool_call.function.arguments": "{\n \"a\": 23,\n \"b\": 87\n}"
}
]
},
{
"message.role": "tool",
"message.content": "2001",
"message.name": "multiply"
}
],
"llm.model_name": "gpt-3.5-turbo-0613",
"llm.invocation_parameters": "{\"model\": \"gpt-3.5-turbo-0613\", \"temperature\": 0.1, \"max_tokens\": null}",
"output.value": "The product of 23 times 87 is 2001.",
"output.mime_type": "text/plain",
"llm.output_messages": [
{
"message.role": "assistant",
"message.content": "The product of 23 times 87 is 2001."
}
],
"llm.token_count.prompt": 259,
"llm.token_count.completion": 14,
"llm.token_count.total": 273
},
"events": [],
"conversation": null
}
Completions
A span for a simple completion (shown in logical JSON format for clarity)
{
"name": "Completion",
"context": {
"trace_id": "12345678-1234-5678-1234-567812345678",
"span_id": "87654321-4321-8765-4321-876543218765"
},
"span_kind": "SPAN_KIND_INTERNAL",
"parent_id": null,
"start_time": "2025-09-29T03:42:49.000000Z",
"end_time": "2025-09-29T03:42:50.284841Z",
"status_code": "OK",
"status_message": "",
"attributes": {
"openinference.span.kind": "LLM",
"llm.system": "openai",
"llm.model_name": "babbage:2023-07-21-v2",
"llm.invocation_parameters": "{\"model\": \"babbage-002\", \"temperature\": 0.4, \"top_p\": 0.9, \"max_tokens\": 25}",
"input.value": "{\"model\": \"babbage-002\", \"prompt\": \"def fib(n):\\n if n <= 1:\\n return n\\n else:\\n return fib(n-1) + fib(n-2)\", \"temperature\": 0.4, \"top_p\": 0.9, \"max_tokens\": 25}",
"input.mime_type": "application/json",
"llm.prompts.0.prompt.text": "def fib(n):\n if n <= 1:\n return n\n else:\n return fib(n-1) + fib(n-2)",
"output.value": "{\"id\": \"cmpl-CKz4klHa1MMqAa4hQn3yzIMlLMZHd\", \"object\": \"text_completion\", \"created\": 1759117370, \"model\": \"babbage:2023-07-21-v2\", \"choices\": [{\"text\": \" + fib(n-3) + fib(n-4)\\n\\ndef fib(n):\\n if n <= 1:\\n return\", \"index\": 0, \"finish_reason\": \"length\"}], \"usage\": {\"prompt_tokens\": 31, \"completion_tokens\": 25, \"total_tokens\": 56}}",
"output.mime_type": "application/json",
"llm.choices.0.completion.text": " + fib(n-3) + fib(n-4)\n\ndef fib(n):\n if n <= 1:\n return",
"llm.token_count.prompt": 31,
"llm.token_count.completion": 25,
"llm.token_count.total": 56
},
"events": []
}