openinference

OpenInference Instrumentation for OpenAI Node.js SDK

This module provides automatic instrumentation for the OpenAI Node.js SDK which may be used in conjunction with @opentelemetry/sdk-trace-node.

Installation

npm install --save @arizeai/openinference-instrumentation-openai

Usage

To load the OpenAI instrumentation, specify it in the registerInstrumentations call along with any additional instrumentation you wish to enable.

const { NodeTracerProvider } = require("@opentelemetry/sdk-trace-node");
const { OpenAIInstrumentation } = require("@arizeai/openinference-instrumentation-openai");
const { registerInstrumentations } = require("@opentelemetry/instrumentation");

const provider = new NodeTracerProvider();
provider.register();

registerInstrumentations({
  instrumentations: [new OpenAIInstrumentation()],
});

Supported APIs

OpenAI API	Span kind	Span name
Chat Completions (chat.completions.create, incl. streaming)	LLM	OpenAI Chat Completions
Completions (completions.create, incl. streaming)	LLM	OpenAI Completions
Embeddings (embeddings.create)	EMBEDDING	OpenAI Embeddings
Responses (responses.create, incl. streaming)	LLM	OpenAI Responses
Decisions (decisions.create, requires openai >= 7.30.0)	DECISION	OpenAI Decisions

Decisions API

Calls to the Decisions API are recorded as DECISION spans rather than LLM spans: the model scores caller-supplied answers instead of generating text. The span carries decision.system / decision.provider (openai), decision.request.model_name, decision.response.model_name, decision.model_name, decision.token_count.input / decision.token_count.output, and the raw request and response as JSON on input.value / output.value. No llm.* attributes are set on decision spans.

Examples

To run an example, run the following commands:

cd js/packages/openinference-instrumentation-openai
pnpm install
pnpm -r build
npx -y tsx examples/chat.ts # or responses.ts, embed.ts, etc

For more information on OpenTelemetry Node.js SDK, see the OpenTelemetry Node.js SDK documentation.

Using a Custom Tracer Provider

You can specify a custom tracer provider when creating the OpenAI instrumentation. This is useful when you want to use a non-global tracer provider or have more control over the tracing configuration.

import { NodeTracerProvider } from "@opentelemetry/sdk-trace-node";
import { Resource } from "@opentelemetry/resources";
import { SEMRESATTRS_PROJECT_NAME } from "@arizeai/openinference-semantic-conventions";
import { OpenAIInstrumentation } from "@arizeai/openinference-instrumentation-openai";
import { registerInstrumentations } from "@opentelemetry/instrumentation";

// Create a custom tracer provider
const customTracerProvider = new NodeTracerProvider({
  resource: new Resource({
    [SEMRESATTRS_PROJECT_NAME]: "my-openai-project",
  }),
});

// Pass the custom tracer provider to the instrumentation
const openaiInstrumentation = new OpenAIInstrumentation({
  tracerProvider: customTracerProvider,
});

// Register the instrumentation with the custom tracer provider
registerInstrumentations({
  instrumentations: [openaiInstrumentation],
  tracerProvider: customTracerProvider,
});

Alternatively, you can set the tracer provider after creating the instrumentation:

const openaiInstrumentation = new OpenAIInstrumentation();
openaiInstrumentation.setTracerProvider(customTracerProvider);

Compatibility

@arizeai/openinference-instrumentation-openai is compatible with the following versions of the openai package:

OpenAI Version	OpenInference Instrumentation Version
^7.0.0	^4.4.0
^6.0.0	^4.0.0
^5.0.0	^3.0.0
^4.0.0	^2.0.0

Prompt cache token counts

OpenAI cache reads (cached_tokens) and writes (cache_write_tokens) are recorded as llm.token_count.prompt_details.cache_read and cache_write. See the semantic conventions for how these sub-counts relate to the prompt total, and the cache token example for a Phoenix round trip.

This site is open source. Improve this page.