The arguments for creating the document relevancy evaluator.
Optional
choices?: ClassificationChoicesMapOptional
name?: stringOptional
optimizationDirection?: OptimizationDirectionOptional
promptTemplate?: stringOptional
telemetry?: { isEnabled?: boolean; tracer?: Tracer }Optional
isEnabled?: booleanWhether OpenTelemetry is enabled on the call. Defaults to true for visibility into the evals calls.
Optional
tracer?: TracerThe tracer to use for the call. If not provided, the traces will get picked up by the global tracer.
An evaluator function that takes a DocumentRelevancyExample and returns a classification result indicating whether the document is relevant to the input question.
const evaluator = createDocumentRelevancyEvaluator({ model: openai("gpt-4o-mini") });
const result = await evaluator.evaluate({
input: "What is the capital of France?",
documentText: "Paris is the capital and most populous city of France.",
});
console.log(result.label); // "relevant" or "unrelated"
Creates a document relevancy evaluator function.
This function returns an evaluator that determines whether a given document text is relevant to a provided input question. The evaluator uses a classification model and a prompt template to make its determination.