The arguments for creating the hallucination evaluator.
Optionalchoices?: ClassificationChoicesMapOptionalinputMapping?: ObjectMapping<RecordType>The mapping of the input to evaluate to the shape that the evaluator expects
Optionalname?: stringOptionaloptimizationDirection?: OptimizationDirectionOptionalpromptTemplate?: PromptTemplateOptionaltelemetry?: TelemetryConfigAn evaluator function that takes a HallucinationEvaluationRecord and returns a classification result indicating whether the response is grounded or hallucinated relative to the conversation.
const evaluator = createHallucinationEvaluator({ model: openai("gpt-4o-mini") });
const result = await evaluator.evaluate({
input:
"User: What's our refund window?\nTool (lookup_policy): Refunds: 30 days from delivery.\nAssistant: 30 days from delivery.\nUser: And for electronics?",
output: "Electronics can be returned within 90 days.",
});
console.log(result.label); // "hallucinated"
Creates a hallucination evaluator function.
This function returns an evaluator that detects whether an assistant's latest response contains claims that are not grounded in — unsupported by, or contradicting — the conversation. Unlike the faithfulness evaluator, which grounds a response against a specific provided context (e.g. retrieved documents), this grounds it against the conversation itself.