Interface Evaluator<ExampleType>

The Base Evaluator interface This is the interface that all evaluators must implement

interface Evaluator<ExampleType extends Record<string, unknown>> {
    evaluate: EvaluatorFn<ExampleType>;
    name: string;
    optimizationDirection?: OptimizationDirection;
    source: EvaluationSource;
}

Type Parameters

ExampleType extends Record<string, unknown>

Hierarchy

EvaluatorDescription
- Evaluator

Implemented by

LLMEvaluator

Index

Properties

evaluate name optimizationDirection? source

Properties

evaluate

evaluate: EvaluatorFn<ExampleType>

The function that evaluates the example

name

name: string

The name of the evaluator / the metric that it measures

`Optional`optimizationDirection

optimizationDirection?: OptimizationDirection

The direction to optimize the numeric evaluation score E.x. "MAXIMIZE" means that the higher the score, the better the evaluation

source

source: EvaluationSource

The source of the evaluation. Also known as the "kind" of evaluator.