Class ClassificationEvaluator<ExampleType>

An LLM evaluator that performs evaluation via classification

Type Parameters

ExampleType extends Record<string, unknown>

Hierarchy (View Summary)

LLMEvaluator<ExampleType>
- ClassificationEvaluator

Index

Constructors

Properties

evaluatorFn name optimizationDirection? promptTemplate source

Accessors

promptTemplateVariables

Methods

Constructors

constructor

new ClassificationEvaluator<ExampleType extends Record<string, unknown>>(
args: CreateClassificationEvaluatorArgs,
): ClassificationEvaluator<ExampleType>
Type Parameters
- ExampleType extends Record<string, unknown>
Parameters
- args: CreateClassificationEvaluatorArgs
Returns ClassificationEvaluator<ExampleType>
Overrides LLMEvaluator.constructor
- Defined in llm/ClassificationEvaluator.ts:19

Properties

`Readonly`evaluatorFn

evaluatorFn: EvaluatorFn<ExampleType>

`Readonly`name

name: string

The name of the evaluator / the metric that it measures

`Optional` `Readonly`optimizationDirection

optimizationDirection?: OptimizationDirection

The direction to optimize the numeric evaluation score E.x. "MAXIMIZE" means that the higher the score, the better the evaluation

`Readonly`promptTemplate

promptTemplate: string

`Readonly`source

source: "LLM" = ...

The source of the evaluation. Also known as the "kind" of evaluator.

Accessors

promptTemplateVariables

get promptTemplateVariables(): string[]
List out the prompt template variables needed to perform evaluation

Returns string[]
- Defined in llm/ClassificationEvaluator.ts:30

Methods

evaluate

evaluate(example: ExampleType): Promise<EvaluationResult>
The function that evaluates the example
Parameters
- example: ExampleType
Returns Promise<EvaluationResult>
Overrides LLMEvaluator.evaluate
- Defined in llm/ClassificationEvaluator.ts:24