Class ClassificationEvaluator<RecordType>

An LLM evaluator that performs evaluation via classification

Type Parameters

RecordType extends Record<string, unknown>

Hierarchy (View Summary)

LLMEvaluator<RecordType>
- ClassificationEvaluator

Index

Constructors

Properties

evaluatorFn name optimizationDirection? promptTemplate source

Accessors

promptTemplateVariables

Methods

Constructors

constructor

new ClassificationEvaluator<RecordType extends Record<string, unknown>>(
args: CreateClassificationEvaluatorArgs,
): ClassificationEvaluator<RecordType>
Type Parameters
- RecordType extends Record<string, unknown>
Parameters
- args: CreateClassificationEvaluatorArgs
Returns ClassificationEvaluator<RecordType>
Overrides LLMEvaluator.constructor
- Defined in llm/ClassificationEvaluator.ts:20

Properties

`Readonly`evaluatorFn

evaluatorFn: EvaluatorFn<RecordType>

`Readonly`name

name: string

The name of the evaluator / the metric that it measures

`Optional` `Readonly`optimizationDirection

optimizationDirection?: OptimizationDirection

The direction to optimize the numeric evaluation score E.x. "MAXIMIZE" means that the higher the score, the better the evaluation

`Readonly`promptTemplate

promptTemplate: string

`Readonly`source

source: "LLM" = ...

The source of the evaluation. Also known as the "kind" of evaluator.

Accessors

promptTemplateVariables

get promptTemplateVariables(): string[]
List out the prompt template variables needed to perform evaluation

Returns string[]
- Defined in llm/ClassificationEvaluator.ts:31

Methods

evaluate

evaluate(example: RecordType): Promise<EvaluationResult>
The function that evaluates the example
Parameters
- example: RecordType
Returns Promise<EvaluationResult>
Overrides LLMEvaluator.evaluate
- Defined in llm/ClassificationEvaluator.ts:25