Class LLMEvaluator<RecordType>`Abstract`

Base class for llm evaluation metrics / scores

Type Parameters

RecordType extends Record<string, unknown>

Hierarchy (View Summary)

LLMEvaluator
- ClassificationEvaluator

Implements

Evaluator<RecordType>

Index

Constructors

Properties

name optimizationDirection? source

Methods

Constructors

constructor

new LLMEvaluator<RecordType extends Record<string, unknown>>(
__namedParameters: CreateEvaluatorArgs,
): LLMEvaluator<RecordType>
Type Parameters
- RecordType extends Record<string, unknown>
Parameters
- __namedParameters: CreateEvaluatorArgs
Returns LLMEvaluator<RecordType>
- Defined in llm/LLMEvaluator.ts:17

Properties

`Readonly`name

name: string

The name of the evaluator / the metric that it measures

`Optional` `Readonly`optimizationDirection

optimizationDirection?: OptimizationDirection

The direction to optimize the numeric evaluation score E.x. "MAXIMIZE" means that the higher the score, the better the evaluation

`Readonly`source

source: "LLM" = ...

The source of the evaluation. Also known as the "kind" of evaluator.

Methods

`Abstract`evaluate

evaluate(_example: RecordType): Promise<EvaluationResult>
The function that evaluates the example
Parameters
- _example: RecordType
Returns Promise<EvaluationResult>
Implementation of Evaluator.evaluate
- Defined in llm/LLMEvaluator.ts:21