Class LLMEvaluator<ExampleType>`Abstract`

Base class for llm evaluation metrics / scores

Type Parameters

ExampleType extends Record<string, unknown>

Hierarchy (View Summary)

LLMEvaluator
- ClassificationEvaluator

Implements

Evaluator<ExampleType>

Index

Constructors

Properties

name optimizationDirection? source

Methods

Constructors

constructor

new LLMEvaluator<ExampleType extends Record<string, unknown>>(
__namedParameters: CreateEvaluatorArgs,
): LLMEvaluator<ExampleType>
Type Parameters
- ExampleType extends Record<string, unknown>
Parameters
- __namedParameters: CreateEvaluatorArgs
Returns LLMEvaluator<ExampleType>
- Defined in llm/LLMEvaluator.ts:17

Properties

`Readonly`name

name: string

The name of the evaluator / the metric that it measures

`Optional` `Readonly`optimizationDirection

optimizationDirection?: OptimizationDirection

The direction to optimize the numeric evaluation score E.x. "MAXIMIZE" means that the higher the score, the better the evaluation

`Readonly`source

source: "LLM" = ...

The source of the evaluation. Also known as the "kind" of evaluator.

Methods

`Abstract`evaluate

evaluate(_example: ExampleType): Promise<EvaluationResult>
The function that evaluates the example
Parameters
- _example: ExampleType
Returns Promise<EvaluationResult>
Implementation of Evaluator.evaluate
- Defined in llm/LLMEvaluator.ts:21