Type Alias RunExperimentParamsExperimental

RunExperimentParams: ClientFn & {
    concurrency?: number;
    dataset: DatasetSelector;
    dryRun?: number | boolean;
    evaluators?: Evaluator[];
    experimentDescription?: string;
    experimentMetadata?: Record<string, unknown>;
    experimentName?: string;
    logger?: Logger;
    record?: boolean;
    task: ExperimentTask;
}

Parameters for running an experiment.

This feature is not complete, and will change in the future.

Type declaration

  • Optionalconcurrency?: number

    The number of dataset examples to run in parallel

  • dataset: DatasetSelector

    The dataset to run the experiment on

  • OptionaldryRun?: number | boolean

    Whether or not to run the experiment as a dry run. If a number is privided, n examples will be run.

    Default

    false
  • Optionalevaluators?: Evaluator[]

    The evaluators to use

  • OptionalexperimentDescription?: string

    The description of the experiment

  • OptionalexperimentMetadata?: Record<string, unknown>

    Experiment metadata E.x. modelName

  • OptionalexperimentName?: string

    An optional name for the experiment. Defaults to the dataset name + a timestamp

  • Optionallogger?: Logger

    The logger to use

  • Optionalrecord?: boolean

    Whether to record the experiment results

  • task: ExperimentTask

    The task to run

Settings

Member Visibility