Type Alias RunExperimentParamsExperimental

RunExperimentParams: ClientFn & {
    concurrency?: number;
    dataset: DatasetSelector;
    diagLogLevel?: DiagLogLevel;
    dryRun?: number | boolean;
    evaluators?: Evaluator[];
    experimentDescription?: string;
    experimentMetadata?: Record<string, unknown>;
    experimentName?: string;
    logger?: Logger;
    record?: boolean;
    repetitions?: number;
    setGlobalTracerProvider?: boolean;
    task: ExperimentTask;
    useBatchSpanProcessor?: boolean;
}

Parameters for running an experiment.

This feature is not complete, and will change in the future.

Type declaration

  • Optionalconcurrency?: number

    The number of dataset examples to run in parallel

  • dataset: DatasetSelector

    The dataset to run the experiment on

  • OptionaldiagLogLevel?: DiagLogLevel

    Log level to set for the default DiagConsoleLogger when tracing. Omit to disable default diag logging, or to bring your own.

  • OptionaldryRun?: number | boolean

    Whether or not to run the experiment as a dry run. If a number is privided, n examples will be run.

    Default

    false
  • Optionalevaluators?: Evaluator[]

    The evaluators to use

  • OptionalexperimentDescription?: string

    The description of the experiment

  • OptionalexperimentMetadata?: Record<string, unknown>

    Experiment metadata E.x. modelName

  • OptionalexperimentName?: string

    An optional name for the experiment. Defaults to the dataset name + a timestamp

  • Optionallogger?: Logger

    The logger to use

  • Optionalrecord?: boolean

    Whether to record the experiment results

  • Optionalrepetitions?: number

    Number of times to repeat each dataset example

    Default

    1
  • OptionalsetGlobalTracerProvider?: boolean

    Whether to set the global tracer provider when running the task. If set to false, a locally scoped tracer will be created but will not get registered. This may cause certain spans to not be picked up by Phoenix. Notably libraries like the AI SDK that leverage the global tracer.

    Default

    true
  • task: ExperimentTask

    The task to run

  • OptionaluseBatchSpanProcessor?: boolean

Settings

Member Visibility