OptionalconfigPer-test config (tags + metadata recorded on the run).
OptionaldryWhen true, this test runs as an ordinary local test only — no dataset
example is created and no experiment run or annotations are uploaded to
Phoenix. Useful for scaffolding a case before it's ready to track.
OptionalidOptional stable example id; used to upsert the example between runs.
The example's input — fed to the task under evaluation. Required.
OptionalmetadataAdditional metadata stored on the example and its run.
OptionalrepetitionsNumber of times to run this test case. Each repetition becomes a
separate experiment run against the same dataset example (carrying a
distinct repetition_number). Overrides the suite-level repetitions.
Defaults to the suite value, then PHOENIX_TEST_REPETITIONS, then 1.
OptionalsplitsSplit assignment(s) for the example, used to slice the dataset and
experiment in the Phoenix UI (e.g. ["factual_accuracy", "correct"]).
The
Examplefields that define a single test case, excluding its expected output (which is supplied separately via ReferenceOutput).inputis the example's input — the value fed to the task under evaluation. When the case is tracked, this becomes the dataset example'sinput.