Interface PhoenixTest

Declare a single Phoenix eval test case.

Drop-in replacement for the test runner's own test / it. The params argument carries the input and the reference output (expected / reference / output) that become the dataset example; whatever the handler returns (or passes to logOutput()) is recorded as the experiment run's output and made available to evaluators.

it is the canonical alias for test; the two are identical.

Example

px.test(
  "summarizes the article",
  { input: { article }, expected: { summary } },
  async ({ input, expected }) => {
    const output = await summarize(input.article);
    px.logOutput(output);
    await px.evaluate({ name: "matches", evaluate: () => output === expected.summary });
  }
);

interface PhoenixTest {
    each<Input extends KVMap, Expected extends KVMap>(
        table: TestEachRow<Input, Expected>[],
    ): PhoenixTestEach<Input, Expected>;
    only<Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
        name: string,
        params: TestParams<Input, Expected>,
        fn: TestFn<Input, Expected>,
        timeout?: number,
    ): void;
    skip<Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
        name: string,
        params: TestParams<Input, Expected>,
        fn: TestFn<Input, Expected>,
        timeout?: number,
    ): void;
    <Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
        name: string,
        params: TestParams<Input, Expected>,
        fn: TestFn<Input, Expected>,
        timeout?: number,
    ): void;
}

PhoenixTest<Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
    name: string,
    params: TestParams<Input, Expected>,
    fn: TestFn<Input, Expected>,
    timeout?: number,
): void
Declare a single Phoenix eval test case.
Type Parameters
- Input extends KVMap = KVMap
- Expected extends KVMap = KVMap
Parameters
- name: string
  Test case name; doubles as the dataset example label.
- params: TestParams<Input, Expected>
  Inline input and reference output that become the dataset example.
- fn: TestFn<Input, Expected>
  Test handler; receives { input, expected, metadata }.
- Optionaltimeout: number
  Optional per-test timeout in milliseconds.
Returns void
- Defined in packages/phoenix-client/src/testing/define-api.ts:109

Index

Methods

each only skip

Methods

each

each<Input extends KVMap, Expected extends KVMap>(
table: TestEachRow<Input, Expected>[],
): PhoenixTestEach<Input, Expected>
Run the same test handler across many examples. Returns a function that takes a name (or template / name-builder) and the shared test body; each row in table becomes its own dataset example and experiment run.
Type Parameters
- Input extends KVMap
- Expected extends KVMap
Parameters
- table: TestEachRow<Input, Expected>[]
  Rows of { input, expected?, metadata?, ... } to fan out over.
Returns PhoenixTestEach<Input, Expected>
A PhoenixTestEach that binds the name and shared handler.
Example
```
px.test.each([
  { input: { a: 1, b: 2 }, expected: { sum: 3 } },
  { input: { a: 2, b: 2 }, expected: { sum: 4 } },
])("adds %j", async ({ input, expected }) => {
  // ...
});
```
- Defined in packages/phoenix-client/src/testing/define-api.ts:163

only

only<Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
    name: string,
    params: TestParams<Input, Expected>,
    fn: TestFn<Input, Expected>,
    timeout?: number,
): void
Run only this test case, skipping its siblings (matches the runner's test.only).
Type Parameters
- Input extends KVMap = KVMap
- Expected extends KVMap = KVMap
Parameters
- name: string
  Test case name; doubles as the dataset example label.
- params: TestParams<Input, Expected>
  Inline input and reference output that become the dataset example.
- fn: TestFn<Input, Expected>
  Test handler; receives { input, expected, metadata }.
- Optionaltimeout: number
  Optional per-test timeout in milliseconds.
Returns void
- Defined in packages/phoenix-client/src/testing/define-api.ts:124

skip

skip<Input extends KVMap = KVMap, Expected extends KVMap = KVMap>(
    name: string,
    params: TestParams<Input, Expected>,
    fn: TestFn<Input, Expected>,
    timeout?: number,
): void
Skip this test case (matches the runner's test.skip). No dataset example or experiment run is created on Phoenix.
Type Parameters
- Input extends KVMap = KVMap
- Expected extends KVMap = KVMap
Parameters
- name: string
  Test case name; doubles as the dataset example label.
- params: TestParams<Input, Expected>
  Inline input and reference output (not used while skipped).
- fn: TestFn<Input, Expected>
  Test handler (not executed).
- Optionaltimeout: number
  Optional per-test timeout in milliseconds.
Returns void
- Defined in packages/phoenix-client/src/testing/define-api.ts:139

Interface PhoenixTest

Example

Type Parameters

Parameters

Returns void

Index

Methods

Methods

each

Type Parameters

Parameters

Returns PhoenixTestEach<Input, Expected>

Example

only

Type Parameters

Parameters

Returns void

skip

Type Parameters

Parameters

Returns void

Settings

On This Page