Interface: EvalDefinition

A single eval, default-exported from a *.eval.ts file.

Properties

agent?

optional agent: string;

Target agent id. Defaults to the eval's parent server/agents/<id> dir.

dataset?

optional dataset: {
  limit?: number;
  table: string;
};

Run this eval once per row of a Databricks managed evaluation dataset (a Unity Catalog catalog.schema.table with inputs/expectations columns). Each row is bound to t.input/t.expected. Requires the runner to have a workspace client + warehouse (--warehouse-id). Omit for a single-run eval.

limit?

optional limit: number;

table

table: string;

description?

optional description: string;

Short human description, shown in reports.

tags?

optional tags: string[];

Free-form tags for filtering (see the runner's tags / --tag option).

timeoutMs?

optional timeoutMs: number;

Per-eval timeout (ms): runEval races the test against it and records a non-passing result instead of hanging. Overrides the runner/CLI default.

Methods

test()

test(t: TestContext): void | Promise<void>;

The eval body: drive the agent and assert on its behavior.

Parameters

Returns

void | Promise<void>