Assertions

Every recorded step has assertion points — named fields in the result that are compared against the saved baseline on replay. For each point you choose a mode that controls how strict the comparison is.

Assertion modes

exact

Default for most fields

Deep structural equality. The live result must match the recorded baseline exactly.

shape

Content changes, structure must not

Infers a JSON Schema from the recorded value and validates the live result against it. Same keys and types are required; values may differ.

ignore

Irrelevant fields

Always passes. Use to skip an assertion point that is not meaningful for the test.

Assertion points per action

Each action type exposes a fixed set of points you can assert on. State (how many times something was called) is also checked automatically on every step.

Action Points

Tool call success, isError, content, structuredContent, errorMessage, widget

Resource read success, contents, errorMessage

Widget click success, matched, errorMessage

Widget text input success, matched, applied, errorMessage

State assertions

In addition to result fields, each step checks the state delta — how many times tools were called, resources were read, and widgets were rendered during that step. This catches side-effects that don't show up in the direct result, such as a widget click that silently triggers an extra tool call.

Configuring assertion modes

After a replay, open the result for any step and find the assertion point row. Use the mode selector to change the mode — the change applies immediately without re-running the test. Set a default state mode for all steps in the test settings if you want to relax state checking globally.

When to use each mode

Use exact for tool results that must not change — stable APIs, deterministic outputs.
Use shape when the content is dynamic but the structure (keys, types) must stay the same.
Use ignore for fields you don't care about in a given test, like metadata or debug info.

Next Overview →