Can your extractor return a record you can actually use?

1. Start with a clear extraction contract

In the Test Bench, choose Load support extraction starter. It replaces the current project after confirmation, so download existing work first. Or open the downloaded starter using Open project JSON.

Variant A asks for fields. Variant B specifies types, allowed issue labels, missing-data handling, ambiguity and raw JSON output. Both see the same customer message.

• order_id: a string, or null when missing or ambiguous.
• issue: damaged_item, late_delivery, duplicate_charge or other.
• refund_requested: a boolean; true only for an explicit request.
• amount: the stated number, or null. No calculations.
• currency: an explicit three-letter code, or null. A bare dollar sign is not enough.

These are choices for this task, not universal definitions. Change both the instructions and expected values when your actual workflow needs different rules.

2. Cover ordinary, missing and ambiguous inputs

The starter includes expected JSON for each case and rejects extra keys. Expectations belong to the test, not to the model prompt: the rendered prompt does not include the expected answer. Inspect the prompt before copying it.

For the first case, the expected record is:

{
  "order_id": "ORD-1042",
  "issue": "damaged_item",
  "refund_requested": true,
  "amount": 29.9,
  "currency": "EUR"
}

A response with "amount": "29.90" is valid JSON but fails the expected number type. A response with "refund_requested": "true" fails the boolean type. A fenced Markdown block fails the raw-JSON requirement. Reordering object keys does not fail; changing a value does.

3. Run six prompts, then review the differences

For less scrolling, choose Open focused collection in step 3 of the bench and collect one variant across the set. Copy → paste raw response → record and next unfinished; then switch variants. Current recordings with no draft edits are skipped, even when their checks fail. Finish collection first, then review quality in the whole-set matrix. Drafts use the same project saving/downloads as the main view.

1. Copy A for the first case. Run it in a fresh chat and paste the response unchanged. Record your model/settings label and click Record response for current prompt.
2. Run B in another fresh chat with the same model/settings. Do not repair either answer before recording it; failures are useful evidence.
3. Read the field-level differences below each response. Accept or reject it yourself, with a note explaining the impact on your workflow.
4. Use Next case for the remaining two inputs. Compare the whole-set matrix, not just one impressive answer.
5. Download the editable project and the Markdown comparison report. Both contain your inputs, responses and notes; review them before sharing.

A passing record matches your configured expectation; it does not prove the expectation is correct, cover all future inputs or measure model reliability. Review the source and decide whether the contract itself needs changing. Record repeated trials if variability matters; earlier recordings now stay in Trial history.

4. Adapt the test to your own recurring work

Replace the fictional tickets with representative, appropriately redacted examples. Keep at least one incomplete and one ambiguous case. Use duplicate case to copy the checks without copying responses. Expected JSON can be a partial object: turn off Reject extra object keys if you only care about a subset of fields. Nested objects work too; arrays always require the expected order and length.

Changing expected values rechecks existing recorded responses using the current rules. Changing the actual prompt or case input marks the old response stale. These are different operations: a changed test is not a new model run.

JSON checks use JavaScript parsing and numbers, not arbitrary-precision arithmetic or JSON Schema. Duplicate keys retain the last value; object key order is ignored, strings are case-sensitive, and no Markdown wrapper is removed. Blank expected JSON checks syntax only. If exact source representation, duplicate-key rejection, schemas or automatic model execution matter, this first version does not provide them.

Remember is opt-in and browser-local; server backups cannot recover your projects. No API key, account or project upload is needed.

5. Revise the contract without losing the earlier evidence

Record another response for the same case and variant. The previous response, exact prompt, rules and review move into Trial history; they are not replaced or counted twice. Compare any earlier recording with the latest using the case, variant and recording selectors.

Try this without making a model call: download the worked history example, then use Open project JSON in the bench. This replaces your on-screen project after confirmation; download existing work first. Open Trial history, select Explicit refund and B · explicit extraction contract.

All three responses in this worked example were authored by the agent to illustrate the tool. The last deliberately contains an error. They are not sampled model outputs, a comparison of model quality or real customer data. Device timestamps record the local exercise, not model executions. Other cases are left unrun.

1. The first recording passes the original five-field contract. Its acceptance note remains intact.
2. The second uses revised instructions and adds triage_note. Select the first earlier recording: its saved rules pass, but current rules fail because the new field is missing. That does not retroactively make the old review dishonest; the contract changed.
3. The latest is a deliberately failing repeat under the same six-field instructions: refund_requested is false despite an explicit request. Select the second earlier recording to compare same-prompt outputs. Only the latest result counts in the matrix; two older acceptances do not hide the new failure.

When changing shared instructions, review every case’s expected values. The worked example updates the other two cases with triage_note: "clarify" but does not pretend they have been run again.

Projects and Markdown reports include earlier trials and their rules. Up to 20 recordings are retained per case/variant, including the latest, with a 10 MB project-file cap. Nothing is silently trimmed; download before explicitly deleting earlier entries to make space. Browser storage may fill sooner. Version-1/2 imports preserve their results, but their original dates and rules are unavailable rather than reconstructed. This is recording history, not full project-version recovery or verified execution metadata.

Expand to a reusable regression pack

Download sixteen support-extraction cases with expected JSON and per-case rationales. The free pack covers corrections, negated and hypothetical requests, speaker attribution, zero amounts, task-specific issue priority and untrusted input. It supplies a refined contract, an unrun project and portable CSV/JSON; it is authored task material, not measured model performance.