Repeatability · stored evidence, not an accuracy estimate
Keep the earlier answers in the decision.
The latest result can hide yesterday’s failure. Repeated recordings let you see whether the same prompt keeps failing, alternates outcomes, or simply repeats the same mistake.
Use your own repeated recordings
- Choose representative cases and write checks before inspecting responses. Keep both prompt variants, model and generation settings fixed for your planned comparison; decide how many repetitions you need beforehand. This tool does not choose a sample size or guarantee independent samples.
- Collect each repetition as a new recording. In the main editor, recording again preserves the previous output and review. With response worksheets, import each completed round separately into the unchanged project. The focused collection view intentionally avoids recording an unchanged current answer twice; use the main editor or batch import for deliberate repeats.
- For the optional local runner, use the original blank worksheet and a new output filename for each planned round. Resume is for finishing an interrupted round, not repeating completed calls. Intentionally changed seeds/settings must be documented. Keep the raw receipts; the browser imports outputs/model labels, not execution metadata.
- Open Inspect repeatability across retained recordings near Trial history. The whole-set matrix shows eligible, excluded and repeated groups. Choose a case/variant; inspect each exact-label group and expand raw recordings, including excluded evidence.
- Review failures before editing the prompt. Download the repeated-recording report and project JSON. A saved project decision or latest-only HTML handoff captures latest recordings only, not these repeated groups. Choose the separate all-recordings HTML handoff to share every retained response and review context in one offline reading file.
Review every round—not only the last answer
Importing a third worksheet moves the first two responses into history. You can now assess those earlier recordings directly, without copying them back into the editor or falsely recording them as new runs. Start with the 24-response practice project without reviews: it contains the actual retained outputs, not newly generated answers or new requests. This is a separate practice copy; the original reviewed project and raw execution worksheets stay unchanged.
- Open the practice project and expand Inspect repeatability across retained recordings. Select a case and variant, expand an earlier recording, and choose Review this earlier recording. Alternatively use the case, variant and earlier-recording selectors in Trial history. The latest recording’s link still goes to the main review editor.
- Read the exact recorded prompt and raw response in the earlier-review panel. It does not substitute today’s edited source or response draft. Choose Start / restart this earlier review; assess each current criterion, add a reason and explicitly decide. All-pass ratings do not auto-accept.
- Review the other retained rounds, then inspect the repeatability counts. Earlier reviews do not change the latest-only comparison, baseline or saved-decision evidence. No earlier answer is promoted to latest, even if you accept it.
- If you change your criteria or case checks, existing bound reviews become out of date. Restart the chosen earlier review to archive its old criteria, assessments, checks, verdict and note before evaluating it under the new policy. Up to ten earlier review contexts per recording; at the limit nothing is silently removed. Download before explicitly deleting an archived review.
- Keep project JSON and the full repeated-recording report. Remember saves these changes only if browser storage succeeds. Latest-only HTML handoffs still omit earlier recordings; the separate all-recordings handoff includes them, with all saved review contexts and explicit exclusions. Prompt-stale records may be documented, but stay excluded from repeated counts even after a complete review.
Review is not execution. Reassessing a stored answer does not rerun the prompt, update its recording date, verify its model label or establish independent agreement. Editing within one review replaces that assessment; restarting is what retains its earlier context. The practice file removes review fields in a separate copy so you can assess it yourself; it does not erase the original published judgments. The existing same-agent review is available for comparison, not an independent answer key.
What is counted?
- Eligible: a nonblank retained response whose exact recorded prompt matches the currently rendered prompt for that case/variant. Earlier and latest recordings both count; unrecorded drafts never count. A prompt edit excludes old records without deleting them; exact restoration can make them eligible again.
- Groups: same case, variant and exact model/settings label. Different labels are not pooled. A matching label is still unverified and may conceal changed settings or pasted duplicates. Blank means unknown, not controlled execution.
- Automatic checks: rerun current case rules on every eligible output. Changing expected values does not change the prompt; it changes this recheck. No checks means unchecked, not pass. All checks passing is not a guarantee of semantic correctness. A mixed-check group has both passes and failures.
- Review: the latest saved review on each recording counts only when its criteria and case-check context match the current project. Older review contexts stay inspectable. All-pass criteria never automatically accept an answer.
- Distinct response text: exact stored text, including whitespace and JSON key order. Identical outputs remain separate recordings, not deduplicated evidence. Normalized editor line endings are part of the existing project format.
These are descriptive counts of retained recordings, not a trial registry. Deletions, cherry-picking, pasted duplicates, unknown settings and correlated generations can bias them. No pooled pass rate, confidence interval, model ranking, causal improvement or predicted reliability is inferred. Counts do not reconstruct deleted records, decisions’ older snapshots or previous project files.
Actual exercise: stable answers can still be wrong
We fixed four new fictional support messages and their expected records before requesting outputs, keeping the original three-case starter’s A/B prompt templates unchanged. We then ran three sequential rounds, each with both variants on all four cases: 24 actual local-model requests. These are not authored “model” answers. Cases, expectations and the subsequent per-response review are authored by the same agent, however—not independent ground truth or a blinded held-out evaluation.
Qwen2.5-0.5B-Instruct Q4_K_M through CPU-only llama.cpp b12081, context 1,024, one request at a time, maximum requested output 160 tokens. Temperature 0.7; requested seeds 101, 202 and 303 for rounds 1–3. The labels in the project identify the requested model only; settings and device timings live in the three original worksheets. Seed changes are deliberate, not a claim of identical generation settings or statistical independence. The temporary, resource-limited loopback service is stopped; there is no public inference endpoint.
Hold this configuration for unattended extraction. In all eight case/variant groups, all three responses fail the full expected contract. A has 0/3 passes in each of four cases; B also has 0/3 in each. Every recording is reviewed and rejected. There is no mixed-pass group and no successful case to promote.
24/24 retained recordings match a current prompt and have nonblank output; 0 excluded. 8 same-case/variant/label groups have at least two recordings; 0 groups contain both automatic passes and failures. 0/8 case/variant pairs have no eligible recording. These are counts of stored evidence, not independent trials or measured reliability.
Local provider-reported tokens: 3396 prompt + 1421 completion = 4817 total, separate from agent-controller tokens. Summed observed request duration 162.760 seconds, not a speed benchmark. Twenty-three finish reasons were stop; one was length. The truncated answer remains unchanged. No paid API endpoint used; incremental host cost unmeasured.
Model-service peak cgroup memory 576,966,656 bytes includes cache, not just process RSS. The model and exercise services stopped after completion, with no listening model port at the recorded check.
| Case | Variant A | Variant B |
|---|---|---|
| Small explicit refund | 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) | 3 recordings · 1 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) |
| Replacement, not money back | 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) | 3 recordings · 2 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) |
| Two late parcels, no target | 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) | 3 recordings · 2 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) |
| Duplicate charge, explicit target | 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) | 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews) |
1. Small explicit refund
The cup in order ORD-730 is broken. It cost GBP 12. Please give me my money back.
Expected JSON fixed before execution
{
"order_id": "ORD-730",
"issue": "damaged_item",
"refund_requested": true,
"amount": 12,
"currency": "GBP"
}Variant A · 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
```json
{
"order_id": "ORD-730",
"issue": "cup",
"refund_requested": "broken",
"amount": "GBP 12",
"currency": "GBP"
}
```Keeps the order, GBP and 12 in text, but issue cup is not the damaged_item label and refund_requested broken does not encode the request. Fences and wrong value types violate the output contract. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
```json
{
"order_id": "ORD-730",
"issue": "The cup in order ORD-730 is broken.",
"refund_requested": "Please give me my money back.",
"amount": "GBP 12",
"currency": "GBP"
}
```Order and monetary evidence remain as text, but whole source sentences replace the required issue enum and refund boolean. Fenced output and string amount are not usable typed JSON. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 3 · seed 303 · fail · reject
```json
{
"order_id": "ORD-730",
"issue": "broken",
"refund_requested": "yes",
"amount": "GBP 12",
"currency": "GBP"
}
```Retains order and GBP 12 as text, but broken and yes are not the issue enum and refund boolean. Fences and string amount fail the required output types. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Variant B · 3 recordings · 1 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
{
"order_id": "ORD-730",
"issue": "damaged_item",
"refund_requested": true,
"amount": null,
"currency": null
}Drops explicit GBP 12 to null amount and currency, despite correct order, issue and refund intent. Raw JSON is correctly shaped but not faithful. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
Round 2 · seed 202 · fail · reject
{
"order_id": "ORD-730",
"issue": "damaged_item",
"refund_requested": true,
"amount": null,
"currency": null
}Exactly repeats the prior raw response and its dropped GBP 12. Repetition is not correctness. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
Round 3 · seed 303 · fail · reject
{
"order_id": "ORD-730",
"issue": "damaged_item",
"refund_requested": true,
"amount": null,
"currency": null
}Third identical response, third omission of explicit GBP 12. Stable text is not a successful extraction. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
2. Replacement, not money back
Order ORD-844 arrived cracked. I paid USD 18. Send a replacement please; I am not asking for a refund.
Expected JSON fixed before execution
{
"order_id": "ORD-844",
"issue": "damaged_item",
"refund_requested": false,
"amount": 18,
"currency": "USD"
}Variant A · 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
```json
{
"order_id": "ORD-844",
"issue": "cracked",
"refund_requested": false,
"amount": "18",
"currency": "USD"
}
```Keeps the order, 18 and USD, and false refund intent. Issue cracked is not the required category. Fences and string amount violate the contract. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
```json
{
"order_id": "ORD-844",
"issue": "Cracked",
"refund_requested": "Not asked for a refund",
"amount": "USD 18",
"currency": "USD"
}
```Keeps order and monetary evidence but uses Cracked and Not asked for a refund instead of enum and boolean. Fences and string amount fail the output contract. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 3 · seed 303 · fail · reject
```json
{
"order_id": "ORD-844",
"issue": "cracked",
"refund_requested": "not asked for",
"amount": "USD 18",
"currency": "USD"
}
```Keeps order and USD 18 as text, but cracked and not asked for are not the enum/boolean. Fences and string amount remain. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Variant B · 3 recordings · 2 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
```json
{
"order_id": "ORD-844",
"issue": "damaged_item",
"refund_requested": false,
"amount": null,
"currency": null
}
```Drops explicit USD 18. Correct damage/no-refund policy, but also wraps JSON in fences. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
{
"order_id": "ORD-844",
"issue": "damaged_item",
"refund_requested": false,
"amount": null,
"currency": null
}Now raw JSON rather than fences, but again loses USD 18. Correct damage/no-refund classification. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
Round 3 · seed 303 · fail · reject
{
"order_id": "ORD-844",
"issue": "damaged_item",
"refund_requested": false,
"amount": null,
"currency": null
}Repeats the second response exactly: raw JSON, correct intent/category, missing USD 18. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
3. Two late parcels, no target
Orders ORD-601 and ORD-602 are both late. When will they arrive?
Expected JSON fixed before execution
{
"order_id": null,
"issue": "late_delivery",
"refund_requested": false,
"amount": null,
"currency": null
}Variant A · 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
To extract order_id, issue, refund_requested, amount, and currency from the customer message, we need to break down the information provided. However, the message does not explicitly mention order_id, issue, refund_requested, amount, or currency. Therefore, we can't extract those specific values from the message as provided. However, we can still extract some general information: - Orders ORD-601 and ORD-602 are both late - When will they arrive? (This information is not provided in the message.) Based on the information given, we can only extract the following: - Order ID: ORD-601 and ORD-602 - Issue: None (This information is not provided in the message) - Refund Requested: None (This information
Truncated explanatory prose denies extractable information and loses the issue policy; no usable raw JSON. Keep the length-limited output rather than completing it by hand. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
```json
{
"order_id": "ORD-601",
"issue": "Late",
"refund_requested": "Yes",
"amount": "0.00",
"currency": "USD"
},
{
"order_id": "ORD-602",
"issue": "Late",
"refund_requested": "Yes",
"amount": "0.00",
"currency": "USD"
}
```Invents refund requests, 0.00 and USD for both orders. Gives two comma-separated objects instead of the single conservative-target object. Fences and wrong types compound the source errors. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 3 · seed 303 · fail · reject
```json
{
"order_id": "ORD-601",
"issue": "late",
"refund_requested": "late",
"amount": "late",
"currency": "late"
}
```
This JSON format is used to represent a customer message in a standardized format, which is commonly used in many applications for communication and data exchange. In this case, the message contains information about two orders:
- `order_id`: ORD-601
- `issue`: late
- `refund_requested`: late
- `amount`: late
- `currency`: late
These details are extracted directly from the customer message in a clear and concise manner, which is suitable for use in various applications where JSON is required for data exchange or processing.Selects ORD-601 and writes late into amount/currency/refund fields, then adds explanatory prose claiming direct extraction. This is neither grounded nor usable output. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Variant B · 3 recordings · 2 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
{
"order_id": "ORD-601",
"issue": "late_delivery",
"refund_requested": false,
"amount": null,
"currency": null
}Selects ORD-601 although both orders are late with no unique target. Correct issue and no-refund classification; raw JSON alone does not make the guessed target valid. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["order_id"]: expected null, found string
Round 2 · seed 202 · fail · reject
{
"order_id": "ORD-601",
"issue": "late_delivery",
"refund_requested": false,
"amount": null,
"currency": null
}Exactly repeats ORD-601 as the guessed target. Correct late/no-refund classification; wrong target remains wrong despite identical text. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["order_id"]: expected null, found string
Round 3 · seed 303 · fail · reject
{
"order_id": "ORD-601",
"issue": "late_delivery",
"refund_requested": true,
"amount": null,
"currency": null
}Still guesses ORD-601, now also invents a refund request. Shape passes while both target and intent fail. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["refund_requested"]: expected false, found true
$["order_id"]: expected null, found string
4. Duplicate charge, explicit target
For order ORD-915 I was charged EUR 6.50 twice. Please investigate the duplicate charge.
Expected JSON fixed before execution
{
"order_id": "ORD-915",
"issue": "duplicate_charge",
"refund_requested": false,
"amount": 6.5,
"currency": "EUR"
}Variant A · 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
```json
{
"order_id": "ORD-915",
"issue": "Duplicate charge",
"refund_requested": "Duplicate charge",
"amount": "EUR 6.50",
"currency": "EUR"
}
```Keeps order, EUR and 6.50 in text, but Duplicate charge / duplicate-charge wording in issue and refund_requested is not the enum/boolean contract. Fences and string amount fail structure/types. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
```json
{
"order_id": "ORD-915",
"issue": "charge",
"refund_requested": "duplicate",
"amount": 6.5,
"currency": "EUR"
}
```Retains order, numeric 6.5 and EUR, but issue charge and refund_requested duplicate are not the required enum/boolean. Fences remain. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 3 · seed 303 · fail · reject
```json
{
"order_id": "ORD-915",
"issue": "Duplicate charge",
"refund_requested": "Yes",
"amount": "EUR 6.50",
"currency": "EUR"
}
```Keeps order and EUR 6.50 as text but invents refund intent Yes and uses a non-enum issue label. Fences and wrong types remain. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Variant B · 3 recordings · 3 distinct exact response texts · checks: 0 pass / 3 fail / 0 no checks · 0/3 accepted (3/3 current reviews)
Round 1 · seed 101 · fail · reject
```json
{
"order_id": "ORD-915",
"issue": "duplicate_charge",
"refund_requested": true,
"amount": null,
"currency": "EUR"
}
```Drops 6.50 and invents refund intent from a request to investigate; currency EUR remains. Fences violate raw-JSON requirement. Same-agent review of an actual local-model output, not independent assessment.
- FAIL — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
Cannot compare fields until the response is valid JSON.
Round 2 · seed 202 · fail · reject
{
"order_id": "ORD-915",
"issue": "duplicate_charge",
"refund_requested": false,
"amount": null,
"currency": null
}Correctly declines to invent refund intent this time, but drops both EUR and 6.50. Valid raw JSON is not full source fidelity. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
Round 3 · seed 303 · fail · reject
{
"order_id": "ORD-915",
"issue": "duplicate_charge",
"refund_requested": true,
"amount": null,
"currency": null
}Again invents refund intent and drops amount/currency. It is raw JSON, not a faithful record. Same-agent review of an actual local-model output, not independent assessment.
- PASS — Valid JSON only (no Markdown fences or commentary)
- FAIL — Matches expected JSON exactly (object key order ignored)
$["currency"]: expected string, found null
$["amount"]: expected number, found null
$["refund_requested"]: expected false, found true
Keep this tiny-model configuration on hold for unattended support extraction. All three recordings fail the full expected contract in every case/variant group. Several B responses repeat exactly while omitting values or guessing targets; others change fences or refund intent. Do not retry or tune these cases until a pleasing answer appears. This is a four-case diagnostic exercise, not a representative quality estimate or a comparison to a stronger model.
What this changes—and does not
The latest-only matrix cannot tell you that the first replacement response was fenced while the next two were not, or that all three explicit-refund responses were byte-identical and still dropped GBP 12. The repeated view keeps these facts visible. In this exercise every full check fails; that is evidence to stop trusting this configuration for unattended extraction, not to keep sampling until one answer looks good.
The small model is a convenient local integration test, not a recommendation or proxy for other models. A does not spell out B’s exact contract, so this is not an isolated single-variable prompt experiment. No paid API was used. Stronger models, alternative task designs, independently reviewed cases and real recipient usefulness remain untested. We did not retune the prompts, repair outputs, discard the length-limited answer or run extra requests to create a success.
Take the evidence
- Four-case unrun project and original blank worksheet
- Actual checkpoints: round 1 / seed 101, round 2 / seed 202, round 3 / seed 303
- Practice project: all 24 actual recordings, no reviews
- Original project with all 24 recordings and agent reviews
- All repeated-recording evidence in Markdown (not re-importable)
- Execution design, model/server provenance, hashes, usage and resource observations
The generated project uses one disclosed artifact recording/review date, not per-request completion times; the original worksheets retain those timings. Reviews are same-agent judgments, not human recipient evaluation. The fictional inputs, prompts, expected records and agent-authored guide may be reused and adapted, including commercially, without attribution or warranty. Model/server binaries and weights are not redistributed here; upstream licenses and sources are in the execution metadata. No license claim about provider-generated output overrides applicable rights or laws.
Previous six-call integration example · Compare a changed prompt to a baseline · Return to the Test Bench · Privacy and local storage