PromptKB

Dataset preparation · local-first

Turn a spreadsheet into a reusable prompt test.

Preparing cases should not mean copying the same fields into a form dozens of times. Bring a CSV or paste spreadsheet cells, map the columns, inspect the cases, and keep your existing prompt variants.

Open the Test BenchDownload 12-case CSVDownload ready-to-use project

The example uses real, already-public notes from this experiment’s timeline, captured through November 3, 2026 at 16:04 UTC. The date/category expectations are agent-authored task labels, not externally reviewed ground truth or model benchmarks. There are no model responses or pre-scored runs in the download.

1. Prepare one source per row

Your header names do not have to match ours. Map an input column, optionally a case-name column, and optionally an expected-JSON column. Other columns are ignored. The example has:

The 12 notes cover five user-facing product changes, six operating events and one owner instruction change. They include failures, recovery and planned maintenance—not just successful launches. The corresponding project compares a broad digest request with an explicit date/category/summary contract. The summary is deliberately not an exact-match field and still needs manual review.

Use UTF-8 CSV up to 1 MB, with up to 50 data rows and 30 columns. Cells containing commas, quotes or line breaks must be quoted; double any quotes inside a quoted cell. Spreadsheet CSV export normally handles this. For pasted spreadsheet cells or TSV files, choose Tab. Blank lines are skipped; inconsistent row lengths or malformed quoting are rejected before any project changes. Line endings normalize to LF.

2. Map, preview, then commit

  1. In the bench, expand Import a case dataset from CSV or a spreadsheet. Choose a file or paste delimited text, choose the separator, then click Read columns.
  2. Check the three column selectors. Common header names are suggested, not a guarantee that the mapping is right. You must map an input column; name and expected JSON can be left unmapped.
  3. For this example, map source_notes to input and expected_record to expected JSON. Uncheck Reject extra object keys so a response can include its summary alongside the checked date/category.
  4. Normally leave Copy the selected case’s current checks off. Enable it only when you intentionally want the same literal phrases, word cap and other checks copied into each new case. Copies are independent, not shared rules. Mapped expected JSON replaces expected values and exact/subset matching but retains an inherited structure contract. Copies are independent; use the case editor to inspect the result.
  5. Choose Append to preserve existing cases and recordings, or Replace all cases to discard that case set after confirmation. Both prompt variants and the project name stay unchanged.
  6. Click Preview cases. Expand rows to inspect the full source and resulting rules. Nothing has changed yet. Click Import previewed cases only when the proposed set is right, then confirm.

A mapped expected-JSON column requires JSON output on every imported row. A blank cell means valid JSON only, not a missing test or an expected empty string. Nonblank cells must contain valid JSON; an invalid expectation blocks the entire import. Select Not mapped if you do not want JSON checks.

For a quick practice run, open the ready-to-use project above first. It supplies the two digest prompts and all 12 cases. Import the CSV with Replace all cases to reproduce the same case set while keeping those prompts. For your own work, start from your own project and use Append unless you really intend to replace its cases.

3. Evaluate the set, not the import

A successful import is not a successful prompt test. Imported cases have no response drafts, recorded results or trial history. Copy the rendered prompts to your chosen model, record responses without repairing them, and review both structured checks and meaning. For a whole set, export an exact-prompt worksheet and import completed responses together. No model is called by the bench.

Example: a note about a stalled monitoring timer should be categorized as operations, but a summary must also preserve that the monitoring gap is not measured request downtime. Passing the date/category check cannot catch every misleading summary. Keep a manual review note with the response.

The bench now supports up to 50 cases. Its whole-set totals include unrun cases explicitly; importing 12 cases must not create 12 passing results. Existing appended cases keep their drafts, reviews and trial history unchanged. Matching names are not merged or updated, so appending the same file twice creates duplicate cases.

4. Keep the whole project, not just the dataset

Download project JSON after preparing your cases. CSV is an input dataset, not a project backup: it does not contain the prompt variants, response history, reviews or complete per-case rules. The ready project and your exported projects use Test Bench format 3. Earlier bench releases that only supported ten cases reject larger projects instead of loading a partial set.

The input file and staging text are read locally and are not saved as a separate document or uploaded. Only the imported cases enter the project and its optional browser storage/downloads. Remember is opt-in; the 10 MB project-file limit does not guarantee browser storage has that much space. Server backups cannot recover browser-local work. Review source notes and exports before sharing.

Case CSV does not import model results: use the separate response worksheet and paired-review workflow for that. Dataset import does not merge cases by ID, edit arbitrary spreadsheet formulas or maintain linked/shared rules. It treats cells as text and never evaluates formulas. Requirements and limits are provisional product choices, not a claim that every task should fit this format.

Bring your case set into the Test Bench → · Start with the smaller support-extraction walkthrough