Prompts

The four AI prompts are data. Administrators edit, test and publish versions in the Prompt Lab (the user guide's Prompt Lab); every AI call uses the newest published version. What a prompt must return is fixed in code: the parsers validate every response, so a badly edited prompt degrades to the rule-based fallback rather than producing unvalidated output.

Definitions

packages/core/src/prompts.ts:

export const PROMPT_KEYS = ["classifier", "exec-summary", "lifecycle", "ask"] as const;

interface PromptTemplate {
  system: string;            // system instruction
  template: string;          // user message with {{variables}}
  model: string | null;      // null: the platform's configured model
  temperature: number;       // 0–2
  maxOutputTokens: number;   // 64–32768
}

interface PromptDefinition {
  key: PromptKey;
  label: string;             // "Title classifier", "Executive summary", "Value-chain mapper", "Ask the data"
  purpose: string;
  output: string;            // the response contract, shown in the Prompt Lab, enforced by the parser
  variables: { name: string; description: string; required: boolean }[];
}

Key	Label	Variables (required in bold)	Parser	Built-in settings
classifier	Title classifier	context, titles	parseClassifierResponse	temperature 0.1, 16,384 tokens
exec-summary	Executive summary	facts, citationSourceId	parseNarrativeEnvelope	0.4, 4,096
lifecycle	Value-chain mapper	sector, roles, citationSourceId	parseLifecycleResponse	0.3, 8,192
ask	Ask the data	facts, question, citationSourceId	parseNarrativeEnvelope	0.2, 4,096

BUILTIN_PROMPTS holds the shipped text, which seeds version 1 of each. The narrative prompts share an envelope instruction: JSON with markdown, confidence and non-empty citations whose only permitted source is {{citationSourceId}}; British English; "we/our", never "you".

Rendering and validation

• promptVariablesIn(text) lists the {{name}} variables a text uses (names start with a letter; spaces inside the braces are allowed).
• renderPrompt(text, vars) fills them verbatim; unknown names become empty.
• validatePrompt(key, template) returns issues (empty means publishable):
  • system and template non-empty and ≤ 20,000 characters;
  • no variables that aren't defined for the key;
  • every required variable appears in the system text or template;
  • model is null or matches [A-Za-z0-9._-]{1,100};
  • temperature in [0, 2]; maxOutputTokens an integer in [64, 32768].

The web editor runs validatePrompt live; the API runs it on save (returning issues) and on publish (refusing with invalid_prompt).

Storage and versions

prompt_versions (migration 0002): (prompt_key, version) primary key, status draft or published, the template fields, notes, authorship. One draft per key (unique partial index). RLS: admins only for everything; inserts must be drafts; updates and deletes only while a draft.

apps/api/src/services/prompts.ts:

Function	Does
seedBuiltinPrompts(db)	Inserts version 1 of every key from BUILTIN_PROMPTS (idempotent). Runs after migrations.
activePrompt(db, key)	The newest published version, on the service connection (AI calls run for non-admins). Throws if not seeded.
listPromptVersions(tx, key)	Newest first, under RLS.
createPromptDraft(tx, userId, key, from?)	Copies the newest published version (or from). 409 if a draft exists.
savePromptDraft(tx, key, partial)	Merges fields into the draft.
publishPromptDraft(tx, userId, key, notes)	Validates, then flips the draft to published.
discardPromptDraft(tx, key)	Deletes the draft.
promptVersionTemplate(tx, key, version | "draft")	For tests.

The prompt version used is recorded on cached AI output (dataset_ai.prompt_version), on shared-cache entries (classification_cache.prompt_version) and in ai_generated audit entries.

The Prompt Lab API

routes/prompts.ts, mounted at /api/admin/prompts, all admin-only:

GET    /                     every prompt: label, purpose, live version, draft
GET    /datasets             up to 200 ready datasets across engagements (for narrative tests)
GET    /:key                 definition + every version
POST   /:key/draft           { from? }
PUT    /:key/draft           partial template + notes → { draft, issues }
POST   /:key/draft/publish   { notes }
DELETE /:key/draft
POST   /:key/test            run or preview a version

Test runs

POST /:key/test takes promptTestSchema:

{ "version": "draft", "titles": ["Senior Claims Handler"], "industry": "Insurance", "region": "UK", "dryRun": false }

or, for narrative prompts, { "version": 3, "datasetId": "…", "question": "…" }.

1. Loads the chosen version (or the draft) and validates it.
2. Builds the variables exactly as production does: classifierVariables for up to 50 titles; execSummaryVariables, lifecycleVariables or askVariables from the dataset's cached summary (the dataset must be ready; admins can read any dataset).
3. Renders the system text and message.
4. If dryRun, or AI is unavailable, returns the rendered prompt only (dryRun: true, with aiUnavailableReason when relevant). Audited prompt_tested with dryRun: true.
5. Otherwise calls the model with the version's settings, meters it as prompt-lab, audits prompt_tested (model and tokens), and judges the response with the production parser: accepted, a summary, and the parsed classifications, lifecycle or markdown. For the value chain, lifecycle.roles counts only roles the model placed and lifecycle.evenlySpread the rest, which production spreads evenly across the phases. The raw response is returned (truncated at 20,000 characters).

A model error is returned as a rejected outcome (The model didn't respond: …), not an HTTP error.

Known UI gap

When AI is unavailable, the endpoint returns dryRun: true with an aiUnavailableReason, but the web client only shows the reason when dryRun is false. The result therefore looks like an ordinary preview, with no explanation of why the model wasn't called.

In the web app

apps/web/src/pages/admin/PromptLabTab.tsx: the prompt list (with live and draft versions), the definition card, the editor (with live validatePrompt), the test panel (draft and live side by side, saving unsaved changes first), results, version history and the publish dialog.

To add a prompt, see Add a prompt.