Skip to main content

Add a prompt

A prompt is more than text: it's a definition with a fixed output contract, a parser that enforces it, a deterministic fallback, metering, auditing and labelling. This walk-through adds a hypothetical role-brief prompt that writes a short brief for one role from aggregates.

1. Define it in core​

packages/core/src/prompts.ts:

  1. Add the key to PROMPT_KEYS:

    export const PROMPT_KEYS = ["classifier", "exec-summary", "lifecycle", "ask", "role-brief"] as const;

  2. Add a PromptDefinition to PROMPT_DEFINITIONS: label (shown in the Prompt Lab), purpose, output (the contract, in words, that the parser enforces) and variables with required flags. Include citationSourceId for any narrative, so it can only cite the dataset.

  3. Add the shipped text to BUILTIN_PROMPTS: system, template, model: null, a temperature and maxOutputTokens. Follow the existing prompts: JSON only, British English, "we/our", no new figures, the citation envelope.

  4. Extend packages/core/test/prompts.test.ts: the built-in prompt validates, and required variables are enforced.

TypeScript then flags every Record<PromptKey, …> that needs the new key.

2. Allow it in the database​

prompt_versions.prompt_key has a CHECK constraint listing the keys. Add a migration (see Add a migration) that replaces it:

-- 0003 — The role-brief prompt.
ALTER TABLE prompt_versions DROP CONSTRAINT prompt_versions_prompt_key_check;
ALTER TABLE prompt_versions ADD CONSTRAINT prompt_versions_prompt_key_check
  CHECK (prompt_key IN ('classifier', 'exec-summary', 'lifecycle', 'ask', 'role-brief'));

Check the constraint's actual name first (\d prompt_versions); Postgres names unnamed column checks <table>_<column>_check.

seedBuiltinPrompts inserts version 1 for every key in PROMPT_KEYS with ON CONFLICT DO NOTHING, so after the migration, npm run db:migrate (or starting the dev server) seeds the new prompt without touching the others.

3. Call it from a service​

In apps/api/src/services:

  1. A …Variables(ctx) function that builds the prompt's variables from anonymised inputs only (never rows, IDs, names or emails).
  2. A parser (like parseNarrativeEnvelope) that accepts only responses meeting the contract, validates citations with validCitations against an allow-list, and returns null otherwise.
  3. The feature function: take an Llm | null ({ client, prompt }), render with renderPrompt(prompt.system, vars) and renderPrompt(prompt.template, vars), call client.generateJson({ system, user, temperature, maxOutputTokens, model: prompt.model, hints }), parse, and fall back to a deterministic result when there's no model, the call fails or the parse is rejected. Return usage for metering.

Get the prompt with activePrompt(db, "role-brief") (the service connection) outside any user transaction.

4. Teach the development stand-in​

Add a variant to the AiHints union in services/ai.ts carrying the structured inputs, and a branch in ai-fake.ts that answers from them with an obviously synthetic, valid response. This keeps local development and the e2e suite working however the prompt text is edited.

5. The route​

Add or extend a route (see Add an API route) that:

  • checks access (and write access if it generates cached output);
  • checks aiUnavailableReason or assertAiAvailable;
  • records usage with recordAiUsage under a new feature name;
  • audits ai_generated with the kind and promptVersion;
  • caches in dataset_ai with prompt_version if the output should persist (it's cleared on re-score).

6. The Prompt Lab​

apps/api/src/routes/prompts.ts, POST /:key/test, builds variables and a judge per key. Add a branch for the new key that builds the same variables as production (from datasetFor for dataset-based prompts) and judges the response with the same parser, returning accepted, a summary and any parsed output (markdown, classifications or lifecycle; extend PromptTestResult in core if a new shape is needed).

In apps/web/src/pages/admin/PromptLabTab.tsx, the test panel chooses its inputs by key (needsDataset, the question field). Add any new inputs, and render new outcome shapes in TestResult.

7. The UI​

Frame the output with AiSurface (the AI pill, confidence, Review before sharing, citations), hide or disable the control when aiEnabled is false, and show the fallback's label when AI wasn't used.

8. Tests and docs​

  • API tests with FakeAi: accepted, rejected (fallback) and budget-exhausted paths, access, audit and metering.
  • prompts.test.ts in the API: the new prompt is seeded, can be drafted, tested and published.
  • Update the user guide's Prompt Lab table and AI usage features, and Prompts here.