Skip to main content

Sentiment & topic coding

VerbatimService (api/src/domains/verbatim.ts) runs two distinct AI passes over free-text answers: sentiment analysis (analyze) and topic/theme classification (classify, reached via autoCodeResponse / autoCodeSurvey). Both are idempotent and both share a structural safety guard.

analyze — sentiment​

async analyze(ctx, surveyId, max = 50): Promise<{ scored: number }>
  1. Is gated by the survey's sentiment toggle — analyze returns immediately if the per-survey targeting (targeting(ctx, surveyId).sentiment) is off.
  2. Pulls up to max un-scored verbatims for the survey (r.verbatim.unscored).
  3. Builds a single prompt asking the model to return, for each verbatim, a sentiment label (positive / neutral / negative) and a numeric score from −100 (strongly negative) through 0 to +100 (strongly positive) — numbered in input order, each verbatim whitespace-collapsed and clamped to 500 chars.
  4. Calls ai.completeJson with a Zod schema { results: z.array(z.object({ sentiment: z.enum(SENTIMENTS), score: z.number().min(-100).max(100) })) }.
  5. Writes the label and the clamped intensity back via setSentiment — the label is kept for back-compat, the score feeds the numeric sentiment aggregates.

Because it only touches un-scored answers, it is idempotent — safe to re-run as new responses arrive, and safe against queue redelivery.

Per-response rollup​

rollupResponse collapses a response's per-answer sentiments into one overall label: majority positive vs negative, ties → neutral, no scored verbatims → null. The ingestion pipeline calls it right after analyze, so a captured response carries an overall sentiment the moment its verbatims are scored. (The demo generator in scheduled.ts mirrors this same rollup so the demo shows sentiment without a live queue.)

classify — topics & themes​

The taxonomy is two levels: themes (groups) contain topics (the codes actually applied). Topics carry an optional analyst description that guides the classifier on intent, not just the label string. Both are tenant-scoped and configured in the admin Topics page.

autoCodeResponse(ctx, responseId) codes one response's un-coded verbatims (called on capture); autoCodeSurvey(ctx, surveyId, max) is the on-demand backfill counterpart. Both delegate to the private classify:

  1. Load the tenant's codes (listCodes). No-op if no topics are configured or nothing is un-coded.
  2. Build a %topics% block: topics grouped under their theme, each line - "label": description.
  3. Build a %verbatims% block: the numbered, whitespace-collapsed, 500-char-clamped list.
  4. Resolve the prompt — the tenant's active verbatim_coding prompt if set, else the built-in default — and substitute the two tags (see Prompts).
  5. Call ai.completeJson with schema { results: z.array(z.array(z.string())) } (one label array per verbatim).
  6. Assign matched codes. Only labels that exist in the taxonomy are kept — hallucinated or duplicate labels are ignored (matched case-insensitively against byLabel). Assignment de-dupes, so re-runs are idempotent.

The length-mismatch guard​

Both passes map the model's results positionally onto the input list. If the model returns a different number of results, the alignment is broken and every label could land on the wrong verbatim. Rather than mis-score, both bail and write nothing:

// analyze
if (results.length !== unscored.length) return { scored: 0 };
// classify
if (results.length !== verbatims.length) return { coded: 0 };

This is safe precisely because both passes are idempotent — a clean re-run is cheap, so dropping a misaligned batch costs nothing but a retry.

Exports​

exportCodedCsv emits every coded verbatim for a survey (text + sentiment + applied code labels, profanity-censored per account settings). The CSV escaper quotes any cell containing ", ,, or a newline.