Skip to main content

Capture pipeline

Every channel funnels respondent answers into the same idempotent capture into response.transaction. There are three entry points, all converging on the repository's capture / saveProgress methods (packages/adapters/src/repositories/responses.ts):

EntryRouteRepository call
Public respondent submitPOST /public/surveys/:id/responsescapturePubliccapture
Public web per-page progressPOST /public/surveys/:id/progresssaveWebProgresssaveProgress
Authenticated/server-sidePOST /v1/surveys/:id/responses (perm responses.edit)ResponseService.capturecapture
SMS/IVR final turn(conversation engine)responses.capture with key chan:<sessionId>

The tenant is always resolved server-side: from the survey id on the public paths, from c.var.tenant on the authenticated path, from the session row on the channel path. Nothing trusts a client-supplied tenant.

Idempotency

NewResponse carries an idempotencyKey. The insert into response.transaction uses ON CONFLICT (client_account_id, idempotency_key) DO UPDATE and returns (xmax = 0) as inserted to distinguish a fresh insert from an idempotent replay. Answers — and side-effects — are written only when inserted is true, so a redelivery or double-submit is safe:

const result = (await rows(tx, sql`insert into response.transaction
(public_id, client_account_id, survey_id, survey_version, channel, state, idempotency_key, custom_data, agent_id)
values (${publicId}, ${tenantId}, ${survey.id}, ${survey.current_version}, ${r.channel}, 'complete', ${r.idempotencyKey}, ${payload}::jsonb, ${agentId})
on conflict (client_account_id, idempotency_key) do update set idempotency_key = excluded.idempotency_key
returning id, public_id, (xmax = 0) as inserted`))[0];

if (result.inserted && r.answers?.length) {
await writeAnswers(tx, tenantId, survey.id, result.id, r.channel, r.answers);
await raiseDetractorAlert(tx, tenantId, survey.id, result.id);
await raisePromoterLead(tx, tenantId, survey.id, result.id);
}
if (result.inserted && agentId) {
await evaluateBadgeRules(tx, tenantId, survey.id, result.id, agentId);
}

DO UPDATE (not DO NOTHING) is deliberate: a concurrent same-key insert blocks on the winner so RETURNING always sees the row, and answers are written exactly once.

Partial → complete transition

The web path uses saveProgress(r, complete), which reuses one idempotency key across all pages of a session. It finds any existing row for the key and:

  • In-flight (complete: false) — upserts the row as in_flight and replaces the answer set with the respondent's latest state.
  • Complete (complete: true) — upserts/updates to complete.
  • Never downgrades a row that is already complete back to in_flight.
  • A replay of an already-complete response never touches answers (isReplayOfComplete), because they may already be enriched/coded and verbatim_coding FKs reference them.
const prevState = existing?.state ?? null;
const newState = complete ? 'complete' : 'in_flight';
const isReplayOfComplete = prevState === 'complete';
// … upsert row to (prevState === 'complete' ? 'complete' : newState) …
if (!isReplayOfComplete) {
await rows(tx, sql`delete from response.answer where transaction_id = ${txnId} and client_account_id = ${tenantId}`);
if (r.answers?.length) await writeAnswers(tx, tenantId, survey.id, txnId, r.channel, r.answers);
}

Finalisation side-effects fire once

Alert, lead, and badge evaluation are finalisation side-effects — they fire exactly once, on the transition into complete, and never on a partial save or a replay:

// saveProgress: only on the in_flight → complete transition
if (complete && !isReplayOfComplete) {
await raiseDetractorAlert(tx, tenantId, survey.id, txnId);
await raisePromoterLead(tx, tenantId, survey.id, txnId);
if (agentId) await evaluateBadgeRules(tx, tenantId, survey.id, txnId, agentId);
}

capture (the single-shot path) inserts directly as complete, so its inserted-gated block fires them once on the fresh insert. Either way: one completion, one set of side-effects. Badge evaluation only runs when the response is attributed to an agent.

Ingestion enqueue

After a response is finalised, the route hands it off for asynchronous enrichment (sentiment scoring, verbatim auto-coding) via enqueueIngestion(c, tenantId, responseId, surveyId) (api/src/ingest.ts). This is best-effort and off the respondent's request path:

  • The enqueue is awaited inside c.executionCtx.waitUntil(...), so it adds no latency.
  • A missing queue binding (local/tests) or a transient publish failure never fails a capture that already committed.
  • Partials are not enqueued — only complete: true on the progress path triggers it.

The consumer (api/src/queue.ts, processIngestion) runs outside any request on a system context pinned to the message's tenant: it scores un-scored verbatims, rolls the response up to one overall sentiment label, and auto-codes topics/themes. It is idempotent under redelivery. See the completion runtime for the respondent-facing side.