Phase 07 · Retry

The Archivist exercises two distinct retry shapes:

1. Per-call retry — every scout and the LLM ranker wrap their external calls in RetryPolicy.run, so transient failures (network errors, malformed LLM JSON) are automatically retried with exponential backoff before the node reports its output.
2. DAG-level retry loop — validateResponse routes back to compose-response when the draft fails the quality check, bounded by state.attempts.compose so the loop terminates instead of spinning.

Neither shape throws. The dispatcher always sees a named output.

Flow

Code

Per-call retry: scouts

The #scout-retry region shows the scoutRetry policy used by all four scouts — exponential backoff, 2 max attempts, signal-aware:

const scoutRetry = new RetryPolicy({
  "maxAttempts": 2,
  "strategy":    BackoffStrategy.EXPONENTIAL,
  "baseDelay":   400,
});

Per-call retry: LLM ranking

The #rank-retry region shows the rankRetry policy used by rankCandidates — same shape, wrapping the LLM rank call so schema-violation responses get a second chance:

/**
 * RetryPolicy for transient LLM ranking failures.
 * Schema-violation or network errors are retried up to 2 times before
 * the catch block logs and routes forward with unscored candidates.
 */
const rankRetry = new RetryPolicy({
  "maxAttempts": 2,
  "strategy":    BackoffStrategy.EXPONENTIAL,
  "baseDelay":   400,
});

DAG-level retry loop

The complete ComposeRetryLoopDAG — a bounded compose → validate → retry loop built from plain .node() routes:

/**
 * ComposeRetryLoopDAG — reusable compose / validate / retry loop.
 *
 * Internal flow:
 *
 *   crl-compose-response
 *     └─ drafted ──► crl-validate-response
 *          ├─ approved  ──► END (success) ─► parent: respond-to-visitor
 *          ├─ retry     ──► crl-compose-response   (bounded by state.attempts.compose)
 *          └─ exhausted ──► END (success) ─► parent: respond-to-visitor
 *
 * Outputs:
 *   success — draft composed (approved or best-effort); parent routes to
 *             the shared respond-to-visitor terminal.
 *   error   — child-state errors accumulated (propagated by executeDeepDAG)
 *
 * Fan-in policy: this deep-DAG does NOT contain respondToVisitor. It is a
 * pure compose/validate unit that produces state.draft and exits. The
 * single shared respond-to-visitor placement lives at the parent DAG level
 * so that every converging branch strikes exactly one terminal node per run.
 *
 * Molecular import pattern:
 *   import { ComposeRetryLoopDAG, registerComposeRetryLoopNodes } from './deepdags/ComposeRetryLoopDAG.ts';
 *   registerComposeRetryLoopNodes(dispatcher);
 *   dispatcher.registerDAG(ComposeRetryLoopDAG);
 *
 * The deep-DAG operates on the parent's state directly (no stateMapping
 * needed) — it reads `state.shortlist` / `state.intent` / `state.priorContext`
 * and writes `state.draft` / `state.approved`, which the parent DAG already
 * manages. Every intent branch funnels through this one composed loop rather
 * than each branch owning its own compose→validate chain.
 */

import type { ArchivistState }    from '../ArchivistState.ts';
import { composeResponse, validateResponse } from '../nodes/composeResponse.ts';
import type { ArchivistServices } from '../services.ts';

import type { Dagonizer } from '@noocodex/dagonizer';
import { DAGBuilder } from '@noocodex/dagonizer/builder';
import type { DAG } from '@noocodex/dagonizer/entities';


/**
 * The `compose-retry-loop` DAG — one packaged compose/validate unit that every
 * intent branch references via `.deepDAG('compose-loop', 'compose-retry-loop', routes)`.
 *
 * Exits with `success` when the draft is approved or attempts are exhausted.
 * The parent DAG routes `compose-loop → success → respond-to-visitor` so
 * exactly ONE respond-to-visitor fires per run regardless of how many branches
 * converge into this deep-DAG.
 */
export const ComposeRetryLoopDAG: DAG = new DAGBuilder('compose-retry-loop', '1.1')

  // ── 1. compose-response ──────────────────────────────────────────────────
  // LLM call wrapped with RetryPolicy for transient failures. Writes
  // state.draft. Intent-specific compose methods dispatched inside the node
  // via state.intent switch.
  .node('crl-compose-response', composeResponse, {
    'drafted': 'crl-validate-response',
  })

  // ── 2. validate-response ─────────────────────────────────────────────────
  // Quality gate: length, citations, tone. On 'retry', routes back to
  // compose (bounded by MAX_COMPOSE_ATTEMPTS on state.attempts.compose).
  // 'approved' and 'exhausted' both exit the deep-DAG cleanly (null terminal)
  // so the parent receives output 'success' and routes to respond-to-visitor.
  .node('crl-validate-response', validateResponse, {
    'approved':  null,
    'retry':     'crl-compose-response',
    'exhausted': null,
  })

  .build();

/**
 * Register all nodes used by `ComposeRetryLoopDAG` onto a dispatcher.
 *
 * Call this before `dispatcher.registerDAG(ComposeRetryLoopDAG)`. Accepts
 * any `Dagonizer`-compatible dispatcher to allow consumers to use their
 * own subclass while still pulling in the molecular node set.
 *
 * @example
 * ```ts
 * registerComposeRetryLoopNodes(dispatcher);
 * dispatcher.registerDAG(ComposeRetryLoopDAG);
 * ```
 */
export function registerComposeRetryLoopNodes(
  dispatcher: Dagonizer<ArchivistState, ArchivistServices>,
): void {
  for (const node of [
    composeResponse,
    validateResponse,
  ]) {
    dispatcher.registerNode(node);
  }
}

What it demonstrates

• RetryPolicy.run(task, signal) — composable per-call retry with EXPONENTIAL / LINEAR / CONSTANT / DECORRELATED_JITTER backoff. The second argument is context.signal; the policy aborts mid-backoff when the signal fires (see Phase 06).
• Bounded loop modeled in the DAG itself — validateResponse routes 'retry' back to 'crl-compose-response'. The bound is tracked on state.attempts.compose inside the node — no special loop placement type.
• Best-effort fallback — 'exhausted' and 'approved' both route to crl-respond-to-visitor. The visitor always gets a response; the dispatcher never throws on exhaustion.
• Ranking is best-effort too — if rankRetry exhausts without a valid score, the catch block routes 'ranked' with zero-scored candidates so mergeCandidates can still soft-gate.

See this in action in the Archivist live demo.