Dagonizer
Dagonizer

Example 04: Scatter Scout ​

What It Is ​

Scatter Scout is The Archivist's "send several librarians into the stacks at once" pattern. One visitor question becomes a set of JSON-serializable tool worksets: OpenLibrary keyword search, Google Books, OpenLibrary subject search, Wikipedia enrichment, or whichever registered tool DAGs the plan selects.

The point is not raw parallelism for its own sake. The parent DAG asks for one ranked answer, while the scatter branch lets each source work independently, report its own outcome, and return candidates in a deterministic shape that downstream ranking can trust.

How It Works ​

The scout cluster runs as one ScatterNode over state.bookWorksets. Each clone reads its item-scoped dagIri value as a DAG IRI, resolves that value through a dynamic DagReference with explicit tool DAG candidates, and executes the selected tool DAG as the clone body.

Clone output does not disappear into scatter-local magic. The scatter routes success and error to the first-class book-search-gather placement, whose tool-candidate-merge strategy folds each clone's tool output into parent state.candidates. The any-success reducer decides whether the scatter leaves through success, error, or empty; the gather placement owns the parent-state merge.

Diagrams, Examples, and Outputs ​

DAG registration and diagram ​

The Archivist queries registered book tools from one scatter shape. The BookSearchScatterDAG packages the full scout cluster as a reusable sub-DAG body that the parent embeds from multiple branches.

book-search-scatter

16 placements
DAG JSON-LD registered with the dispatcher
{
  "@context": {
    "@version": 1.1,
    "name": {
      "@id": "https://noocodec.dev/ontology/dag/name"
    },
    "version": {
      "@id": "https://noocodec.dev/ontology/dag/version"
    },
    "entrypoints": {
      "@id": "https://noocodec.dev/ontology/dag/entrypoints",
      "@container": "@index"
    },
    "nodes": {
      "@id": "https://noocodec.dev/ontology/dag/nodes",
      "@container": "@set"
    },
    "outputs": {
      "@id": "https://noocodec.dev/ontology/dag/outputs"
    },
    "node": {
      "@id": "https://noocodec.dev/ontology/dag/node"
    },
    "dag": {
      "@id": "https://noocodec.dev/ontology/dag/dag"
    },
    "body": {
      "@id": "https://noocodec.dev/ontology/dag/body"
    },
    "source": {
      "@id": "https://noocodec.dev/ontology/dag/source"
    },
    "sources": {
      "@id": "https://noocodec.dev/ontology/dag/sources",
      "@container": "@index"
    },
    "itemKey": {
      "@id": "https://noocodec.dev/ontology/dag/itemKey"
    },
    "execution": {
      "@id": "https://noocodec.dev/ontology/dag/execution"
    },
    "concurrency": {
      "@id": "https://noocodec.dev/ontology/dag/concurrency"
    },
    "throttle": {
      "@id": "https://noocodec.dev/ontology/dag/throttle"
    },
    "reservoir": {
      "@id": "https://noocodec.dev/ontology/dag/reservoir"
    },
    "gather": {
      "@id": "https://noocodec.dev/ontology/dag/gather"
    },
    "dagReference": {
      "@id": "https://noocodec.dev/ontology/dag/dagReference",
      "@type": "@id"
    },
    "DagReference": {
      "@id": "https://noocodec.dev/ontology/dag/DagReference"
    },
    "from": {
      "@id": "https://noocodec.dev/ontology/dag/from"
    },
    "path": {
      "@id": "https://noocodec.dev/ontology/dag/path"
    },
    "candidates": {
      "@id": "https://noocodec.dev/ontology/dag/candidates",
      "@container": "@set"
    },
    "candidateDag": {
      "@id": "https://noocodec.dev/ontology/dag/candidateDag",
      "@type": "@id"
    },
    "selectedDag": {
      "@id": "https://noocodec.dev/ontology/dag/selectedDag",
      "@type": "@id"
    },
    "resultField": {
      "@id": "https://noocodec.dev/ontology/dag/resultField"
    },
    "policy": {
      "@id": "https://noocodec.dev/ontology/dag/policy"
    },
    "reducer": {
      "@id": "https://noocodec.dev/ontology/dag/reducer"
    },
    "outcome": {
      "@id": "https://noocodec.dev/ontology/dag/outcome"
    },
    "phase": {
      "@id": "https://noocodec.dev/ontology/dag/phase"
    },
    "stateMapping": {
      "@id": "https://noocodec.dev/ontology/dag/stateMapping"
    },
    "container": {
      "@id": "https://noocodec.dev/ontology/dag/container"
    },
    "DAG": {
      "@id": "https://noocodec.dev/ontology/dag/DAG"
    },
    "Placement": {
      "@id": "https://noocodec.dev/ontology/dag/Placement"
    },
    "SingleNode": {
      "@id": "https://noocodec.dev/ontology/dag/SingleNode"
    },
    "ScatterNode": {
      "@id": "https://noocodec.dev/ontology/dag/ScatterNode"
    },
    "EmbeddedDAGNode": {
      "@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
    },
    "GatherNode": {
      "@id": "https://noocodec.dev/ontology/dag/GatherNode"
    },
    "TerminalNode": {
      "@id": "https://noocodec.dev/ontology/dag/TerminalNode"
    },
    "PhaseNode": {
      "@id": "https://noocodec.dev/ontology/dag/PhaseNode"
    }
  },
  "@id": "urn:noocodec:dag:book-search-scatter",
  "@type": "DAG",
  "name": "book-search-scatter",
  "version": "1.0",
  "entrypoints": {
    "main": "urn:noocodec:dag:book-search-scatter/node/extract-query"
  },
  "nodes": [
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/extract-query",
      "@type": "SingleNode",
      "name": "extract-query",
      "node": "urn:noocodec:node:extract-query",
      "outputs": {
        "success": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
        "retry": "urn:noocodec:dag:book-search-scatter/node/extract-query",
        "salvage": "urn:noocodec:dag:book-search-scatter/node/extract-query-salvage"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/extract-query-salvage",
      "@type": "SingleNode",
      "name": "extract-query-salvage",
      "node": "urn:noocodec:node:extract-query-salvage",
      "outputs": {
        "done": "urn:noocodec:dag:book-search-scatter/node/decide-tools"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
      "@type": "SingleNode",
      "name": "decide-tools",
      "node": "urn:noocodec:node:decide-tools",
      "outputs": {
        "tools": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
        "no-tools": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
        "retry": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
        "salvage": "urn:noocodec:dag:book-search-scatter/node/decide-tools-salvage"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/decide-tools-salvage",
      "@type": "SingleNode",
      "name": "decide-tools-salvage",
      "node": "urn:noocodec:node:decide-tools-salvage",
      "outputs": {
        "done": "urn:noocodec:dag:book-search-scatter/node/recall-candidates"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
      "@type": "SingleNode",
      "name": "recall-candidates",
      "node": "urn:noocodec:node:recall-candidates",
      "outputs": {
        "recalled": "urn:noocodec:dag:book-search-scatter/node/build-book-worksets"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/build-book-worksets",
      "@type": "SingleNode",
      "name": "build-book-worksets",
      "node": "urn:noocodec:node:build-book-worksets",
      "outputs": {
        "ready": "urn:noocodec:dag:book-search-scatter/node/book-search-scatter"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/book-search-scatter",
      "@type": "ScatterNode",
      "name": "book-search-scatter",
      "source": "bookWorksets",
      "body": {
        "dag": {
          "@type": "DagReference",
          "from": "item",
          "path": "dagIri",
          "candidates": [
            "urn:noocodec:tool:web_search_books",
            "urn:noocodec:tool:google_books_search",
            "urn:noocodec:tool:subject_search",
            "urn:noocodec:tool:wikipedia_summary"
          ]
        }
      },
      "outputs": {
        "success": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
        "error": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
        "empty": "urn:noocodec:dag:book-search-scatter/node/rank-candidates"
      },
      "itemKey": "currentItem",
      "reducer": "any-success",
      "execution": {
        "mode": "item",
        "concurrency": 4
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
      "@type": "GatherNode",
      "name": "book-search-gather",
      "sources": {
        "urn:noocodec:dag:book-search-scatter/node/book-search-scatter": {}
      },
      "gather": {
        "strategy": "tool-candidate-merge"
      },
      "outputs": {
        "success": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
        "error": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
        "empty": "urn:noocodec:dag:book-search-scatter/node/rank-candidates"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
      "@type": "SingleNode",
      "name": "rank-candidates",
      "node": "urn:noocodec:node:rank-candidates",
      "outputs": {
        "ranked": "urn:noocodec:dag:book-search-scatter/node/merge-candidates",
        "retry": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
        "salvage": "urn:noocodec:dag:book-search-scatter/node/rank-candidates-salvage"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/rank-candidates-salvage",
      "@type": "SingleNode",
      "name": "rank-candidates-salvage",
      "node": "urn:noocodec:node:rank-candidates-salvage",
      "outputs": {
        "done": "urn:noocodec:dag:book-search-scatter/node/merge-candidates"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/merge-candidates",
      "@type": "SingleNode",
      "name": "merge-candidates",
      "node": "urn:noocodec:node:merge-candidates",
      "outputs": {
        "ranked": "urn:noocodec:dag:book-search-scatter/node/record-findings",
        "empty": "urn:noocodec:dag:book-search-scatter/node/no-results"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/record-findings",
      "@type": "SingleNode",
      "name": "record-findings",
      "node": "urn:noocodec:node:record-findings",
      "outputs": {
        "recorded": "urn:noocodec:dag:book-search-scatter/node/has-citations-gate"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/has-citations-gate",
      "@type": "SingleNode",
      "name": "has-citations-gate",
      "node": "urn:noocodec:node:has-citations-gate",
      "outputs": {
        "pass": "urn:noocodec:dag:book-search-scatter/node/recall-past-visits",
        "fail": "urn:noocodec:dag:book-search-scatter/node/no-results"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/recall-past-visits",
      "@type": "SingleNode",
      "name": "recall-past-visits",
      "node": "urn:noocodec:node:recall-past-visits",
      "outputs": {
        "recalled": "urn:noocodec:dag:book-search-scatter/node/found"
      }
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/found",
      "@type": "TerminalNode",
      "name": "found",
      "outcome": "completed"
    },
    {
      "@id": "urn:noocodec:dag:book-search-scatter/node/no-results",
      "@type": "TerminalNode",
      "name": "no-results",
      "outcome": "failed"
    }
  ]
}
Mermaid generated from the same DAG
Mermaid source
%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
  %% book-search-scatter (v1.0)
  entry_main(["main"])
  entry_main --> urn_noocodec_dag_book-search-scatter/node/extract-query
  urn_noocodec_dag_book-search-scatter/node/extract-query["extract-query"]
  urn_noocodec_dag_book-search-scatter/node/extract-query -->|success| urn_noocodec_dag_book-search-scatter/node/decide-tools
  urn_noocodec_dag_book-search-scatter/node/extract-query -->|retry| urn_noocodec_dag_book-search-scatter/node/extract-query
  urn_noocodec_dag_book-search-scatter/node/extract-query -->|salvage| urn_noocodec_dag_book-search-scatter/node/extract-query-salvage
  urn_noocodec_dag_book-search-scatter/node/extract-query-salvage["extract-query-salvage"]
  urn_noocodec_dag_book-search-scatter/node/extract-query-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/decide-tools
  urn_noocodec_dag_book-search-scatter/node/decide-tools["decide-tools"]
  urn_noocodec_dag_book-search-scatter/node/decide-tools -->|tools| urn_noocodec_dag_book-search-scatter/node/recall-candidates
  urn_noocodec_dag_book-search-scatter/node/decide-tools -->|no-tools| urn_noocodec_dag_book-search-scatter/node/recall-candidates
  urn_noocodec_dag_book-search-scatter/node/decide-tools -->|retry| urn_noocodec_dag_book-search-scatter/node/decide-tools
  urn_noocodec_dag_book-search-scatter/node/decide-tools -->|salvage| urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage
  urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage["decide-tools-salvage"]
  urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/recall-candidates
  urn_noocodec_dag_book-search-scatter/node/recall-candidates["recall-candidates"]
  urn_noocodec_dag_book-search-scatter/node/recall-candidates -->|recalled| urn_noocodec_dag_book-search-scatter/node/build-book-worksets
  urn_noocodec_dag_book-search-scatter/node/build-book-worksets["build-book-worksets"]
  urn_noocodec_dag_book-search-scatter/node/build-book-worksets -->|ready| urn_noocodec_dag_book-search-scatter/node/book-search-scatter
  urn_noocodec_dag_book-search-scatter/node/book-search-scatter[/"book-search-scatter"/]
  urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|success| urn_noocodec_dag_book-search-scatter/node/book-search-gather
  urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|error| urn_noocodec_dag_book-search-scatter/node/book-search-gather
  urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|empty| urn_noocodec_dag_book-search-scatter/node/rank-candidates
  urn_noocodec_dag_book-search-scatter/node/book-search-gather{"book-search-gather"}
  urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|success| urn_noocodec_dag_book-search-scatter/node/rank-candidates
  urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|error| urn_noocodec_dag_book-search-scatter/node/rank-candidates
  urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|empty| urn_noocodec_dag_book-search-scatter/node/rank-candidates
  urn_noocodec_dag_book-search-scatter/node/rank-candidates["rank-candidates"]
  urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|ranked| urn_noocodec_dag_book-search-scatter/node/merge-candidates
  urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|retry| urn_noocodec_dag_book-search-scatter/node/rank-candidates
  urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|salvage| urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage
  urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage["rank-candidates-salvage"]
  urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/merge-candidates
  urn_noocodec_dag_book-search-scatter/node/merge-candidates["merge-candidates"]
  urn_noocodec_dag_book-search-scatter/node/merge-candidates -->|ranked| urn_noocodec_dag_book-search-scatter/node/record-findings
  urn_noocodec_dag_book-search-scatter/node/merge-candidates -->|empty| urn_noocodec_dag_book-search-scatter/node/no-results
  urn_noocodec_dag_book-search-scatter/node/record-findings["record-findings"]
  urn_noocodec_dag_book-search-scatter/node/record-findings -->|recorded| urn_noocodec_dag_book-search-scatter/node/has-citations-gate
  urn_noocodec_dag_book-search-scatter/node/has-citations-gate["has-citations-gate"]
  urn_noocodec_dag_book-search-scatter/node/has-citations-gate -->|pass| urn_noocodec_dag_book-search-scatter/node/recall-past-visits
  urn_noocodec_dag_book-search-scatter/node/has-citations-gate -->|fail| urn_noocodec_dag_book-search-scatter/node/no-results
  urn_noocodec_dag_book-search-scatter/node/recall-past-visits["recall-past-visits"]
  urn_noocodec_dag_book-search-scatter/node/recall-past-visits -->|recalled| urn_noocodec_dag_book-search-scatter/node/found
  urn_noocodec_dag_book-search-scatter/node/found((("found")))
  urn_noocodec_dag_book-search-scatter/node/no-results>"no-results"]

The JSON-LD and Mermaid are generated from the same BookSearchScatterDAG export. The static graph view shows the scatter, the dynamic DAG reference body, the book-search-gather fan-in barrier, retry/salvage edges, and terminal exits rather than a hand-drawn approximation.

Running in a container ​

A scatter placement whose body is a DAG (rather than a single node) can run each clone's sub-DAG in an isolate. Add container: "cpu" to the scatter placement and bind a DagContainerInterface backend at dispatcher construction:

The scatter inbox, first-class gather placement, and outcome reducer are identical in both paths. See Example 12: Worker Containers for a complete runnable example of the container binding, including the worker container role and registry module.

What It Lets You Do ​

Scatter lets an application fan out similar work without hand-writing orchestration loops. Use it when one parent state contains a source collection, each item should run through the same placement or body DAG, and the parent needs one merged result before continuing.

In product terms, this gives you a clean way to ask several services, tools, model prompts, parsers, or data partitions the same question. The DAG owns the fan-out, the gather contract owns the merge surface, and the next node receives normal state instead of a pile of ad hoc promises.

Code Samples ​

The complete BookSearchScatterDAG, the sub-DAG the Archivist places three times for on-topic, author, and similar-search branches:

ts
/**
 * BookSearchScatterDAG: reusable query-extract + tool-registry scatter cluster.
 *
 * Internal flow:
 *
 *   extract-query
 *     └─ success ──► decide-tools
 *   decide-tools
 *     └─ (tools | no-tools) ──► recall-candidates
 *   recall-candidates
 *     └─ recalled ──► build-book-worksets
 *   build-book-worksets
 *     └─ ready ──► book-search-scatter (scatter over bookWorksets, concurrency 4)
 *          body: DagReference(item.dagIri) (resolves declared tool DAG IRI per item)
 *          book-search-gather: tool-candidate-merge (reads clone output via accessor, no cast)
 *          reducer: any-success (routes 'success' if any tool found results)
 *     └─ rank-candidates
 *     └─ merge-candidates
 *          ├─ ranked ──► record-findings
 *          └─ empty  ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
 *     └─ record-findings
 *     └─ has-citations-gate
 *          ├─ pass ──► recall-past-visits ──► END (success)
 *          └─ fail ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
 *
 * Outputs:
 *   success: query extracted, candidates found, ranked, recorded, and recalled
 *   error:   no candidates after merge, or citations gate failed;
 *             signalled by the no-results TerminalNode(failed) placement so
 *             the parent EmbeddedDAGNode routes the parent placement to its
 *             'error' branch
 *
 * Molecular import pattern:
 *   import { bookSearchScatterDAG } from './embedded-dags/BookSearchScatterDAG.ts';
 *   const nodes = ArchivistNodes.build(services);
 *   dispatcher.registerBundle(toolRegistry.bundle<ArchivistServices>());
 *   dispatcher.registerBundle({ nodes: nodes.bookSearchScatterNodes, dags: [bookSearchScatterDAG] });
 *
 * The sub-DAG reads `state.query` directly (no input stateMapping; the field
 * names already align with the parent). Each parent placement supplies an
 * `outputs` stateMapping that copies the fields the sub-DAG writes:
 * `terms`, `toolPlan`, `candidates`, `shortlist`, `priorContext`,
 * `failureCause` back onto the parent state so the downstream compose,
 * group-by-year, and recall steps can read them.
 *
 * Three EmbeddedDAGNode placements in the parent `the-archivist` DAG reference
 * this one definition. One definition, three usages:
 *   on-topic-search:  general web book search
 *   author-search:    author body-of-work search
 *   similar-search:   recommend-similar search
 *
 * Reviews and describe branches are inlined in the parent because they use
 * distinct post-scout steps (rankByRating and pickBestMatch respectively).
 */

import type { ArchivistState } from '../ArchivistState.ts';

import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';

const BOOK_SEARCH_SCATTER_DAG_IRI = 'urn:noocodec:dag:book-search-scatter';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(BOOK_SEARCH_SCATTER_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });

const BOOK_SEARCH_TOOL_DAGS = [
  'urn:noocodec:tool:web_search_books',
  'urn:noocodec:tool:google_books_search',
  'urn:noocodec:tool:subject_search',
  'urn:noocodec:tool:wikipedia_summary',
] as const;

const extractQuery          = new PlaceholderNode<ArchivistState, 'success' | 'retry' | 'salvage'>('urn:noocodec:node:extract-query', ['success', 'retry', 'salvage']);
const extractQuerySalvage   = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:extract-query-salvage', ['done']);
const decideTools           = new PlaceholderNode<ArchivistState, 'tools' | 'no-tools' | 'retry' | 'salvage'>('urn:noocodec:node:decide-tools', ['tools', 'no-tools', 'retry', 'salvage']);
const decideToolsSalvage    = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:decide-tools-salvage', ['done']);
const recallCandidates      = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-candidates', ['recalled']);
const buildBookWorksets     = new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:build-book-worksets', ['ready']);
const rankCandidates        = new PlaceholderNode<ArchivistState, 'ranked' | 'retry' | 'salvage'>('urn:noocodec:node:rank-candidates', ['ranked', 'retry', 'salvage']);
const rankCandidatesSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:rank-candidates-salvage', ['done']);
const mergeCandidates       = new PlaceholderNode<ArchivistState, 'ranked' | 'empty'>('urn:noocodec:node:merge-candidates', ['ranked', 'empty']);
const recordFindings        = new PlaceholderNode<ArchivistState, 'recorded'>('urn:noocodec:node:record-findings', ['recorded']);
const hasCitationsGate      = new PlaceholderNode<ArchivistState, 'pass' | 'fail'>('urn:noocodec:node:has-citations-gate', ['pass', 'fail']);
const recallPastVisits      = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-past-visits', ['recalled']);

export const bookSearchScatterDAG: DAGType = new DAGBuilder(BOOK_SEARCH_SCATTER_DAG_IRI, '1.0', display('book-search-scatter'))

      // ── 1. extract-query ─────────────────────────────────────────────────────
      // LLM parses the raw visitor question into structured search terms.
      // Writes state.terms for the scouts and decide-tools to consume.
      // 'retry' loops back (bounded by the state retry budget); 'salvage' routes to
      // a deterministic recovery node; never a fabricated term list on the node.
      // #region retry-salvage-wiring
      .node(placement('extract-query'), extractQuery, {
        'success': placement('decide-tools'),
        'retry':   placement('extract-query'),          // flow-shape retry loop (self-edge)
        'salvage': placement('extract-query-salvage'),  // recovery route
      }, display('extract-query'))
      .node(placement('extract-query-salvage'), extractQuerySalvage, {
        'done': placement('decide-tools'),              // deterministic recovery rejoins the happy path
      }, display('extract-query-salvage'))
      // #endregion retry-salvage-wiring

      // ── 2. decide-tools ──────────────────────────────────────────────────────
      // LLM decides which external sources to invoke. Both outputs route into
      // recall-candidates so prior memory is loaded before scouts fire.
      // 'retry' loops back (bounded); 'salvage' routes to the minimal-plan node.
      .node(placement('decide-tools'), decideTools, {
        'tools':    placement('recall-candidates'),
        'no-tools': placement('recall-candidates'),
        'retry':    placement('decide-tools'),
        'salvage':  placement('decide-tools-salvage'),
      }, display('decide-tools'))
      .node(placement('decide-tools-salvage'), decideToolsSalvage, {
        'done': placement('recall-candidates'),
      }, display('decide-tools-salvage'))

      // ── 2b. recall-candidates ────────────────────────────────────────────────
      // Pre-loads state.priorCandidates from memory: shortlisted books from prior
      // runs whose visitor query has Jaccard >= 0.35 overlap with the current
      // query. Cap 10. Always routes 'recalled', even when no prior runs match.
      .node(placement('recall-candidates'), recallCandidates, {
        'recalled': placement('build-book-worksets'),
      }, display('recall-candidates'))

      // ── 2c. build-book-worksets ──────────────────────────────────────────────
      // Converts state.toolPlan into a bookWorksets array where each entry
      // carries { dagIri: 'urn:noocodec:tool:<name>', arguments: {...} }. The scatter
      // placement reads dagIri through an item-scoped DagReference to resolve
      // the body DAG at runtime.
      .node(placement('build-book-worksets'), buildBookWorksets, {
        'ready': placement('book-search-scatter'),
      }, display('build-book-worksets'))

      // ── 3. book-search-scatter ───────────────────────────────────────────────
      // Tool-registry scatter: bookWorksets items fan out concurrently. Each item
      // carries its own tool DAG IRI via dagIri; the DagReference
      // resolves the body DAG at runtime from the item. ToolInvokeNode reads the
      // item's arguments field and calls the bound tool. The following GatherNode
      // reads each clone's ToolInvocationState.output (via accessor, no cast)
      // and folds the CandidateType[] into the parent state's candidates.
      // any-success reducer: 'success' → rank-candidates when at least one tool hit;
      // 'error' → rank-candidates to allow graceful empty-candidates handling.
      .scatter(placement('book-search-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
        'success': placement('book-search-gather'),
        'error':   placement('book-search-gather'),
        'empty':   placement('rank-candidates'),
      }, {
        'name': 'book-search-scatter',
        'execution': { 'mode': 'item', 'concurrency': 4 },
        'reducer': 'any-success',
      })
      .gather(placement('book-search-gather'), { [placement('book-search-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
        'success': placement('rank-candidates'),
        'error':   placement('rank-candidates'),
        'empty':   placement('rank-candidates'),
      }, display('book-search-gather'))

      // ── 4. rank-candidates ───────────────────────────────────────────────────
      // LLM-driven relevance scoring. Routes 'ranked' on success (an empty set is
      // still a valid ranking, so merge can soft-gate on zero candidates).
      // 'retry' loops back (bounded); 'salvage' passes candidates through unranked
      // via a dedicated node rather than emitting them as if they were ranked.
      .node(placement('rank-candidates'), rankCandidates, {
        'ranked':  placement('merge-candidates'),
        'retry':   placement('rank-candidates'),
        'salvage': placement('rank-candidates-salvage'),
      }, display('rank-candidates'))
      .node(placement('rank-candidates-salvage'), rankCandidatesSalvage, {
        'done': placement('merge-candidates'),
      }, display('rank-candidates-salvage'))

      // ── 5. merge-candidates ──────────────────────────────────────────────────
      // Cross-source dedupe via CanonicalId, top-5. Routes 'empty' to
      // no-results (TerminalNode(failed)) so the parent EmbeddedDAGNode's
      // terminal outcome routes the parent placement to its 'error' branch.
      .node(placement('merge-candidates'), mergeCandidates, {
        'ranked': placement('record-findings'),
        'empty':  placement('no-results'),
      }, display('merge-candidates'))

      // ── 6. record-findings ───────────────────────────────────────────────────
      // Deterministic RDF write: same input always produces the same triples.
      .node(placement('record-findings'), recordFindings, {
        'recorded': placement('has-citations-gate'),
      }, display('record-findings'))

      // ── 7. has-citations-gate ────────────────────────────────────────────────
      // SPARQL ASK over the per-run state graph. Symbolic fence for the LLM.
      // 'fail' routes to no-results (TerminalNode(failed)) so the parent
      // EmbeddedDAGNode routes the parent placement to 'error'.
      .node(placement('has-citations-gate'), hasCitationsGate, {
        'pass': placement('recall-past-visits'),
        'fail': placement('no-results'),
      }, display('has-citations-gate'))

      // ── 8. recall-past-visits ────────────────────────────────────────────────
      // Injects prior-session context (prior queries + shortlisted titles) into
      // state.priorContext, then routes to the canonical `found` TerminalNode
      // (completed) so the parent EmbeddedDAGNode resolves its 'success' branch.
      .node(placement('recall-past-visits'), recallPastVisits, {
        'recalled': placement('found'),
      }, display('recall-past-visits'))

      // ── 9. Terminal nodes ────────────────────────────────────────────────────
      // Both sub-DAG exits are canonical TerminalNode placements (no bare null
      // routes): `found` (completed) drives the parent EmbeddedDAGNode's 'success'
      // branch; `no-results` (failed) drives its 'error' branch.
      .terminal(placement('found'), { outcome: 'completed', name: 'found' })
      .terminal(placement('no-results'), { outcome: 'failed', name: 'no-results' })

      .build();

Details for Nerds ​

  • ScatterNode with workset source. The source is state.bookWorksets, written by build-book-worksets from the selected tool plan. One clone runs per workset; execution: { mode: 'item', concurrency: 4 } caps how many clones run at once.
  • Heterogeneous fan-out via DagReference. Each workset carries dagIri as a DAG IRI, and the scatter body resolves { dag: { from: 'item', path: 'dagIri', candidates: [...] } } against explicit tool DAG candidates.
  • First-class gather. book-search-gather is a GatherNode placement after the scatter. It uses tool-candidate-merge to fold clone tool outputs into parent state.candidates.
  • any-success outcome reducer. Routes 'success' when at least one clone succeeded; routes 'error' when every clone errored; routes 'empty' when there were no source items.
  • Scout gating via state.toolPlan. decideTools populates the plan; build-book-worksets turns only selected calls into scatter items. Wikipedia enrichment still enters when useful terms exist, so the deep-one context can surface even when the model under-plans.
  • scoutRetry pass-through. Every scout calls scoutRetry.run(() => tool.execute(..., context.signal), context.signal). The signal propagates from the dispatcher through the retry policy: if the parent flow is cancelled, retries abort mid-backoff.
  • bookSearchScatterDAG. The sub-DAG module exports the canonical JSON-LD DAG. The caller registers { nodes: nodes.bookSearchScatterNodes, dags: [bookSearchScatterDAG] } before the parent DAG so every embedded reference resolves.

See this in action in the Archivist live demo.

Watched over by the Order of Dagon.

MIT License, © 2026 Andrew Studnicky