Example 04B: Scatter Collect
What It Is
Scatter Collect is the "every clone brings something back" half of scatter. The Archivist sends tool worksets into a scatter branch, each clone produces candidate books, and a gather placement folds those candidates into parent state for ranking.
This page narrows in on the gather contract. The scatter can finish clones in any order, but book-search-gather is the graph-visible barrier that decides how clone outputs become parent state.candidates.
How It Works
The scatter source creates one clone per workset. Each clone runs the body DAG selected by item.dagIri, writes its output into clone state, and returns a declared outcome. The tool-candidate-merge gather strategy reads those clone outputs through the accessor and folds candidate arrays into the parent before rank-candidates runs.
That ordering matters in real applications. A result from a slow provider should not jump ahead of a faster provider just because the network happened to answer later; selection logic should compare data, not race conditions.
Diagrams, Examples, and Outputs
DAG registration and diagram
The in-browser owner is The Archivist: its book-search-scatter sub-DAG scatters tool worksets, routes clone outcomes into book-search-gather, and then ranks the collected candidates. This is the live Archivist graph, not a separate miniature.
book-search-scatter
16 placements{
"@context": {
"@version": 1.1,
"name": {
"@id": "https://noocodec.dev/ontology/dag/name"
},
"version": {
"@id": "https://noocodec.dev/ontology/dag/version"
},
"entrypoints": {
"@id": "https://noocodec.dev/ontology/dag/entrypoints",
"@container": "@index"
},
"nodes": {
"@id": "https://noocodec.dev/ontology/dag/nodes",
"@container": "@set"
},
"outputs": {
"@id": "https://noocodec.dev/ontology/dag/outputs"
},
"node": {
"@id": "https://noocodec.dev/ontology/dag/node"
},
"dag": {
"@id": "https://noocodec.dev/ontology/dag/dag"
},
"body": {
"@id": "https://noocodec.dev/ontology/dag/body"
},
"source": {
"@id": "https://noocodec.dev/ontology/dag/source"
},
"sources": {
"@id": "https://noocodec.dev/ontology/dag/sources",
"@container": "@index"
},
"itemKey": {
"@id": "https://noocodec.dev/ontology/dag/itemKey"
},
"execution": {
"@id": "https://noocodec.dev/ontology/dag/execution"
},
"concurrency": {
"@id": "https://noocodec.dev/ontology/dag/concurrency"
},
"throttle": {
"@id": "https://noocodec.dev/ontology/dag/throttle"
},
"reservoir": {
"@id": "https://noocodec.dev/ontology/dag/reservoir"
},
"gather": {
"@id": "https://noocodec.dev/ontology/dag/gather"
},
"dagReference": {
"@id": "https://noocodec.dev/ontology/dag/dagReference",
"@type": "@id"
},
"DagReference": {
"@id": "https://noocodec.dev/ontology/dag/DagReference"
},
"from": {
"@id": "https://noocodec.dev/ontology/dag/from"
},
"path": {
"@id": "https://noocodec.dev/ontology/dag/path"
},
"candidates": {
"@id": "https://noocodec.dev/ontology/dag/candidates",
"@container": "@set"
},
"candidateDag": {
"@id": "https://noocodec.dev/ontology/dag/candidateDag",
"@type": "@id"
},
"selectedDag": {
"@id": "https://noocodec.dev/ontology/dag/selectedDag",
"@type": "@id"
},
"resultField": {
"@id": "https://noocodec.dev/ontology/dag/resultField"
},
"policy": {
"@id": "https://noocodec.dev/ontology/dag/policy"
},
"reducer": {
"@id": "https://noocodec.dev/ontology/dag/reducer"
},
"outcome": {
"@id": "https://noocodec.dev/ontology/dag/outcome"
},
"phase": {
"@id": "https://noocodec.dev/ontology/dag/phase"
},
"stateMapping": {
"@id": "https://noocodec.dev/ontology/dag/stateMapping"
},
"container": {
"@id": "https://noocodec.dev/ontology/dag/container"
},
"DAG": {
"@id": "https://noocodec.dev/ontology/dag/DAG"
},
"Placement": {
"@id": "https://noocodec.dev/ontology/dag/Placement"
},
"SingleNode": {
"@id": "https://noocodec.dev/ontology/dag/SingleNode"
},
"ScatterNode": {
"@id": "https://noocodec.dev/ontology/dag/ScatterNode"
},
"EmbeddedDAGNode": {
"@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
},
"GatherNode": {
"@id": "https://noocodec.dev/ontology/dag/GatherNode"
},
"TerminalNode": {
"@id": "https://noocodec.dev/ontology/dag/TerminalNode"
},
"PhaseNode": {
"@id": "https://noocodec.dev/ontology/dag/PhaseNode"
}
},
"@id": "urn:noocodec:dag:book-search-scatter",
"@type": "DAG",
"name": "book-search-scatter",
"version": "1.0",
"entrypoints": {
"main": "urn:noocodec:dag:book-search-scatter/node/extract-query"
},
"nodes": [
{
"@id": "urn:noocodec:dag:book-search-scatter/node/extract-query",
"@type": "SingleNode",
"name": "extract-query",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
"retry": "urn:noocodec:dag:book-search-scatter/node/extract-query",
"salvage": "urn:noocodec:dag:book-search-scatter/node/extract-query-salvage"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/extract-query-salvage",
"@type": "SingleNode",
"name": "extract-query-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:book-search-scatter/node/decide-tools"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
"@type": "SingleNode",
"name": "decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
"no-tools": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
"retry": "urn:noocodec:dag:book-search-scatter/node/decide-tools",
"salvage": "urn:noocodec:dag:book-search-scatter/node/decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/decide-tools-salvage",
"@type": "SingleNode",
"name": "decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:book-search-scatter/node/recall-candidates"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/recall-candidates",
"@type": "SingleNode",
"name": "recall-candidates",
"node": "urn:noocodec:node:recall-candidates",
"outputs": {
"recalled": "urn:noocodec:dag:book-search-scatter/node/build-book-worksets"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/build-book-worksets",
"@type": "SingleNode",
"name": "build-book-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:book-search-scatter/node/book-search-scatter"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/book-search-scatter",
"@type": "ScatterNode",
"name": "book-search-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
"error": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
"empty": "urn:noocodec:dag:book-search-scatter/node/rank-candidates"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/book-search-gather",
"@type": "GatherNode",
"name": "book-search-gather",
"sources": {
"urn:noocodec:dag:book-search-scatter/node/book-search-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
"error": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
"empty": "urn:noocodec:dag:book-search-scatter/node/rank-candidates"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
"@type": "SingleNode",
"name": "rank-candidates",
"node": "urn:noocodec:node:rank-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:book-search-scatter/node/merge-candidates",
"retry": "urn:noocodec:dag:book-search-scatter/node/rank-candidates",
"salvage": "urn:noocodec:dag:book-search-scatter/node/rank-candidates-salvage"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/rank-candidates-salvage",
"@type": "SingleNode",
"name": "rank-candidates-salvage",
"node": "urn:noocodec:node:rank-candidates-salvage",
"outputs": {
"done": "urn:noocodec:dag:book-search-scatter/node/merge-candidates"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/merge-candidates",
"@type": "SingleNode",
"name": "merge-candidates",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:book-search-scatter/node/record-findings",
"empty": "urn:noocodec:dag:book-search-scatter/node/no-results"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/record-findings",
"@type": "SingleNode",
"name": "record-findings",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:book-search-scatter/node/has-citations-gate"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/has-citations-gate",
"@type": "SingleNode",
"name": "has-citations-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:book-search-scatter/node/recall-past-visits",
"fail": "urn:noocodec:dag:book-search-scatter/node/no-results"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/recall-past-visits",
"@type": "SingleNode",
"name": "recall-past-visits",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:book-search-scatter/node/found"
}
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/found",
"@type": "TerminalNode",
"name": "found",
"outcome": "completed"
},
{
"@id": "urn:noocodec:dag:book-search-scatter/node/no-results",
"@type": "TerminalNode",
"name": "no-results",
"outcome": "failed"
}
]
}Mermaid source
%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
%% book-search-scatter (v1.0)
entry_main(["main"])
entry_main --> urn_noocodec_dag_book-search-scatter/node/extract-query
urn_noocodec_dag_book-search-scatter/node/extract-query["extract-query"]
urn_noocodec_dag_book-search-scatter/node/extract-query -->|success| urn_noocodec_dag_book-search-scatter/node/decide-tools
urn_noocodec_dag_book-search-scatter/node/extract-query -->|retry| urn_noocodec_dag_book-search-scatter/node/extract-query
urn_noocodec_dag_book-search-scatter/node/extract-query -->|salvage| urn_noocodec_dag_book-search-scatter/node/extract-query-salvage
urn_noocodec_dag_book-search-scatter/node/extract-query-salvage["extract-query-salvage"]
urn_noocodec_dag_book-search-scatter/node/extract-query-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/decide-tools
urn_noocodec_dag_book-search-scatter/node/decide-tools["decide-tools"]
urn_noocodec_dag_book-search-scatter/node/decide-tools -->|tools| urn_noocodec_dag_book-search-scatter/node/recall-candidates
urn_noocodec_dag_book-search-scatter/node/decide-tools -->|no-tools| urn_noocodec_dag_book-search-scatter/node/recall-candidates
urn_noocodec_dag_book-search-scatter/node/decide-tools -->|retry| urn_noocodec_dag_book-search-scatter/node/decide-tools
urn_noocodec_dag_book-search-scatter/node/decide-tools -->|salvage| urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage
urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage["decide-tools-salvage"]
urn_noocodec_dag_book-search-scatter/node/decide-tools-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/recall-candidates
urn_noocodec_dag_book-search-scatter/node/recall-candidates["recall-candidates"]
urn_noocodec_dag_book-search-scatter/node/recall-candidates -->|recalled| urn_noocodec_dag_book-search-scatter/node/build-book-worksets
urn_noocodec_dag_book-search-scatter/node/build-book-worksets["build-book-worksets"]
urn_noocodec_dag_book-search-scatter/node/build-book-worksets -->|ready| urn_noocodec_dag_book-search-scatter/node/book-search-scatter
urn_noocodec_dag_book-search-scatter/node/book-search-scatter[/"book-search-scatter"/]
urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|success| urn_noocodec_dag_book-search-scatter/node/book-search-gather
urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|error| urn_noocodec_dag_book-search-scatter/node/book-search-gather
urn_noocodec_dag_book-search-scatter/node/book-search-scatter -->|empty| urn_noocodec_dag_book-search-scatter/node/rank-candidates
urn_noocodec_dag_book-search-scatter/node/book-search-gather{"book-search-gather"}
urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|success| urn_noocodec_dag_book-search-scatter/node/rank-candidates
urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|error| urn_noocodec_dag_book-search-scatter/node/rank-candidates
urn_noocodec_dag_book-search-scatter/node/book-search-gather -->|empty| urn_noocodec_dag_book-search-scatter/node/rank-candidates
urn_noocodec_dag_book-search-scatter/node/rank-candidates["rank-candidates"]
urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|ranked| urn_noocodec_dag_book-search-scatter/node/merge-candidates
urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|retry| urn_noocodec_dag_book-search-scatter/node/rank-candidates
urn_noocodec_dag_book-search-scatter/node/rank-candidates -->|salvage| urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage
urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage["rank-candidates-salvage"]
urn_noocodec_dag_book-search-scatter/node/rank-candidates-salvage -->|done| urn_noocodec_dag_book-search-scatter/node/merge-candidates
urn_noocodec_dag_book-search-scatter/node/merge-candidates["merge-candidates"]
urn_noocodec_dag_book-search-scatter/node/merge-candidates -->|ranked| urn_noocodec_dag_book-search-scatter/node/record-findings
urn_noocodec_dag_book-search-scatter/node/merge-candidates -->|empty| urn_noocodec_dag_book-search-scatter/node/no-results
urn_noocodec_dag_book-search-scatter/node/record-findings["record-findings"]
urn_noocodec_dag_book-search-scatter/node/record-findings -->|recorded| urn_noocodec_dag_book-search-scatter/node/has-citations-gate
urn_noocodec_dag_book-search-scatter/node/has-citations-gate["has-citations-gate"]
urn_noocodec_dag_book-search-scatter/node/has-citations-gate -->|pass| urn_noocodec_dag_book-search-scatter/node/recall-past-visits
urn_noocodec_dag_book-search-scatter/node/has-citations-gate -->|fail| urn_noocodec_dag_book-search-scatter/node/no-results
urn_noocodec_dag_book-search-scatter/node/recall-past-visits["recall-past-visits"]
urn_noocodec_dag_book-search-scatter/node/recall-past-visits -->|recalled| urn_noocodec_dag_book-search-scatter/node/found
urn_noocodec_dag_book-search-scatter/node/found((("found")))
urn_noocodec_dag_book-search-scatter/node/no-results>"no-results"]A ScatterNode runs a body DAG over every tool workset; each clone produces candidates through a registered tool DAG, and the following GatherNode folds those candidates into the parent clone before ranking.
The generate-and-select pattern is common in LLM pipelines: scatter over a set of prompts or queries, each clone generates one candidate, and the parent picks from the collected array.
Run
bash
npx tsx examples/the-archivist/runArchivist.tsWhat It Lets You Do
Scatter collect lets applications run many clones concurrently and then continue with a deterministic parent-state collection. Use it for generate-and-select flows: ask several providers, tools, prompts, or strategies for candidates, then rank or merge the gathered outputs once all relevant clones finish.
The application-facing value is simple: the parent DAG still looks linear after the scatter. Ranking, merging, auditing, or response composition can treat gathered candidates as ordinary state while Dagonizer handles clone lifecycle and ordering.
Code Samples
The same BookSearchScatterDAG drives the Archivist demo and the Mermaid diagram above. Read the scatter placement and book-search-gather declaration together; the JSON-LD is the contract the runtime enforces.
ts
/**
* BookSearchScatterDAG: reusable query-extract + tool-registry scatter cluster.
*
* Internal flow:
*
* extract-query
* └─ success ──► decide-tools
* decide-tools
* └─ (tools | no-tools) ──► recall-candidates
* recall-candidates
* └─ recalled ──► build-book-worksets
* build-book-worksets
* └─ ready ──► book-search-scatter (scatter over bookWorksets, concurrency 4)
* body: DagReference(item.dagIri) (resolves declared tool DAG IRI per item)
* book-search-gather: tool-candidate-merge (reads clone output via accessor, no cast)
* reducer: any-success (routes 'success' if any tool found results)
* └─ rank-candidates
* └─ merge-candidates
* ├─ ranked ──► record-findings
* └─ empty ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
* └─ record-findings
* └─ has-citations-gate
* ├─ pass ──► recall-past-visits ──► END (success)
* └─ fail ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
*
* Outputs:
* success: query extracted, candidates found, ranked, recorded, and recalled
* error: no candidates after merge, or citations gate failed;
* signalled by the no-results TerminalNode(failed) placement so
* the parent EmbeddedDAGNode routes the parent placement to its
* 'error' branch
*
* Molecular import pattern:
* import { bookSearchScatterDAG } from './embedded-dags/BookSearchScatterDAG.ts';
* const nodes = ArchivistNodes.build(services);
* dispatcher.registerBundle(toolRegistry.bundle<ArchivistServices>());
* dispatcher.registerBundle({ nodes: nodes.bookSearchScatterNodes, dags: [bookSearchScatterDAG] });
*
* The sub-DAG reads `state.query` directly (no input stateMapping; the field
* names already align with the parent). Each parent placement supplies an
* `outputs` stateMapping that copies the fields the sub-DAG writes:
* `terms`, `toolPlan`, `candidates`, `shortlist`, `priorContext`,
* `failureCause` back onto the parent state so the downstream compose,
* group-by-year, and recall steps can read them.
*
* Three EmbeddedDAGNode placements in the parent `the-archivist` DAG reference
* this one definition. One definition, three usages:
* on-topic-search: general web book search
* author-search: author body-of-work search
* similar-search: recommend-similar search
*
* Reviews and describe branches are inlined in the parent because they use
* distinct post-scout steps (rankByRating and pickBestMatch respectively).
*/
import type { ArchivistState } from '../ArchivistState.ts';
import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';
const BOOK_SEARCH_SCATTER_DAG_IRI = 'urn:noocodec:dag:book-search-scatter';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(BOOK_SEARCH_SCATTER_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });
const BOOK_SEARCH_TOOL_DAGS = [
'urn:noocodec:tool:web_search_books',
'urn:noocodec:tool:google_books_search',
'urn:noocodec:tool:subject_search',
'urn:noocodec:tool:wikipedia_summary',
] as const;
const extractQuery = new PlaceholderNode<ArchivistState, 'success' | 'retry' | 'salvage'>('urn:noocodec:node:extract-query', ['success', 'retry', 'salvage']);
const extractQuerySalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:extract-query-salvage', ['done']);
const decideTools = new PlaceholderNode<ArchivistState, 'tools' | 'no-tools' | 'retry' | 'salvage'>('urn:noocodec:node:decide-tools', ['tools', 'no-tools', 'retry', 'salvage']);
const decideToolsSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:decide-tools-salvage', ['done']);
const recallCandidates = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-candidates', ['recalled']);
const buildBookWorksets = new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:build-book-worksets', ['ready']);
const rankCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'retry' | 'salvage'>('urn:noocodec:node:rank-candidates', ['ranked', 'retry', 'salvage']);
const rankCandidatesSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:rank-candidates-salvage', ['done']);
const mergeCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'empty'>('urn:noocodec:node:merge-candidates', ['ranked', 'empty']);
const recordFindings = new PlaceholderNode<ArchivistState, 'recorded'>('urn:noocodec:node:record-findings', ['recorded']);
const hasCitationsGate = new PlaceholderNode<ArchivistState, 'pass' | 'fail'>('urn:noocodec:node:has-citations-gate', ['pass', 'fail']);
const recallPastVisits = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-past-visits', ['recalled']);
export const bookSearchScatterDAG: DAGType = new DAGBuilder(BOOK_SEARCH_SCATTER_DAG_IRI, '1.0', display('book-search-scatter'))
// ── 1. extract-query ─────────────────────────────────────────────────────
// LLM parses the raw visitor question into structured search terms.
// Writes state.terms for the scouts and decide-tools to consume.
// 'retry' loops back (bounded by the state retry budget); 'salvage' routes to
// a deterministic recovery node; never a fabricated term list on the node.
// #region retry-salvage-wiring
.node(placement('extract-query'), extractQuery, {
'success': placement('decide-tools'),
'retry': placement('extract-query'), // flow-shape retry loop (self-edge)
'salvage': placement('extract-query-salvage'), // recovery route
}, display('extract-query'))
.node(placement('extract-query-salvage'), extractQuerySalvage, {
'done': placement('decide-tools'), // deterministic recovery rejoins the happy path
}, display('extract-query-salvage'))
// #endregion retry-salvage-wiring
// ── 2. decide-tools ──────────────────────────────────────────────────────
// LLM decides which external sources to invoke. Both outputs route into
// recall-candidates so prior memory is loaded before scouts fire.
// 'retry' loops back (bounded); 'salvage' routes to the minimal-plan node.
.node(placement('decide-tools'), decideTools, {
'tools': placement('recall-candidates'),
'no-tools': placement('recall-candidates'),
'retry': placement('decide-tools'),
'salvage': placement('decide-tools-salvage'),
}, display('decide-tools'))
.node(placement('decide-tools-salvage'), decideToolsSalvage, {
'done': placement('recall-candidates'),
}, display('decide-tools-salvage'))
// ── 2b. recall-candidates ────────────────────────────────────────────────
// Pre-loads state.priorCandidates from memory: shortlisted books from prior
// runs whose visitor query has Jaccard >= 0.35 overlap with the current
// query. Cap 10. Always routes 'recalled', even when no prior runs match.
.node(placement('recall-candidates'), recallCandidates, {
'recalled': placement('build-book-worksets'),
}, display('recall-candidates'))
// ── 2c. build-book-worksets ──────────────────────────────────────────────
// Converts state.toolPlan into a bookWorksets array where each entry
// carries { dagIri: 'urn:noocodec:tool:<name>', arguments: {...} }. The scatter
// placement reads dagIri through an item-scoped DagReference to resolve
// the body DAG at runtime.
.node(placement('build-book-worksets'), buildBookWorksets, {
'ready': placement('book-search-scatter'),
}, display('build-book-worksets'))
// ── 3. book-search-scatter ───────────────────────────────────────────────
// Tool-registry scatter: bookWorksets items fan out concurrently. Each item
// carries its own tool DAG IRI via dagIri; the DagReference
// resolves the body DAG at runtime from the item. ToolInvokeNode reads the
// item's arguments field and calls the bound tool. The following GatherNode
// reads each clone's ToolInvocationState.output (via accessor, no cast)
// and folds the CandidateType[] into the parent state's candidates.
// any-success reducer: 'success' → rank-candidates when at least one tool hit;
// 'error' → rank-candidates to allow graceful empty-candidates handling.
.scatter(placement('book-search-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('book-search-gather'),
'error': placement('book-search-gather'),
'empty': placement('rank-candidates'),
}, {
'name': 'book-search-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('book-search-gather'), { [placement('book-search-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('rank-candidates'),
'error': placement('rank-candidates'),
'empty': placement('rank-candidates'),
}, display('book-search-gather'))
// ── 4. rank-candidates ───────────────────────────────────────────────────
// LLM-driven relevance scoring. Routes 'ranked' on success (an empty set is
// still a valid ranking, so merge can soft-gate on zero candidates).
// 'retry' loops back (bounded); 'salvage' passes candidates through unranked
// via a dedicated node rather than emitting them as if they were ranked.
.node(placement('rank-candidates'), rankCandidates, {
'ranked': placement('merge-candidates'),
'retry': placement('rank-candidates'),
'salvage': placement('rank-candidates-salvage'),
}, display('rank-candidates'))
.node(placement('rank-candidates-salvage'), rankCandidatesSalvage, {
'done': placement('merge-candidates'),
}, display('rank-candidates-salvage'))
// ── 5. merge-candidates ──────────────────────────────────────────────────
// Cross-source dedupe via CanonicalId, top-5. Routes 'empty' to
// no-results (TerminalNode(failed)) so the parent EmbeddedDAGNode's
// terminal outcome routes the parent placement to its 'error' branch.
.node(placement('merge-candidates'), mergeCandidates, {
'ranked': placement('record-findings'),
'empty': placement('no-results'),
}, display('merge-candidates'))
// ── 6. record-findings ───────────────────────────────────────────────────
// Deterministic RDF write: same input always produces the same triples.
.node(placement('record-findings'), recordFindings, {
'recorded': placement('has-citations-gate'),
}, display('record-findings'))
// ── 7. has-citations-gate ────────────────────────────────────────────────
// SPARQL ASK over the per-run state graph. Symbolic fence for the LLM.
// 'fail' routes to no-results (TerminalNode(failed)) so the parent
// EmbeddedDAGNode routes the parent placement to 'error'.
.node(placement('has-citations-gate'), hasCitationsGate, {
'pass': placement('recall-past-visits'),
'fail': placement('no-results'),
}, display('has-citations-gate'))
// ── 8. recall-past-visits ────────────────────────────────────────────────
// Injects prior-session context (prior queries + shortlisted titles) into
// state.priorContext, then routes to the canonical `found` TerminalNode
// (completed) so the parent EmbeddedDAGNode resolves its 'success' branch.
.node(placement('recall-past-visits'), recallPastVisits, {
'recalled': placement('found'),
}, display('recall-past-visits'))
// ── 9. Terminal nodes ────────────────────────────────────────────────────
// Both sub-DAG exits are canonical TerminalNode placements (no bare null
// routes): `found` (completed) drives the parent EmbeddedDAGNode's 'success'
// branch; `no-results` (failed) drives its 'error' branch.
.terminal(placement('found'), { outcome: 'completed', name: 'found' })
.terminal(placement('no-results'), { outcome: 'failed', name: 'no-results' })
.build();Details for Nerds
tool-candidate-mergegather strategy. The first-classbook-search-gatherplacement reads each clone's tool output and writes the merged candidates back to the parent clone'scandidatescollection.- Scatter body DAG. The
bodyuses a dynamicDagReference, so each workset chooses a registered tool DAG at runtime from an explicit candidate set. any-successoutcome reducer. A single successful provider is enough for the search branch to continue.- Source-index mental model. The parent sees deterministic gathered state even though provider work runs concurrently.
Related Concepts
- Example 04: Scatter Scout - scatter mechanics: source, body DAG, gather placement, reduce
- Example 04C: Container-Bound Scatter - bind a container role to a scatter placement
- Example 14: Gather strategies - gather strategy timing and fan-in barriers
- Reference: Core, GatherStrategies