Example 05: Embedded DAGs
What It Is
Embedded DAGs are how reusable flow parts become first-class assembly pieces. The Archivist parent DAG places the same book-search-scatter sub-DAG three times and the compose-retry-loop sub-DAG once via EmbeddedDAGNode.
This is the plugin-shaped mental model: ship a DAG IRI with its node registrations, then let a higher-level DAG place that graph wherever the product flow needs it. One definition, multiple placements, explicit state boundaries.
How It Works
The child DAG is registered in the same dispatcher registry as the parent. The parent placement references the child by DAG IRI, maps selected parent fields into child state before execution, waits for the child terminal outcome, maps selected child fields back to parent state, and then follows the parent placement's success or error route.
stateMapping keeps the boundary honest. The child receives only the parent fields it needs, and the parent receives only the child fields it asks for. The sub-DAG remains reusable because it is not secretly reaching into parent-specific state.
An EmbeddedDAGNode placement can also run the sub-DAG in an isolate by adding a container key to the placement and binding a DagContainerInterface backend at dispatcher construction. The Cartographer worker flow uses the same idea for contained work: execution location changes, but state mapping and routes stay part of the DAG contract.
Diagrams, Examples, and Outputs
DAG registration and diagram
The Archivist uses two packaged sub-DAGs, each placed via .embed():
The Archivist parent DAG
55 placements{
"@context": {
"@version": 1.1,
"name": {
"@id": "https://noocodec.dev/ontology/dag/name"
},
"version": {
"@id": "https://noocodec.dev/ontology/dag/version"
},
"entrypoints": {
"@id": "https://noocodec.dev/ontology/dag/entrypoints",
"@container": "@index"
},
"nodes": {
"@id": "https://noocodec.dev/ontology/dag/nodes",
"@container": "@set"
},
"outputs": {
"@id": "https://noocodec.dev/ontology/dag/outputs"
},
"node": {
"@id": "https://noocodec.dev/ontology/dag/node"
},
"dag": {
"@id": "https://noocodec.dev/ontology/dag/dag"
},
"body": {
"@id": "https://noocodec.dev/ontology/dag/body"
},
"source": {
"@id": "https://noocodec.dev/ontology/dag/source"
},
"sources": {
"@id": "https://noocodec.dev/ontology/dag/sources",
"@container": "@index"
},
"itemKey": {
"@id": "https://noocodec.dev/ontology/dag/itemKey"
},
"execution": {
"@id": "https://noocodec.dev/ontology/dag/execution"
},
"concurrency": {
"@id": "https://noocodec.dev/ontology/dag/concurrency"
},
"throttle": {
"@id": "https://noocodec.dev/ontology/dag/throttle"
},
"reservoir": {
"@id": "https://noocodec.dev/ontology/dag/reservoir"
},
"gather": {
"@id": "https://noocodec.dev/ontology/dag/gather"
},
"dagReference": {
"@id": "https://noocodec.dev/ontology/dag/dagReference",
"@type": "@id"
},
"DagReference": {
"@id": "https://noocodec.dev/ontology/dag/DagReference"
},
"from": {
"@id": "https://noocodec.dev/ontology/dag/from"
},
"path": {
"@id": "https://noocodec.dev/ontology/dag/path"
},
"candidates": {
"@id": "https://noocodec.dev/ontology/dag/candidates",
"@container": "@set"
},
"candidateDag": {
"@id": "https://noocodec.dev/ontology/dag/candidateDag",
"@type": "@id"
},
"selectedDag": {
"@id": "https://noocodec.dev/ontology/dag/selectedDag",
"@type": "@id"
},
"resultField": {
"@id": "https://noocodec.dev/ontology/dag/resultField"
},
"policy": {
"@id": "https://noocodec.dev/ontology/dag/policy"
},
"reducer": {
"@id": "https://noocodec.dev/ontology/dag/reducer"
},
"outcome": {
"@id": "https://noocodec.dev/ontology/dag/outcome"
},
"phase": {
"@id": "https://noocodec.dev/ontology/dag/phase"
},
"stateMapping": {
"@id": "https://noocodec.dev/ontology/dag/stateMapping"
},
"container": {
"@id": "https://noocodec.dev/ontology/dag/container"
},
"DAG": {
"@id": "https://noocodec.dev/ontology/dag/DAG"
},
"Placement": {
"@id": "https://noocodec.dev/ontology/dag/Placement"
},
"SingleNode": {
"@id": "https://noocodec.dev/ontology/dag/SingleNode"
},
"ScatterNode": {
"@id": "https://noocodec.dev/ontology/dag/ScatterNode"
},
"EmbeddedDAGNode": {
"@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
},
"GatherNode": {
"@id": "https://noocodec.dev/ontology/dag/GatherNode"
},
"TerminalNode": {
"@id": "https://noocodec.dev/ontology/dag/TerminalNode"
},
"PhaseNode": {
"@id": "https://noocodec.dev/ontology/dag/PhaseNode"
}
},
"@id": "urn:noocodec:dag:the-archivist",
"@type": "DAG",
"name": "the-archivist",
"version": "6.0",
"entrypoints": {
"main": "urn:noocodec:dag:the-archivist/node/park-for-input"
},
"nodes": [
{
"@id": "urn:noocodec:dag:the-archivist/node/setup",
"@type": "PhaseNode",
"name": "setup",
"node": "urn:noocodec:node:pre-run-setup",
"phase": "pre"
},
{
"@id": "urn:noocodec:dag:the-archivist/node/park-for-input",
"@type": "SingleNode",
"name": "park-for-input",
"node": "urn:noocodec:node:park-for-input",
"outputs": {
"parked": "urn:noocodec:dag:the-archivist/node/park-for-input",
"resumed": "urn:noocodec:dag:the-archivist/node/recall-context"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recall-context",
"@type": "SingleNode",
"name": "recall-context",
"node": "urn:noocodec:node:recall-context",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/classify-intent"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/classify-intent",
"@type": "SingleNode",
"name": "classify-intent",
"node": "urn:noocodec:node:classify-intent",
"outputs": {
"lookup-author": "urn:noocodec:dag:the-archivist/node/author-search",
"find-reviews": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"describe-book": "urn:noocodec:dag:the-archivist/node/describe-extract",
"recommend-similar": "urn:noocodec:dag:the-archivist/node/recommend-similar",
"recall-memories": "urn:noocodec:dag:the-archivist/node/memory-recall",
"on-topic": "urn:noocodec:dag:the-archivist/node/on-topic-search",
"recommend-top-rated": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"off-topic": "urn:noocodec:dag:the-archivist/node/decline-off-topic",
"retry": "urn:noocodec:dag:the-archivist/node/classify-intent",
"salvage": "urn:noocodec:dag:the-archivist/node/classify-intent-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/classify-intent-salvage",
"@type": "SingleNode",
"name": "classify-intent-salvage",
"node": "urn:noocodec:node:classify-intent-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/on-topic-search"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/on-topic-search",
"@type": "EmbeddedDAGNode",
"name": "on-topic-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/compose-loop",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/author-search",
"@type": "EmbeddedDAGNode",
"name": "author-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/group-by-year",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/group-by-year",
"@type": "SingleNode",
"name": "group-by-year",
"node": "urn:noocodec:node:group-by-year",
"outputs": {
"ordered": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"@type": "SingleNode",
"name": "reviews-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/reviews-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-extract-salvage",
"@type": "SingleNode",
"name": "reviews-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"@type": "SingleNode",
"name": "reviews-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools-salvage",
"@type": "SingleNode",
"name": "reviews-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"@type": "SingleNode",
"name": "reviews-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/reviews-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-scatter",
"@type": "ScatterNode",
"name": "reviews-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"error": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"empty": "urn:noocodec:dag:the-archivist/node/reviews-rank"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"@type": "GatherNode",
"name": "reviews-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/reviews-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"error": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"empty": "urn:noocodec:dag:the-archivist/node/reviews-rank"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"@type": "SingleNode",
"name": "reviews-rank",
"node": "urn:noocodec:node:rank-by-rating",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/reviews-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-merge",
"@type": "SingleNode",
"name": "reviews-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/reviews-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-record",
"@type": "SingleNode",
"name": "reviews-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/reviews-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-gate",
"@type": "SingleNode",
"name": "reviews-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/reviews-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-recall",
"@type": "SingleNode",
"name": "reviews-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"@type": "SingleNode",
"name": "recommend-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/recommend-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-extract-salvage",
"@type": "SingleNode",
"name": "recommend-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"@type": "SingleNode",
"name": "recommend-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools-salvage",
"@type": "SingleNode",
"name": "recommend-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"@type": "SingleNode",
"name": "recommend-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/recommend-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-scatter",
"@type": "ScatterNode",
"name": "recommend-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"error": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"empty": "urn:noocodec:dag:the-archivist/node/recommend-rank"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"@type": "GatherNode",
"name": "recommend-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/recommend-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"error": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"empty": "urn:noocodec:dag:the-archivist/node/recommend-rank"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"@type": "SingleNode",
"name": "recommend-rank",
"node": "urn:noocodec:node:rank-by-rating",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/recommend-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-merge",
"@type": "SingleNode",
"name": "recommend-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/recommend-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-record",
"@type": "SingleNode",
"name": "recommend-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/recommend-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-gate",
"@type": "SingleNode",
"name": "recommend-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/recommend-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-recall",
"@type": "SingleNode",
"name": "recommend-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-extract",
"@type": "SingleNode",
"name": "describe-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/describe-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/describe-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-extract-salvage",
"@type": "SingleNode",
"name": "describe-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/describe-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"@type": "SingleNode",
"name": "describe-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/describe-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-decide-tools-salvage",
"@type": "SingleNode",
"name": "describe-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/describe-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"@type": "SingleNode",
"name": "describe-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/describe-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-scatter",
"@type": "ScatterNode",
"name": "describe-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-gather",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-gather",
"@type": "GatherNode",
"name": "describe-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/describe-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-pick",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-pick",
"@type": "SingleNode",
"name": "describe-pick",
"node": "urn:noocodec:node:pick-best-match",
"outputs": {
"picked": "urn:noocodec:dag:the-archivist/node/describe-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-merge",
"@type": "SingleNode",
"name": "describe-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/describe-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-record",
"@type": "SingleNode",
"name": "describe-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/describe-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-gate",
"@type": "SingleNode",
"name": "describe-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/describe-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-recall",
"@type": "SingleNode",
"name": "describe-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-similar",
"@type": "SingleNode",
"name": "recommend-similar",
"node": "urn:noocodec:node:recommend-similar",
"outputs": {
"seeded": "urn:noocodec:dag:the-archivist/node/similar-search",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/similar-search",
"@type": "EmbeddedDAGNode",
"name": "similar-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/compose-loop",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-loop",
"@type": "EmbeddedDAGNode",
"name": "compose-loop",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:compose-retry-loop",
"stateMapping": {
"output": {
"draft": "draft",
"approvalState": "approvalState"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"@type": "SingleNode",
"name": "respond-to-visitor",
"node": "urn:noocodec:node:respond-to-visitor",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/end"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/memory-recall",
"@type": "SingleNode",
"name": "memory-recall",
"node": "urn:noocodec:node:recall-memories",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-memory-recall"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-memory-recall",
"@type": "SingleNode",
"name": "compose-memory-recall",
"node": "urn:noocodec:node:compose-memory-response",
"outputs": {
"drafted": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"retry": "urn:noocodec:dag:the-archivist/node/compose-memory-recall",
"salvage": "urn:noocodec:dag:the-archivist/node/compose-memory-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-memory-salvage",
"@type": "SingleNode",
"name": "compose-memory-salvage",
"node": "urn:noocodec:node:compose-memory-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/respond-to-visitor"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/decline-off-topic",
"@type": "SingleNode",
"name": "decline-off-topic",
"node": "urn:noocodec:node:decline-off-topic",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/end"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-empty",
"@type": "SingleNode",
"name": "compose-empty",
"node": "urn:noocodec:node:compose-empty",
"outputs": {
"drafted": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"retry": "urn:noocodec:dag:the-archivist/node/compose-empty",
"salvage": "urn:noocodec:dag:the-archivist/node/compose-empty-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-empty-salvage",
"@type": "SingleNode",
"name": "compose-empty-salvage",
"node": "urn:noocodec:node:compose-empty-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/respond-to-visitor"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/end",
"@type": "TerminalNode",
"name": "end",
"outcome": "completed"
}
]
}Mermaid source
%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
%% the-archivist (v6.0)
entry_main(["main"])
entry_main --> urn_noocodec_dag_the-archivist/node/park-for-input
urn_noocodec_dag_the-archivist/node/setup(["setup (pre)"])
urn_noocodec_dag_the-archivist/node/park-for-input["park-for-input"]
urn_noocodec_dag_the-archivist/node/park-for-input -->|parked| urn_noocodec_dag_the-archivist/node/park-for-input
urn_noocodec_dag_the-archivist/node/park-for-input -->|resumed| urn_noocodec_dag_the-archivist/node/recall-context
urn_noocodec_dag_the-archivist/node/recall-context["recall-context"]
urn_noocodec_dag_the-archivist/node/recall-context -->|recalled| urn_noocodec_dag_the-archivist/node/classify-intent
urn_noocodec_dag_the-archivist/node/classify-intent["classify-intent"]
urn_noocodec_dag_the-archivist/node/classify-intent -->|lookup-author| urn_noocodec_dag_the-archivist/node/author-search
urn_noocodec_dag_the-archivist/node/classify-intent -->|find-reviews| urn_noocodec_dag_the-archivist/node/reviews-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|describe-book| urn_noocodec_dag_the-archivist/node/describe-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|recommend-similar| urn_noocodec_dag_the-archivist/node/recommend-similar
urn_noocodec_dag_the-archivist/node/classify-intent -->|recall-memories| urn_noocodec_dag_the-archivist/node/memory-recall
urn_noocodec_dag_the-archivist/node/classify-intent -->|on-topic| urn_noocodec_dag_the-archivist/node/on-topic-search
urn_noocodec_dag_the-archivist/node/classify-intent -->|recommend-top-rated| urn_noocodec_dag_the-archivist/node/recommend-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|off-topic| urn_noocodec_dag_the-archivist/node/decline-off-topic
urn_noocodec_dag_the-archivist/node/classify-intent -->|retry| urn_noocodec_dag_the-archivist/node/classify-intent
urn_noocodec_dag_the-archivist/node/classify-intent -->|salvage| urn_noocodec_dag_the-archivist/node/classify-intent-salvage
urn_noocodec_dag_the-archivist/node/classify-intent-salvage["classify-intent-salvage"]
urn_noocodec_dag_the-archivist/node/classify-intent-salvage -->|done| urn_noocodec_dag_the-archivist/node/on-topic-search
urn_noocodec_dag_the-archivist/node/on-topic-search[["on-topic-search"]]
urn_noocodec_dag_the-archivist/node/on-topic-search -->|success| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/on-topic-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/author-search[["author-search"]]
urn_noocodec_dag_the-archivist/node/author-search -->|success| urn_noocodec_dag_the-archivist/node/group-by-year
urn_noocodec_dag_the-archivist/node/author-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/group-by-year["group-by-year"]
urn_noocodec_dag_the-archivist/node/group-by-year -->|ordered| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/reviews-extract["reviews-extract"]
urn_noocodec_dag_the-archivist/node/reviews-extract -->|success| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-extract -->|retry| urn_noocodec_dag_the-archivist/node/reviews-extract
urn_noocodec_dag_the-archivist/node/reviews-extract -->|salvage| urn_noocodec_dag_the-archivist/node/reviews-extract-salvage
urn_noocodec_dag_the-archivist/node/reviews-extract-salvage["reviews-extract-salvage"]
urn_noocodec_dag_the-archivist/node/reviews-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-decide-tools["reviews-decide-tools"]
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage["reviews-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-build-worksets["reviews-build-worksets"]
urn_noocodec_dag_the-archivist/node/reviews-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/reviews-scatter
urn_noocodec_dag_the-archivist/node/reviews-scatter[/"reviews-scatter"/]
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|success| urn_noocodec_dag_the-archivist/node/reviews-gather
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|error| urn_noocodec_dag_the-archivist/node/reviews-gather
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|empty| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather{"reviews-gather"}
urn_noocodec_dag_the-archivist/node/reviews-gather -->|success| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather -->|error| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather -->|empty| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-rank["reviews-rank"]
urn_noocodec_dag_the-archivist/node/reviews-rank -->|ranked| urn_noocodec_dag_the-archivist/node/reviews-merge
urn_noocodec_dag_the-archivist/node/reviews-merge["reviews-merge"]
urn_noocodec_dag_the-archivist/node/reviews-merge -->|ranked| urn_noocodec_dag_the-archivist/node/reviews-record
urn_noocodec_dag_the-archivist/node/reviews-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/reviews-record["reviews-record"]
urn_noocodec_dag_the-archivist/node/reviews-record -->|recorded| urn_noocodec_dag_the-archivist/node/reviews-gate
urn_noocodec_dag_the-archivist/node/reviews-gate["reviews-gate"]
urn_noocodec_dag_the-archivist/node/reviews-gate -->|pass| urn_noocodec_dag_the-archivist/node/reviews-recall
urn_noocodec_dag_the-archivist/node/reviews-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/reviews-recall["reviews-recall"]
urn_noocodec_dag_the-archivist/node/reviews-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/recommend-extract["recommend-extract"]
urn_noocodec_dag_the-archivist/node/recommend-extract -->|success| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-extract -->|retry| urn_noocodec_dag_the-archivist/node/recommend-extract
urn_noocodec_dag_the-archivist/node/recommend-extract -->|salvage| urn_noocodec_dag_the-archivist/node/recommend-extract-salvage
urn_noocodec_dag_the-archivist/node/recommend-extract-salvage["recommend-extract-salvage"]
urn_noocodec_dag_the-archivist/node/recommend-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-decide-tools["recommend-decide-tools"]
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage["recommend-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-build-worksets["recommend-build-worksets"]
urn_noocodec_dag_the-archivist/node/recommend-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/recommend-scatter
urn_noocodec_dag_the-archivist/node/recommend-scatter[/"recommend-scatter"/]
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|success| urn_noocodec_dag_the-archivist/node/recommend-gather
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|error| urn_noocodec_dag_the-archivist/node/recommend-gather
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|empty| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather{"recommend-gather"}
urn_noocodec_dag_the-archivist/node/recommend-gather -->|success| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather -->|error| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather -->|empty| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-rank["recommend-rank"]
urn_noocodec_dag_the-archivist/node/recommend-rank -->|ranked| urn_noocodec_dag_the-archivist/node/recommend-merge
urn_noocodec_dag_the-archivist/node/recommend-merge["recommend-merge"]
urn_noocodec_dag_the-archivist/node/recommend-merge -->|ranked| urn_noocodec_dag_the-archivist/node/recommend-record
urn_noocodec_dag_the-archivist/node/recommend-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/recommend-record["recommend-record"]
urn_noocodec_dag_the-archivist/node/recommend-record -->|recorded| urn_noocodec_dag_the-archivist/node/recommend-gate
urn_noocodec_dag_the-archivist/node/recommend-gate["recommend-gate"]
urn_noocodec_dag_the-archivist/node/recommend-gate -->|pass| urn_noocodec_dag_the-archivist/node/recommend-recall
urn_noocodec_dag_the-archivist/node/recommend-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/recommend-recall["recommend-recall"]
urn_noocodec_dag_the-archivist/node/recommend-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/describe-extract["describe-extract"]
urn_noocodec_dag_the-archivist/node/describe-extract -->|success| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-extract -->|retry| urn_noocodec_dag_the-archivist/node/describe-extract
urn_noocodec_dag_the-archivist/node/describe-extract -->|salvage| urn_noocodec_dag_the-archivist/node/describe-extract-salvage
urn_noocodec_dag_the-archivist/node/describe-extract-salvage["describe-extract-salvage"]
urn_noocodec_dag_the-archivist/node/describe-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-decide-tools["describe-decide-tools"]
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage["describe-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-build-worksets["describe-build-worksets"]
urn_noocodec_dag_the-archivist/node/describe-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/describe-scatter
urn_noocodec_dag_the-archivist/node/describe-scatter[/"describe-scatter"/]
urn_noocodec_dag_the-archivist/node/describe-scatter -->|success| urn_noocodec_dag_the-archivist/node/describe-gather
urn_noocodec_dag_the-archivist/node/describe-scatter -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-scatter -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-gather{"describe-gather"}
urn_noocodec_dag_the-archivist/node/describe-gather -->|success| urn_noocodec_dag_the-archivist/node/describe-pick
urn_noocodec_dag_the-archivist/node/describe-gather -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-gather -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-pick["describe-pick"]
urn_noocodec_dag_the-archivist/node/describe-pick -->|picked| urn_noocodec_dag_the-archivist/node/describe-merge
urn_noocodec_dag_the-archivist/node/describe-merge["describe-merge"]
urn_noocodec_dag_the-archivist/node/describe-merge -->|ranked| urn_noocodec_dag_the-archivist/node/describe-record
urn_noocodec_dag_the-archivist/node/describe-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-record["describe-record"]
urn_noocodec_dag_the-archivist/node/describe-record -->|recorded| urn_noocodec_dag_the-archivist/node/describe-gate
urn_noocodec_dag_the-archivist/node/describe-gate["describe-gate"]
urn_noocodec_dag_the-archivist/node/describe-gate -->|pass| urn_noocodec_dag_the-archivist/node/describe-recall
urn_noocodec_dag_the-archivist/node/describe-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-recall["describe-recall"]
urn_noocodec_dag_the-archivist/node/describe-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/recommend-similar["recommend-similar"]
urn_noocodec_dag_the-archivist/node/recommend-similar -->|seeded| urn_noocodec_dag_the-archivist/node/similar-search
urn_noocodec_dag_the-archivist/node/recommend-similar -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/similar-search[["similar-search"]]
urn_noocodec_dag_the-archivist/node/similar-search -->|success| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/similar-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/compose-loop[["compose-loop"]]
urn_noocodec_dag_the-archivist/node/compose-loop -->|success| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-loop -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/respond-to-visitor["respond-to-visitor"]
urn_noocodec_dag_the-archivist/node/respond-to-visitor -->|success| urn_noocodec_dag_the-archivist/node/end
urn_noocodec_dag_the-archivist/node/memory-recall["memory-recall"]
urn_noocodec_dag_the-archivist/node/memory-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-memory-recall
urn_noocodec_dag_the-archivist/node/compose-memory-recall["compose-memory-recall"]
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|drafted| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|retry| urn_noocodec_dag_the-archivist/node/compose-memory-recall
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|salvage| urn_noocodec_dag_the-archivist/node/compose-memory-salvage
urn_noocodec_dag_the-archivist/node/compose-memory-salvage["compose-memory-salvage"]
urn_noocodec_dag_the-archivist/node/compose-memory-salvage -->|done| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/decline-off-topic["decline-off-topic"]
urn_noocodec_dag_the-archivist/node/decline-off-topic -->|success| urn_noocodec_dag_the-archivist/node/end
urn_noocodec_dag_the-archivist/node/compose-empty["compose-empty"]
urn_noocodec_dag_the-archivist/node/compose-empty -->|drafted| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-empty -->|retry| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/compose-empty -->|salvage| urn_noocodec_dag_the-archivist/node/compose-empty-salvage
urn_noocodec_dag_the-archivist/node/compose-empty-salvage["compose-empty-salvage"]
urn_noocodec_dag_the-archivist/node/compose-empty-salvage -->|done| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/end((("end")))The generated diagram shows the parent graph, not a whiteboard sketch. on-topic-search, author-search, and similar-search all place book-search-scatter; compose-loop places compose-retry-loop; the edges around those placements remain normal parent-DAG routes.
What It Lets You Do
Embedded DAGs let applications package a reusable flow once and place it anywhere a parent DAG needs that behavior. Use this when a branch is large enough to own its own topology, state transfer boundary, and terminal semantics, but still belongs inside a larger product flow.
For plugin authors, this is the dev-ex target: a plugin can expose an embedded DAG IRI and node registry; the host can place it through the same interface it uses for local embedded DAGs. No second plugin assembly language, no hidden callback convention.
Code Samples
Sub-DAG: the packaged scout cluster
ts
/**
* BookSearchScatterDAG: reusable query-extract + tool-registry scatter cluster.
*
* Internal flow:
*
* extract-query
* └─ success ──► decide-tools
* decide-tools
* └─ (tools | no-tools) ──► recall-candidates
* recall-candidates
* └─ recalled ──► build-book-worksets
* build-book-worksets
* └─ ready ──► book-search-scatter (scatter over bookWorksets, concurrency 4)
* body: DagReference(item.dagIri) (resolves declared tool DAG IRI per item)
* book-search-gather: tool-candidate-merge (reads clone output via accessor, no cast)
* reducer: any-success (routes 'success' if any tool found results)
* └─ rank-candidates
* └─ merge-candidates
* ├─ ranked ──► record-findings
* └─ empty ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
* └─ record-findings
* └─ has-citations-gate
* ├─ pass ──► recall-past-visits ──► END (success)
* └─ fail ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
*
* Outputs:
* success: query extracted, candidates found, ranked, recorded, and recalled
* error: no candidates after merge, or citations gate failed;
* signalled by the no-results TerminalNode(failed) placement so
* the parent EmbeddedDAGNode routes the parent placement to its
* 'error' branch
*
* Molecular import pattern:
* import { bookSearchScatterDAG } from './embedded-dags/BookSearchScatterDAG.ts';
* const nodes = ArchivistNodes.build(services);
* dispatcher.registerBundle(toolRegistry.bundle<ArchivistServices>());
* dispatcher.registerBundle({ nodes: nodes.bookSearchScatterNodes, dags: [bookSearchScatterDAG] });
*
* The sub-DAG reads `state.query` directly (no input stateMapping; the field
* names already align with the parent). Each parent placement supplies an
* `outputs` stateMapping that copies the fields the sub-DAG writes:
* `terms`, `toolPlan`, `candidates`, `shortlist`, `priorContext`,
* `failureCause` back onto the parent state so the downstream compose,
* group-by-year, and recall steps can read them.
*
* Three EmbeddedDAGNode placements in the parent `the-archivist` DAG reference
* this one definition. One definition, three usages:
* on-topic-search: general web book search
* author-search: author body-of-work search
* similar-search: recommend-similar search
*
* Reviews and describe branches are inlined in the parent because they use
* distinct post-scout steps (rankByRating and pickBestMatch respectively).
*/
import type { ArchivistState } from '../ArchivistState.ts';
import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';
const BOOK_SEARCH_SCATTER_DAG_IRI = 'urn:noocodec:dag:book-search-scatter';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(BOOK_SEARCH_SCATTER_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });
const BOOK_SEARCH_TOOL_DAGS = [
'urn:noocodec:tool:web_search_books',
'urn:noocodec:tool:google_books_search',
'urn:noocodec:tool:subject_search',
'urn:noocodec:tool:wikipedia_summary',
] as const;
const extractQuery = new PlaceholderNode<ArchivistState, 'success' | 'retry' | 'salvage'>('urn:noocodec:node:extract-query', ['success', 'retry', 'salvage']);
const extractQuerySalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:extract-query-salvage', ['done']);
const decideTools = new PlaceholderNode<ArchivistState, 'tools' | 'no-tools' | 'retry' | 'salvage'>('urn:noocodec:node:decide-tools', ['tools', 'no-tools', 'retry', 'salvage']);
const decideToolsSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:decide-tools-salvage', ['done']);
const recallCandidates = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-candidates', ['recalled']);
const buildBookWorksets = new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:build-book-worksets', ['ready']);
const rankCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'retry' | 'salvage'>('urn:noocodec:node:rank-candidates', ['ranked', 'retry', 'salvage']);
const rankCandidatesSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:rank-candidates-salvage', ['done']);
const mergeCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'empty'>('urn:noocodec:node:merge-candidates', ['ranked', 'empty']);
const recordFindings = new PlaceholderNode<ArchivistState, 'recorded'>('urn:noocodec:node:record-findings', ['recorded']);
const hasCitationsGate = new PlaceholderNode<ArchivistState, 'pass' | 'fail'>('urn:noocodec:node:has-citations-gate', ['pass', 'fail']);
const recallPastVisits = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-past-visits', ['recalled']);
export const bookSearchScatterDAG: DAGType = new DAGBuilder(BOOK_SEARCH_SCATTER_DAG_IRI, '1.0', display('book-search-scatter'))
// ── 1. extract-query ─────────────────────────────────────────────────────
// LLM parses the raw visitor question into structured search terms.
// Writes state.terms for the scouts and decide-tools to consume.
// 'retry' loops back (bounded by the state retry budget); 'salvage' routes to
// a deterministic recovery node; never a fabricated term list on the node.
// #region retry-salvage-wiring
.node(placement('extract-query'), extractQuery, {
'success': placement('decide-tools'),
'retry': placement('extract-query'), // flow-shape retry loop (self-edge)
'salvage': placement('extract-query-salvage'), // recovery route
}, display('extract-query'))
.node(placement('extract-query-salvage'), extractQuerySalvage, {
'done': placement('decide-tools'), // deterministic recovery rejoins the happy path
}, display('extract-query-salvage'))
// #endregion retry-salvage-wiring
// ── 2. decide-tools ──────────────────────────────────────────────────────
// LLM decides which external sources to invoke. Both outputs route into
// recall-candidates so prior memory is loaded before scouts fire.
// 'retry' loops back (bounded); 'salvage' routes to the minimal-plan node.
.node(placement('decide-tools'), decideTools, {
'tools': placement('recall-candidates'),
'no-tools': placement('recall-candidates'),
'retry': placement('decide-tools'),
'salvage': placement('decide-tools-salvage'),
}, display('decide-tools'))
.node(placement('decide-tools-salvage'), decideToolsSalvage, {
'done': placement('recall-candidates'),
}, display('decide-tools-salvage'))
// ── 2b. recall-candidates ────────────────────────────────────────────────
// Pre-loads state.priorCandidates from memory: shortlisted books from prior
// runs whose visitor query has Jaccard >= 0.35 overlap with the current
// query. Cap 10. Always routes 'recalled', even when no prior runs match.
.node(placement('recall-candidates'), recallCandidates, {
'recalled': placement('build-book-worksets'),
}, display('recall-candidates'))
// ── 2c. build-book-worksets ──────────────────────────────────────────────
// Converts state.toolPlan into a bookWorksets array where each entry
// carries { dagIri: 'urn:noocodec:tool:<name>', arguments: {...} }. The scatter
// placement reads dagIri through an item-scoped DagReference to resolve
// the body DAG at runtime.
.node(placement('build-book-worksets'), buildBookWorksets, {
'ready': placement('book-search-scatter'),
}, display('build-book-worksets'))
// ── 3. book-search-scatter ───────────────────────────────────────────────
// Tool-registry scatter: bookWorksets items fan out concurrently. Each item
// carries its own tool DAG IRI via dagIri; the DagReference
// resolves the body DAG at runtime from the item. ToolInvokeNode reads the
// item's arguments field and calls the bound tool. The following GatherNode
// reads each clone's ToolInvocationState.output (via accessor, no cast)
// and folds the CandidateType[] into the parent state's candidates.
// any-success reducer: 'success' → rank-candidates when at least one tool hit;
// 'error' → rank-candidates to allow graceful empty-candidates handling.
.scatter(placement('book-search-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('book-search-gather'),
'error': placement('book-search-gather'),
'empty': placement('rank-candidates'),
}, {
'name': 'book-search-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('book-search-gather'), { [placement('book-search-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('rank-candidates'),
'error': placement('rank-candidates'),
'empty': placement('rank-candidates'),
}, display('book-search-gather'))
// ── 4. rank-candidates ───────────────────────────────────────────────────
// LLM-driven relevance scoring. Routes 'ranked' on success (an empty set is
// still a valid ranking, so merge can soft-gate on zero candidates).
// 'retry' loops back (bounded); 'salvage' passes candidates through unranked
// via a dedicated node rather than emitting them as if they were ranked.
.node(placement('rank-candidates'), rankCandidates, {
'ranked': placement('merge-candidates'),
'retry': placement('rank-candidates'),
'salvage': placement('rank-candidates-salvage'),
}, display('rank-candidates'))
.node(placement('rank-candidates-salvage'), rankCandidatesSalvage, {
'done': placement('merge-candidates'),
}, display('rank-candidates-salvage'))
// ── 5. merge-candidates ──────────────────────────────────────────────────
// Cross-source dedupe via CanonicalId, top-5. Routes 'empty' to
// no-results (TerminalNode(failed)) so the parent EmbeddedDAGNode's
// terminal outcome routes the parent placement to its 'error' branch.
.node(placement('merge-candidates'), mergeCandidates, {
'ranked': placement('record-findings'),
'empty': placement('no-results'),
}, display('merge-candidates'))
// ── 6. record-findings ───────────────────────────────────────────────────
// Deterministic RDF write: same input always produces the same triples.
.node(placement('record-findings'), recordFindings, {
'recorded': placement('has-citations-gate'),
}, display('record-findings'))
// ── 7. has-citations-gate ────────────────────────────────────────────────
// SPARQL ASK over the per-run state graph. Symbolic fence for the LLM.
// 'fail' routes to no-results (TerminalNode(failed)) so the parent
// EmbeddedDAGNode routes the parent placement to 'error'.
.node(placement('has-citations-gate'), hasCitationsGate, {
'pass': placement('recall-past-visits'),
'fail': placement('no-results'),
}, display('has-citations-gate'))
// ── 8. recall-past-visits ────────────────────────────────────────────────
// Injects prior-session context (prior queries + shortlisted titles) into
// state.priorContext, then routes to the canonical `found` TerminalNode
// (completed) so the parent EmbeddedDAGNode resolves its 'success' branch.
.node(placement('recall-past-visits'), recallPastVisits, {
'recalled': placement('found'),
}, display('recall-past-visits'))
// ── 9. Terminal nodes ────────────────────────────────────────────────────
// Both sub-DAG exits are canonical TerminalNode placements (no bare null
// routes): `found` (completed) drives the parent EmbeddedDAGNode's 'success'
// branch; `no-results` (failed) drives its 'error' branch.
.terminal(placement('found'), { outcome: 'completed', name: 'found' })
.terminal(placement('no-results'), { outcome: 'failed', name: 'no-results' })
.build();Parent DAG: the embedded-DAG placements
The #embedded-dag-placements region covers only the .embed(...) calls: the three placements of book-search-scatter and the one placement of compose-retry-loop:
ts
// ── on-topic branch ──────────────────────────────────────────────────────
// EmbeddedDAGNode: book-search-scatter handles extract-query, decide-tools,
// all four scouts, rank-candidates, merge, record, gate, and recall.
// One packaged cluster; first of three placements of the same sub-DAG.
// gather.map copies the fields the sub-DAG writes back to the parent state
// so compose-loop and group-by-year can read them.
.embed(placement('on-topic-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('compose-loop'),
'error': placement('compose-empty'),
}, {
'name': 'on-topic-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// ── lookup-author branch ─────────────────────────────────────────────────
// EmbeddedDAGNode: same book-search-scatter cluster, second placement.
// After success, group-by-year sorts results chronologically before the
// compose loop; author surveys read better in publication-timeline order.
.embed(placement('author-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('group-by-year'),
'error': placement('compose-empty'),
}, {
'name': 'author-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// group-by-year is author-branch-specific: sorts shortlist chronologically.
.node(placement('group-by-year'), nodes.groupByYear, {
'ordered': placement('compose-loop'),
}, display('group-by-year'))
// ── find-reviews branch ───────────────────────────────────────────────────
// Inlined. Uses rankByRating (deterministic, rating-weighted) in place of
// rankCandidates (LLM-driven). The Google Books scout carries notes.rating /
// notes.ratingsCount; rankByRating weights those for reviews-style output.
.node(placement('reviews-extract'), nodes.extractQuery, {
'success': placement('reviews-decide-tools'),
'retry': placement('reviews-extract'),
'salvage': placement('reviews-extract-salvage'),
}, display('reviews-extract'))
.node(placement('reviews-extract-salvage'), nodes.extractQuerySalvage, {
'done': placement('reviews-decide-tools'),
}, display('reviews-extract-salvage'))
.node(placement('reviews-decide-tools'), nodes.decideTools, {
'tools': placement('reviews-build-worksets'),
'no-tools': placement('reviews-build-worksets'),
'retry': placement('reviews-decide-tools'),
'salvage': placement('reviews-decide-tools-salvage'),
}, display('reviews-decide-tools'))
.node(placement('reviews-decide-tools-salvage'), nodes.decideToolsSalvage, {
'done': placement('reviews-build-worksets'),
}, display('reviews-decide-tools-salvage'))
// Build scatter worksets: converts toolPlan into bookWorksets items so the
// scatter can dispatch to each declared tool DAG IRI.
.node(placement('reviews-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('reviews-scatter'),
}, display('reviews-build-worksets'))
// Tool-registry scatter: each bookWorksets item carries its own tool DAG
// IRI. The following GatherNode reads each clone's output via
// accessor (no cast) and folds CandidateType[] into parent candidates.
.scatter(placement('reviews-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('reviews-gather'),
'error': placement('reviews-gather'),
'empty': placement('reviews-rank'),
}, {
'name': 'reviews-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('reviews-gather'), { [placement('reviews-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('reviews-rank'),
'error': placement('reviews-rank'),
'empty': placement('reviews-rank'),
}, display('reviews-gather'))
.node(placement('reviews-rank'), nodes.rankByRating, { 'ranked': placement('reviews-merge') }, display('reviews-rank'))
.node(placement('reviews-merge'), nodes.mergeCandidates, { 'ranked': placement('reviews-record'), 'empty': placement('compose-empty') }, display('reviews-merge'))
.node(placement('reviews-record'), nodes.recordFindings, { 'recorded': placement('reviews-gate') }, display('reviews-record'))
.node(placement('reviews-gate'), nodes.hasCitationsGate, { 'pass': placement('reviews-recall'), 'fail': placement('compose-empty') }, display('reviews-gate'))
.node(placement('reviews-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('reviews-recall'))
// ── recommend-top-rated branch ───────────────────────────────────────────
// Inlined, structural sibling of find-reviews. Reuses rankByRating
// (deterministic, rating-weighted) instead of rankCandidates (LLM-driven)
// because a vague "good book / good story" request carries no topic for
// relevance ranking — rating is the only signal that makes sense.
.node(placement('recommend-extract'), nodes.extractQuery, {
'success': placement('recommend-decide-tools'),
'retry': placement('recommend-extract'),
'salvage': placement('recommend-extract-salvage'),
}, display('recommend-extract'))
.node(placement('recommend-extract-salvage'), nodes.extractQuerySalvage, {
'done': placement('recommend-decide-tools'),
}, display('recommend-extract-salvage'))
.node(placement('recommend-decide-tools'), nodes.decideTools, {
'tools': placement('recommend-build-worksets'),
'no-tools': placement('recommend-build-worksets'),
'retry': placement('recommend-decide-tools'),
'salvage': placement('recommend-decide-tools-salvage'),
}, display('recommend-decide-tools'))
.node(placement('recommend-decide-tools-salvage'), nodes.decideToolsSalvage, {
'done': placement('recommend-build-worksets'),
}, display('recommend-decide-tools-salvage'))
// Build scatter worksets: converts toolPlan into bookWorksets items so the
// scatter can dispatch to each declared tool DAG IRI.
.node(placement('recommend-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('recommend-scatter'),
}, display('recommend-build-worksets'))
// Tool-registry scatter: each bookWorksets item carries its own tool DAG
// IRI. The following GatherNode reads each clone's output via
// accessor (no cast) and folds CandidateType[] into parent candidates.
.scatter(placement('recommend-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('recommend-gather'),
'error': placement('recommend-gather'),
'empty': placement('recommend-rank'),
}, {
'name': 'recommend-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('recommend-gather'), { [placement('recommend-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('recommend-rank'),
'error': placement('recommend-rank'),
'empty': placement('recommend-rank'),
}, display('recommend-gather'))
.node(placement('recommend-rank'), nodes.rankByRating, { 'ranked': placement('recommend-merge') }, display('recommend-rank'))
.node(placement('recommend-merge'), nodes.mergeCandidates, { 'ranked': placement('recommend-record'), 'empty': placement('compose-empty') }, display('recommend-merge'))
.node(placement('recommend-record'), nodes.recordFindings, { 'recorded': placement('recommend-gate') }, display('recommend-record'))
.node(placement('recommend-gate'), nodes.hasCitationsGate, { 'pass': placement('recommend-recall'), 'fail': placement('compose-empty') }, display('recommend-gate'))
.node(placement('recommend-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('recommend-recall'))
// ── describe-book branch ─────────────────────────────────────────────────
// Inlined. Uses pickBestMatch to narrow multi-hit results to the top-3
// title-similar candidates before merge. Ensures the composer receives the
// specific book the visitor named, not arbitrary top-5 hits.
.node(placement('describe-extract'), nodes.extractQuery, { 'success': placement('describe-decide-tools'), 'retry': placement('describe-extract'), 'salvage': placement('describe-extract-salvage') }, display('describe-extract'))
.node(placement('describe-extract-salvage'), nodes.extractQuerySalvage, { 'done': placement('describe-decide-tools') }, display('describe-extract-salvage'))
.node(placement('describe-decide-tools'), nodes.decideTools, { 'tools': placement('describe-build-worksets'), 'no-tools': placement('describe-build-worksets'), 'retry': placement('describe-decide-tools'), 'salvage': placement('describe-decide-tools-salvage') }, display('describe-decide-tools'))
.node(placement('describe-decide-tools-salvage'), nodes.decideToolsSalvage, { 'done': placement('describe-build-worksets') }, display('describe-decide-tools-salvage'))
// Build scatter worksets before dispatch.
.node(placement('describe-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('describe-scatter'),
}, display('describe-build-worksets'))
// Tool-registry scatter: DagReference resolves body DAG from each item's dagName.
// any-success reducer: 'success' → describe-pick, 'error' → compose-empty.
// 'error' fires when all tool scouts return empty.
.scatter(placement('describe-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('describe-gather'),
'error': placement('compose-empty'),
'empty': placement('compose-empty'),
}, {
'name': 'describe-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('describe-gather'), { [placement('describe-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('describe-pick'),
'error': placement('compose-empty'),
'empty': placement('compose-empty'),
}, display('describe-gather'))
.node(placement('describe-pick'), nodes.pickBestMatch, { 'picked': placement('describe-merge') }, display('describe-pick'))
.node(placement('describe-merge'), nodes.mergeCandidates, { 'ranked': placement('describe-record'), 'empty': placement('compose-empty') }, display('describe-merge'))
.node(placement('describe-record'), nodes.recordFindings, { 'recorded': placement('describe-gate') }, display('describe-record'))
.node(placement('describe-gate'), nodes.hasCitationsGate, { 'pass': placement('describe-recall'), 'fail': placement('compose-empty') }, display('describe-gate'))
.node(placement('describe-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('describe-recall'))
// ── recommend-similar branch ─────────────────────────────────────────────
// recommendSimilar seeds state.terms from prior-run shortlist memory.
// 'seeded' routes to the book-search-scatter sub-DAG; third placement of
// the same packaged cluster. 'empty' routes to the compose-empty terminal.
.node(placement('recommend-similar'), nodes.recommendSimilar, {
'seeded': placement('similar-search'),
'empty': placement('compose-empty'),
}, display('recommend-similar'))
// EmbeddedDAGNode: same book-search-scatter, third and final placement.
.embed(placement('similar-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('compose-loop'),
'error': placement('compose-empty'),
}, {
'name': 'similar-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// ── compose-loop: shared compose/validate sub-DAG ──────────────────────────
// All branches that successfully find candidates converge here.
// composeResponse → validateResponse (retry loop, bounded by the retry budget on state (retriesFor('compose'))).
// One sub-DAG definition serves all four convergent branches.
// stateMapping.outputs copies the compose loop's writes back to the parent.
//
// Convergence policy: 'success' routes to the shared respond-to-visitor terminal
// at the parent level; the sub-DAG produces state.draft and exits cleanly;
// exactly ONE respond-to-visitor fires per run regardless of branch count.
// 'error' (retry budget exhausted) falls through to compose-empty so the
// visitor always receives an in-character response rather than a silent drop.
.embed(placement('compose-loop'), COMPOSE_RETRY_LOOP_DAG_IRI, {
'success': placement('respond-to-visitor'),
'error': placement('compose-empty'),
}, {
'name': 'compose-loop',
'outputs': {
'draft': 'draft',
'approvalState': 'approvalState',
},
})Container-ready embedded placement
The Cartographer worker DAG shows the same contained-execution seam from a runnable browser example. A contained embedded placement adds a container role to the DAG document; the stateMapping.input seed and stateMapping.output copy operate identically in both paths. An unbound role uses in-process execution and fires contractWarning.
ts
/** Default reservoir capacity for the process-stream scatter in the workers DAG. */
export const DEFAULT_RESERVOIR_CAPACITY = 1000;
/**
* CartographerWorkersDag: static factory for the cartographer-workers DAG and
* its associated dispatcher bundle. Consumers call CartographerWorkersDag.build()
* (DAG only) or CartographerWorkersDag.bundle() (full dispatcher bundle) with an
* optional reservoir capacity override.
*
* DAG topology — identical to cartographerDAG with containerized boundaries:
* - container: 'cpu' so each stream-event body runs inside a
* WorkerThreadContainer/WebWorkerContainer rather than in-process.
* - container: 'io' so the final summary runs through the same embedded-DAG
* interface used by plugins and nested flows.
* - reservoir.capacity is parameterised; callers pass their UI-controlled
* batch size rather than relying on the compile-time default.
*
* 5 data-type entrypoints → gather('intake-gather', source-intake)
* → scatter('process-stream', 'sources', { dag: 'stream-event' },
* concurrency: 16, container: 'cpu', reservoir: { capacity })
* → gather('fold-insights', strategy: insights-fold)
* → embed('summarize-insights', 'insights-summary', container: 'io')
* → done
*/
export class CartographerWorkersDag {
private constructor() { /* static-only */ }
/**
* Build the cartographer-workers DAG with the given reservoir capacity.
* CLI, smoke tests, and dag-validate consumers use cartographerWorkersDAG
* (the pre-built constant); the browser demo calls this with a UI-controlled value.
*/
static build(capacity: number = DEFAULT_RESERVOIR_CAPACITY): DAGType {
return new DAGBuilder(CARTOGRAPHER_DAG_IRI, '1.0')
.gather(
CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
CARTOGRAPHER_IRIS.intakeSources(CARTOGRAPHER_DAG_IRI),
{ 'strategy': 'source-intake' },
{
'success': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'process-stream'),
'error': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'process-stream'),
'empty': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'failed'),
},
)
.scatter(
CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'process-stream'),
'sources',
{ 'dag': STREAM_EVENT_DAG_IRI },
{
'all-success': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'fold-insights'),
'partial': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'fold-insights'),
'all-error': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'fold-insights'),
'empty': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'summarize-insights'),
},
{
'itemKey': 'source-payload',
'container': 'cpu',
'execution': { 'mode': 'reservoir', 'concurrency': 16, 'reservoir': { 'keyField': 'eventType', 'capacity': capacity } },
},
)
.gather(CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'fold-insights'), {
[CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'process-stream')]: {},
}, { 'strategy': 'insights-fold' }, {
'success': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'summarize-insights'),
'error': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'summarize-insights'),
'empty': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'summarize-insights'),
})
.embed<CartographerState, CartographerState>(CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'summarize-insights'), INSIGHTS_SUMMARY_DAG_IRI, {
'success': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'done'),
'error': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'failed'),
}, {
'container': 'io',
})
.terminal(CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'done'), { outcome: 'completed' })
.terminal(CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'failed'), { outcome: 'failed' })
.entrypoints({
'position-ping': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
'facility-scan': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
'sensor-reading': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
'customs-event': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
'delivery-confirmation': CARTOGRAPHER_IRIS.placementIri(CARTOGRAPHER_DAG_IRI, 'intake-gather'),
})
.build();
}
/**
* Build the workers bundle with a configurable reservoir capacity. The returned
* bundle is identical to cartographerWorkersBundle except that its cartographer
* DAG is built with CartographerWorkersDag.build(capacity) so the process-stream
* scatter uses the caller-supplied batch size.
*
* Used by the browser demo to wire UI-controlled knobs into each run() without
* mutating the shared default-capacity constants.
*/
static bundle(
capacity: number = DEFAULT_RESERVOIR_CAPACITY,
): DispatcherBundleType<CartographerState> {
return {
'nodes': [
...cartographerWorkerRuntimeBundle.nodes,
],
'dags': [
...cartographerWorkerRuntimeBundle.dags,
CartographerWorkersDag.build(capacity),
],
};
}
}
/**
* cartographerWorkersDAG: pre-built workers DAG at DEFAULT_RESERVOIR_CAPACITY.
* CLI, smoke tests, and dag-validate consumers use this constant; the browser
* demo uses CartographerWorkersDag.build(capacity) with a UI-controlled value.
*/
export const cartographerWorkersDAG: DAGType = CartographerWorkersDag.build();Details for Nerds
Typed stateMapping and growing shared state
The .embed() call accepts TChildState and TParentState generic parameters that narrow options.inputs keys and options.outputs paths to dotted paths that exist on the respective state at compile time. The Archivist placements above use those generic parameters for parent/child transfer.
A misspelled parent-state path is a compile error.
stateMapping is the right tool when the relationship between parent and child is a pure field transfer at a single boundary. When multiple embedded-DAG placements accumulate to a single growing structure (agent memory, a ranked-results list, an audit log), pass a Store into each node's constructor instead. The store lives outside the DAG topology; every placement reads and writes to the same instance without threading values through stateMapping at every hop. See Shared state for the decision matrix, the concurrency contract, and checkpoint integration.
What it demonstrates
.embed(placementIri, dagIri, routes, options). The placement references the sub-DAG by its registered DAG IRI. The parent and child run in the same dispatcher; the child shares the same node registry.book-search-scatter: the full 4-source scout cluster (extract query, decide tools, 4 parallel scouts, rank, merge, record, gate, recall). Placed three times in the parent:on-topic-search,author-search, andsimilar-search.compose-retry-loop: the compose, validate, retry, respond terminal. Placed once ascompose-loop; every successful search branch converges on it.
Each embedded-DAG placement uses the wire field stateMapping.input to seed child fields from parent paths before the body runs and stateMapping.output to copy produced child fields back into the parent after the body completes. (The builder option object spells these inputs / outputs; the serialized JSON-LD wire form is singular.)
stateMapping.input(wire) /inputs(builder option). Before the body runs, the dispatcher copies the listed parent fields into the child. The child receives the seed; the body then reads from the child.stateMapping.output(wire) /outputs(builder option). After the body completes, the dispatcher copies the listed child fields back into the parent. Fields not listed stay isolated.- One definition, three placements.
urn:noocodec:dag:book-search-scatteris registered once and placed three times with distinct placement IRIs and display names. Each placement routes its'success'and'error'outputs differently (compose-loop,group-by-year, orcompose-empty). - Errors bubble up. Anything the child accumulates via
state.collectErrorreaches the parent's error accumulator automatically. The child's terminal outcome determines the'error'output. bookSearchScatterDAGandcomposeRetryLoopDAG. Each sub-DAG module exports a canonical JSON-LD DAG. Register literal bundles with the matching concrete node groups before the parentarchivistDAGso embedded references resolve before parent validation.
See this in action in the Archivist live demo.
Related Concepts
- Running domain: The Archivist
- Example 04: Scatter Scout
- Example 02: DAGBuilder - the full parent DAG authored with DAGBuilder
- Reference: Entities,
EmbeddedDAGNode