Checkpoint and Resume
What It Is
Checkpoint and resume lets an interrupted DAG continue from the next runnable placement instead of starting over. Checkpoint.capture snapshots the interrupted execution; Checkpoint.load validates a saved payload; restoreState rehydrates domain state; dispatcher.resume continues from the recorded cursor.
Use it when an application needs interruption to be a controlled lifecycle state: cancellation, timeout, human-in-the-loop parking, long scatter runs, or process hand-off.
How It Works
Checkpoint.capture serializes the interrupted result, state snapshot, cursor, optional store snapshots, and execution metadata. Checkpoint.load validates a saved payload before use. restoreState reconstructs domain state, restoreStores rehydrates named stores, and dispatcher.resume continues the registered DAG from the restored cursor.
Checkpoint is the codec: it turns an interrupted ExecutionResult into a portable record and back. dispatcher.resume(dagName, state, fromStage, options?) picks the execution up from the restored cursor; despite the parameter name, pass the restored DAG IRI/CURIE and placement IRI. Persistence is the application's concern (see persistence).
Diagrams, Examples, and Outputs
Capturing a partial run
When a DAG stops early (cancellation, timeout, error), result.cursor holds the placement IRI of the next placement that would have run. Pass that to Checkpoint.capture():
ts
const dispatcher = new Dagonizer<CountingState>();
dispatcher.registerNode(new IncNode());
dispatcher.registerDAG(dag);
const ctl = new AbortController();
const initial = new CountingState();
// execute() returns an Execution (async-iterable over node results).
// Iterating yields one result per completed node (not per stage internally).
const execution = dispatcher.execute('urn:noocodec:dag:count', initial, { "signal": ctl.signal });
let stages = 0;
for await (const _stage of execution) {
stages++;
if (stages === 1) ctl.abort(new Error('pause after node a')); // fire after 'a' completes
}
const partial = await execution;
process.stdout.write('\nCheckpoint lifecycle: abort -> snapshot -> restore -> resume\n');
process.stdout.write(` partial: count=${partial.state.count} cursor="${partial.cursor}"\n`);
// cursor = 'b': the next node that would run if we resumeCheckpoint.capture() throws DAGError when result.cursor === null (the DAG completed, nothing to resume).
What It Lets You Do
Use when
Use checkpoint and resume when an interrupted DAG should continue from a known cursor instead of restarting at the entrypoint. This applies to cancellation, timeout, human-in-the-loop parking, long scatter runs, and process hand-off.
Code Samples
API surface
| Symbol | Source | Role |
|---|---|---|
Checkpoint.capture(dagName, result, options?) | @studnicky/dagonizer/checkpoint | Async factory: turns a paused execution into a Checkpoint; pass the DAG IRI/CURIE string used for execution |
Checkpoint.load(raw) | @studnicky/dagonizer/checkpoint | Schema-validates an unknown value into a Checkpoint |
Checkpoint.recall(store, key) | @studnicky/dagonizer/checkpoint | Reads + parses + validates from a CheckpointStore |
ckpt.toJson() | instance method | Serializes to a JSON string |
ckpt.persist(store, key) | instance method | Writes via a CheckpointStore |
ckpt.restoreState(adapter) | instance method | Rehydrates { dagName, state, cursor }; dagName is the stored DAG IRI/CURIE string and cursor is a placement IRI |
ckpt.restoreStores(map) | instance method | Restores named stores (any Snapshottable) from the envelope |
dispatcher.resume(dagName, state, fromStage, options?) | @studnicky/dagonizer | Resumes the flow at fromStage; options accepts the same ExecuteOptionsType as execute |
snapshotData and restoreData contract
snapshotData()returns a JSON-serializableJsonObjectType. Noundefinedvalues, no circular references.restoreData(snap)receives the full merged snapshot (base fields plus domain fields). Callsuper.applySnapshot(snap)when overridingapplySnapshotdirectly.- Lifecycle is intentionally not captured.
resume()starts a fresh lifecycle run frompending. - Engine errors are intentionally not captured.
applySnapshotleaves_errorsuntouched; the caller populates errors fromoutcome.errorsafter applying the snapshot.
Details for Nerds
Runnable DAG that drives the example
The Archivist is the runnable checkpoint example. Its compose / validate loop is expensive enough to make resume meaningful: a cancelled run preserves the cursor, draft, retry counters, and serializable ArchivistState fields, then a later process resumes without re-running upstream scouts.
The Archivist parent DAG
55 placements{
"@context": {
"@version": 1.1,
"name": {
"@id": "https://noocodec.dev/ontology/dag/name"
},
"version": {
"@id": "https://noocodec.dev/ontology/dag/version"
},
"entrypoints": {
"@id": "https://noocodec.dev/ontology/dag/entrypoints",
"@container": "@index"
},
"nodes": {
"@id": "https://noocodec.dev/ontology/dag/nodes",
"@container": "@set"
},
"outputs": {
"@id": "https://noocodec.dev/ontology/dag/outputs"
},
"node": {
"@id": "https://noocodec.dev/ontology/dag/node"
},
"dag": {
"@id": "https://noocodec.dev/ontology/dag/dag"
},
"body": {
"@id": "https://noocodec.dev/ontology/dag/body"
},
"source": {
"@id": "https://noocodec.dev/ontology/dag/source"
},
"sources": {
"@id": "https://noocodec.dev/ontology/dag/sources",
"@container": "@index"
},
"itemKey": {
"@id": "https://noocodec.dev/ontology/dag/itemKey"
},
"execution": {
"@id": "https://noocodec.dev/ontology/dag/execution"
},
"concurrency": {
"@id": "https://noocodec.dev/ontology/dag/concurrency"
},
"throttle": {
"@id": "https://noocodec.dev/ontology/dag/throttle"
},
"reservoir": {
"@id": "https://noocodec.dev/ontology/dag/reservoir"
},
"gather": {
"@id": "https://noocodec.dev/ontology/dag/gather"
},
"dagReference": {
"@id": "https://noocodec.dev/ontology/dag/dagReference",
"@type": "@id"
},
"DagReference": {
"@id": "https://noocodec.dev/ontology/dag/DagReference"
},
"from": {
"@id": "https://noocodec.dev/ontology/dag/from"
},
"path": {
"@id": "https://noocodec.dev/ontology/dag/path"
},
"candidates": {
"@id": "https://noocodec.dev/ontology/dag/candidates",
"@container": "@set"
},
"candidateDag": {
"@id": "https://noocodec.dev/ontology/dag/candidateDag",
"@type": "@id"
},
"selectedDag": {
"@id": "https://noocodec.dev/ontology/dag/selectedDag",
"@type": "@id"
},
"resultField": {
"@id": "https://noocodec.dev/ontology/dag/resultField"
},
"policy": {
"@id": "https://noocodec.dev/ontology/dag/policy"
},
"reducer": {
"@id": "https://noocodec.dev/ontology/dag/reducer"
},
"outcome": {
"@id": "https://noocodec.dev/ontology/dag/outcome"
},
"phase": {
"@id": "https://noocodec.dev/ontology/dag/phase"
},
"stateMapping": {
"@id": "https://noocodec.dev/ontology/dag/stateMapping"
},
"container": {
"@id": "https://noocodec.dev/ontology/dag/container"
},
"DAG": {
"@id": "https://noocodec.dev/ontology/dag/DAG"
},
"Placement": {
"@id": "https://noocodec.dev/ontology/dag/Placement"
},
"SingleNode": {
"@id": "https://noocodec.dev/ontology/dag/SingleNode"
},
"ScatterNode": {
"@id": "https://noocodec.dev/ontology/dag/ScatterNode"
},
"EmbeddedDAGNode": {
"@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
},
"GatherNode": {
"@id": "https://noocodec.dev/ontology/dag/GatherNode"
},
"TerminalNode": {
"@id": "https://noocodec.dev/ontology/dag/TerminalNode"
},
"PhaseNode": {
"@id": "https://noocodec.dev/ontology/dag/PhaseNode"
}
},
"@id": "urn:noocodec:dag:the-archivist",
"@type": "DAG",
"name": "the-archivist",
"version": "6.0",
"entrypoints": {
"main": "urn:noocodec:dag:the-archivist/node/park-for-input"
},
"nodes": [
{
"@id": "urn:noocodec:dag:the-archivist/node/setup",
"@type": "PhaseNode",
"name": "setup",
"node": "urn:noocodec:node:pre-run-setup",
"phase": "pre"
},
{
"@id": "urn:noocodec:dag:the-archivist/node/park-for-input",
"@type": "SingleNode",
"name": "park-for-input",
"node": "urn:noocodec:node:park-for-input",
"outputs": {
"parked": "urn:noocodec:dag:the-archivist/node/park-for-input",
"resumed": "urn:noocodec:dag:the-archivist/node/recall-context"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recall-context",
"@type": "SingleNode",
"name": "recall-context",
"node": "urn:noocodec:node:recall-context",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/classify-intent"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/classify-intent",
"@type": "SingleNode",
"name": "classify-intent",
"node": "urn:noocodec:node:classify-intent",
"outputs": {
"lookup-author": "urn:noocodec:dag:the-archivist/node/author-search",
"find-reviews": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"describe-book": "urn:noocodec:dag:the-archivist/node/describe-extract",
"recommend-similar": "urn:noocodec:dag:the-archivist/node/recommend-similar",
"recall-memories": "urn:noocodec:dag:the-archivist/node/memory-recall",
"on-topic": "urn:noocodec:dag:the-archivist/node/on-topic-search",
"recommend-top-rated": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"off-topic": "urn:noocodec:dag:the-archivist/node/decline-off-topic",
"retry": "urn:noocodec:dag:the-archivist/node/classify-intent",
"salvage": "urn:noocodec:dag:the-archivist/node/classify-intent-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/classify-intent-salvage",
"@type": "SingleNode",
"name": "classify-intent-salvage",
"node": "urn:noocodec:node:classify-intent-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/on-topic-search"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/on-topic-search",
"@type": "EmbeddedDAGNode",
"name": "on-topic-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/compose-loop",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/author-search",
"@type": "EmbeddedDAGNode",
"name": "author-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/group-by-year",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/group-by-year",
"@type": "SingleNode",
"name": "group-by-year",
"node": "urn:noocodec:node:group-by-year",
"outputs": {
"ordered": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"@type": "SingleNode",
"name": "reviews-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/reviews-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/reviews-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-extract-salvage",
"@type": "SingleNode",
"name": "reviews-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"@type": "SingleNode",
"name": "reviews-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-decide-tools-salvage",
"@type": "SingleNode",
"name": "reviews-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-build-worksets",
"@type": "SingleNode",
"name": "reviews-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/reviews-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-scatter",
"@type": "ScatterNode",
"name": "reviews-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"error": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"empty": "urn:noocodec:dag:the-archivist/node/reviews-rank"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-gather",
"@type": "GatherNode",
"name": "reviews-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/reviews-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"error": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"empty": "urn:noocodec:dag:the-archivist/node/reviews-rank"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-rank",
"@type": "SingleNode",
"name": "reviews-rank",
"node": "urn:noocodec:node:rank-by-rating",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/reviews-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-merge",
"@type": "SingleNode",
"name": "reviews-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/reviews-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-record",
"@type": "SingleNode",
"name": "reviews-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/reviews-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-gate",
"@type": "SingleNode",
"name": "reviews-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/reviews-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/reviews-recall",
"@type": "SingleNode",
"name": "reviews-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"@type": "SingleNode",
"name": "recommend-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/recommend-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/recommend-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-extract-salvage",
"@type": "SingleNode",
"name": "recommend-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"@type": "SingleNode",
"name": "recommend-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-decide-tools-salvage",
"@type": "SingleNode",
"name": "recommend-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-build-worksets",
"@type": "SingleNode",
"name": "recommend-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/recommend-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-scatter",
"@type": "ScatterNode",
"name": "recommend-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"error": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"empty": "urn:noocodec:dag:the-archivist/node/recommend-rank"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-gather",
"@type": "GatherNode",
"name": "recommend-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/recommend-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"error": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"empty": "urn:noocodec:dag:the-archivist/node/recommend-rank"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-rank",
"@type": "SingleNode",
"name": "recommend-rank",
"node": "urn:noocodec:node:rank-by-rating",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/recommend-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-merge",
"@type": "SingleNode",
"name": "recommend-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/recommend-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-record",
"@type": "SingleNode",
"name": "recommend-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/recommend-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-gate",
"@type": "SingleNode",
"name": "recommend-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/recommend-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-recall",
"@type": "SingleNode",
"name": "recommend-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-extract",
"@type": "SingleNode",
"name": "describe-extract",
"node": "urn:noocodec:node:extract-query",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"retry": "urn:noocodec:dag:the-archivist/node/describe-extract",
"salvage": "urn:noocodec:dag:the-archivist/node/describe-extract-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-extract-salvage",
"@type": "SingleNode",
"name": "describe-extract-salvage",
"node": "urn:noocodec:node:extract-query-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/describe-decide-tools"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"@type": "SingleNode",
"name": "describe-decide-tools",
"node": "urn:noocodec:node:decide-tools",
"outputs": {
"tools": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"no-tools": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"retry": "urn:noocodec:dag:the-archivist/node/describe-decide-tools",
"salvage": "urn:noocodec:dag:the-archivist/node/describe-decide-tools-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-decide-tools-salvage",
"@type": "SingleNode",
"name": "describe-decide-tools-salvage",
"node": "urn:noocodec:node:decide-tools-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/describe-build-worksets"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-build-worksets",
"@type": "SingleNode",
"name": "describe-build-worksets",
"node": "urn:noocodec:node:build-book-worksets",
"outputs": {
"ready": "urn:noocodec:dag:the-archivist/node/describe-scatter"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-scatter",
"@type": "ScatterNode",
"name": "describe-scatter",
"source": "bookWorksets",
"body": {
"dag": {
"@type": "DagReference",
"from": "item",
"path": "dagIri",
"candidates": [
"urn:noocodec:tool:web_search_books",
"urn:noocodec:tool:google_books_search",
"urn:noocodec:tool:subject_search",
"urn:noocodec:tool:wikipedia_summary"
]
}
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-gather",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"itemKey": "currentItem",
"reducer": "any-success",
"execution": {
"mode": "item",
"concurrency": 4
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-gather",
"@type": "GatherNode",
"name": "describe-gather",
"sources": {
"urn:noocodec:dag:the-archivist/node/describe-scatter": {}
},
"gather": {
"strategy": "tool-candidate-merge"
},
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/describe-pick",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-pick",
"@type": "SingleNode",
"name": "describe-pick",
"node": "urn:noocodec:node:pick-best-match",
"outputs": {
"picked": "urn:noocodec:dag:the-archivist/node/describe-merge"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-merge",
"@type": "SingleNode",
"name": "describe-merge",
"node": "urn:noocodec:node:merge-candidates",
"outputs": {
"ranked": "urn:noocodec:dag:the-archivist/node/describe-record",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-record",
"@type": "SingleNode",
"name": "describe-record",
"node": "urn:noocodec:node:record-findings",
"outputs": {
"recorded": "urn:noocodec:dag:the-archivist/node/describe-gate"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-gate",
"@type": "SingleNode",
"name": "describe-gate",
"node": "urn:noocodec:node:has-citations-gate",
"outputs": {
"pass": "urn:noocodec:dag:the-archivist/node/describe-recall",
"fail": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/describe-recall",
"@type": "SingleNode",
"name": "describe-recall",
"node": "urn:noocodec:node:recall-past-visits",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-loop"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/recommend-similar",
"@type": "SingleNode",
"name": "recommend-similar",
"node": "urn:noocodec:node:recommend-similar",
"outputs": {
"seeded": "urn:noocodec:dag:the-archivist/node/similar-search",
"empty": "urn:noocodec:dag:the-archivist/node/compose-empty"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/similar-search",
"@type": "EmbeddedDAGNode",
"name": "similar-search",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/compose-loop",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:book-search-scatter",
"stateMapping": {
"output": {
"terms": "terms",
"toolPlan": "toolPlan",
"candidates": "candidates",
"shortlist": "shortlist",
"priorContext": "priorContext",
"failureCause": "failureCause"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-loop",
"@type": "EmbeddedDAGNode",
"name": "compose-loop",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"error": "urn:noocodec:dag:the-archivist/node/compose-empty"
},
"dag": "urn:noocodec:dag:compose-retry-loop",
"stateMapping": {
"output": {
"draft": "draft",
"approvalState": "approvalState"
}
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"@type": "SingleNode",
"name": "respond-to-visitor",
"node": "urn:noocodec:node:respond-to-visitor",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/end"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/memory-recall",
"@type": "SingleNode",
"name": "memory-recall",
"node": "urn:noocodec:node:recall-memories",
"outputs": {
"recalled": "urn:noocodec:dag:the-archivist/node/compose-memory-recall"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-memory-recall",
"@type": "SingleNode",
"name": "compose-memory-recall",
"node": "urn:noocodec:node:compose-memory-response",
"outputs": {
"drafted": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"retry": "urn:noocodec:dag:the-archivist/node/compose-memory-recall",
"salvage": "urn:noocodec:dag:the-archivist/node/compose-memory-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-memory-salvage",
"@type": "SingleNode",
"name": "compose-memory-salvage",
"node": "urn:noocodec:node:compose-memory-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/respond-to-visitor"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/decline-off-topic",
"@type": "SingleNode",
"name": "decline-off-topic",
"node": "urn:noocodec:node:decline-off-topic",
"outputs": {
"success": "urn:noocodec:dag:the-archivist/node/end"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-empty",
"@type": "SingleNode",
"name": "compose-empty",
"node": "urn:noocodec:node:compose-empty",
"outputs": {
"drafted": "urn:noocodec:dag:the-archivist/node/respond-to-visitor",
"retry": "urn:noocodec:dag:the-archivist/node/compose-empty",
"salvage": "urn:noocodec:dag:the-archivist/node/compose-empty-salvage"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/compose-empty-salvage",
"@type": "SingleNode",
"name": "compose-empty-salvage",
"node": "urn:noocodec:node:compose-empty-salvage",
"outputs": {
"done": "urn:noocodec:dag:the-archivist/node/respond-to-visitor"
}
},
{
"@id": "urn:noocodec:dag:the-archivist/node/end",
"@type": "TerminalNode",
"name": "end",
"outcome": "completed"
}
]
}Mermaid source
%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
%% the-archivist (v6.0)
entry_main(["main"])
entry_main --> urn_noocodec_dag_the-archivist/node/park-for-input
urn_noocodec_dag_the-archivist/node/setup(["setup (pre)"])
urn_noocodec_dag_the-archivist/node/park-for-input["park-for-input"]
urn_noocodec_dag_the-archivist/node/park-for-input -->|parked| urn_noocodec_dag_the-archivist/node/park-for-input
urn_noocodec_dag_the-archivist/node/park-for-input -->|resumed| urn_noocodec_dag_the-archivist/node/recall-context
urn_noocodec_dag_the-archivist/node/recall-context["recall-context"]
urn_noocodec_dag_the-archivist/node/recall-context -->|recalled| urn_noocodec_dag_the-archivist/node/classify-intent
urn_noocodec_dag_the-archivist/node/classify-intent["classify-intent"]
urn_noocodec_dag_the-archivist/node/classify-intent -->|lookup-author| urn_noocodec_dag_the-archivist/node/author-search
urn_noocodec_dag_the-archivist/node/classify-intent -->|find-reviews| urn_noocodec_dag_the-archivist/node/reviews-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|describe-book| urn_noocodec_dag_the-archivist/node/describe-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|recommend-similar| urn_noocodec_dag_the-archivist/node/recommend-similar
urn_noocodec_dag_the-archivist/node/classify-intent -->|recall-memories| urn_noocodec_dag_the-archivist/node/memory-recall
urn_noocodec_dag_the-archivist/node/classify-intent -->|on-topic| urn_noocodec_dag_the-archivist/node/on-topic-search
urn_noocodec_dag_the-archivist/node/classify-intent -->|recommend-top-rated| urn_noocodec_dag_the-archivist/node/recommend-extract
urn_noocodec_dag_the-archivist/node/classify-intent -->|off-topic| urn_noocodec_dag_the-archivist/node/decline-off-topic
urn_noocodec_dag_the-archivist/node/classify-intent -->|retry| urn_noocodec_dag_the-archivist/node/classify-intent
urn_noocodec_dag_the-archivist/node/classify-intent -->|salvage| urn_noocodec_dag_the-archivist/node/classify-intent-salvage
urn_noocodec_dag_the-archivist/node/classify-intent-salvage["classify-intent-salvage"]
urn_noocodec_dag_the-archivist/node/classify-intent-salvage -->|done| urn_noocodec_dag_the-archivist/node/on-topic-search
urn_noocodec_dag_the-archivist/node/on-topic-search[["on-topic-search"]]
urn_noocodec_dag_the-archivist/node/on-topic-search -->|success| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/on-topic-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/author-search[["author-search"]]
urn_noocodec_dag_the-archivist/node/author-search -->|success| urn_noocodec_dag_the-archivist/node/group-by-year
urn_noocodec_dag_the-archivist/node/author-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/group-by-year["group-by-year"]
urn_noocodec_dag_the-archivist/node/group-by-year -->|ordered| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/reviews-extract["reviews-extract"]
urn_noocodec_dag_the-archivist/node/reviews-extract -->|success| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-extract -->|retry| urn_noocodec_dag_the-archivist/node/reviews-extract
urn_noocodec_dag_the-archivist/node/reviews-extract -->|salvage| urn_noocodec_dag_the-archivist/node/reviews-extract-salvage
urn_noocodec_dag_the-archivist/node/reviews-extract-salvage["reviews-extract-salvage"]
urn_noocodec_dag_the-archivist/node/reviews-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-decide-tools["reviews-decide-tools"]
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/reviews-decide-tools
urn_noocodec_dag_the-archivist/node/reviews-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage["reviews-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/reviews-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/reviews-build-worksets
urn_noocodec_dag_the-archivist/node/reviews-build-worksets["reviews-build-worksets"]
urn_noocodec_dag_the-archivist/node/reviews-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/reviews-scatter
urn_noocodec_dag_the-archivist/node/reviews-scatter[/"reviews-scatter"/]
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|success| urn_noocodec_dag_the-archivist/node/reviews-gather
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|error| urn_noocodec_dag_the-archivist/node/reviews-gather
urn_noocodec_dag_the-archivist/node/reviews-scatter -->|empty| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather{"reviews-gather"}
urn_noocodec_dag_the-archivist/node/reviews-gather -->|success| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather -->|error| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-gather -->|empty| urn_noocodec_dag_the-archivist/node/reviews-rank
urn_noocodec_dag_the-archivist/node/reviews-rank["reviews-rank"]
urn_noocodec_dag_the-archivist/node/reviews-rank -->|ranked| urn_noocodec_dag_the-archivist/node/reviews-merge
urn_noocodec_dag_the-archivist/node/reviews-merge["reviews-merge"]
urn_noocodec_dag_the-archivist/node/reviews-merge -->|ranked| urn_noocodec_dag_the-archivist/node/reviews-record
urn_noocodec_dag_the-archivist/node/reviews-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/reviews-record["reviews-record"]
urn_noocodec_dag_the-archivist/node/reviews-record -->|recorded| urn_noocodec_dag_the-archivist/node/reviews-gate
urn_noocodec_dag_the-archivist/node/reviews-gate["reviews-gate"]
urn_noocodec_dag_the-archivist/node/reviews-gate -->|pass| urn_noocodec_dag_the-archivist/node/reviews-recall
urn_noocodec_dag_the-archivist/node/reviews-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/reviews-recall["reviews-recall"]
urn_noocodec_dag_the-archivist/node/reviews-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/recommend-extract["recommend-extract"]
urn_noocodec_dag_the-archivist/node/recommend-extract -->|success| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-extract -->|retry| urn_noocodec_dag_the-archivist/node/recommend-extract
urn_noocodec_dag_the-archivist/node/recommend-extract -->|salvage| urn_noocodec_dag_the-archivist/node/recommend-extract-salvage
urn_noocodec_dag_the-archivist/node/recommend-extract-salvage["recommend-extract-salvage"]
urn_noocodec_dag_the-archivist/node/recommend-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-decide-tools["recommend-decide-tools"]
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/recommend-decide-tools
urn_noocodec_dag_the-archivist/node/recommend-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage["recommend-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/recommend-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/recommend-build-worksets
urn_noocodec_dag_the-archivist/node/recommend-build-worksets["recommend-build-worksets"]
urn_noocodec_dag_the-archivist/node/recommend-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/recommend-scatter
urn_noocodec_dag_the-archivist/node/recommend-scatter[/"recommend-scatter"/]
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|success| urn_noocodec_dag_the-archivist/node/recommend-gather
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|error| urn_noocodec_dag_the-archivist/node/recommend-gather
urn_noocodec_dag_the-archivist/node/recommend-scatter -->|empty| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather{"recommend-gather"}
urn_noocodec_dag_the-archivist/node/recommend-gather -->|success| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather -->|error| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-gather -->|empty| urn_noocodec_dag_the-archivist/node/recommend-rank
urn_noocodec_dag_the-archivist/node/recommend-rank["recommend-rank"]
urn_noocodec_dag_the-archivist/node/recommend-rank -->|ranked| urn_noocodec_dag_the-archivist/node/recommend-merge
urn_noocodec_dag_the-archivist/node/recommend-merge["recommend-merge"]
urn_noocodec_dag_the-archivist/node/recommend-merge -->|ranked| urn_noocodec_dag_the-archivist/node/recommend-record
urn_noocodec_dag_the-archivist/node/recommend-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/recommend-record["recommend-record"]
urn_noocodec_dag_the-archivist/node/recommend-record -->|recorded| urn_noocodec_dag_the-archivist/node/recommend-gate
urn_noocodec_dag_the-archivist/node/recommend-gate["recommend-gate"]
urn_noocodec_dag_the-archivist/node/recommend-gate -->|pass| urn_noocodec_dag_the-archivist/node/recommend-recall
urn_noocodec_dag_the-archivist/node/recommend-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/recommend-recall["recommend-recall"]
urn_noocodec_dag_the-archivist/node/recommend-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/describe-extract["describe-extract"]
urn_noocodec_dag_the-archivist/node/describe-extract -->|success| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-extract -->|retry| urn_noocodec_dag_the-archivist/node/describe-extract
urn_noocodec_dag_the-archivist/node/describe-extract -->|salvage| urn_noocodec_dag_the-archivist/node/describe-extract-salvage
urn_noocodec_dag_the-archivist/node/describe-extract-salvage["describe-extract-salvage"]
urn_noocodec_dag_the-archivist/node/describe-extract-salvage -->|done| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-decide-tools["describe-decide-tools"]
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|tools| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|no-tools| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|retry| urn_noocodec_dag_the-archivist/node/describe-decide-tools
urn_noocodec_dag_the-archivist/node/describe-decide-tools -->|salvage| urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage
urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage["describe-decide-tools-salvage"]
urn_noocodec_dag_the-archivist/node/describe-decide-tools-salvage -->|done| urn_noocodec_dag_the-archivist/node/describe-build-worksets
urn_noocodec_dag_the-archivist/node/describe-build-worksets["describe-build-worksets"]
urn_noocodec_dag_the-archivist/node/describe-build-worksets -->|ready| urn_noocodec_dag_the-archivist/node/describe-scatter
urn_noocodec_dag_the-archivist/node/describe-scatter[/"describe-scatter"/]
urn_noocodec_dag_the-archivist/node/describe-scatter -->|success| urn_noocodec_dag_the-archivist/node/describe-gather
urn_noocodec_dag_the-archivist/node/describe-scatter -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-scatter -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-gather{"describe-gather"}
urn_noocodec_dag_the-archivist/node/describe-gather -->|success| urn_noocodec_dag_the-archivist/node/describe-pick
urn_noocodec_dag_the-archivist/node/describe-gather -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-gather -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-pick["describe-pick"]
urn_noocodec_dag_the-archivist/node/describe-pick -->|picked| urn_noocodec_dag_the-archivist/node/describe-merge
urn_noocodec_dag_the-archivist/node/describe-merge["describe-merge"]
urn_noocodec_dag_the-archivist/node/describe-merge -->|ranked| urn_noocodec_dag_the-archivist/node/describe-record
urn_noocodec_dag_the-archivist/node/describe-merge -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-record["describe-record"]
urn_noocodec_dag_the-archivist/node/describe-record -->|recorded| urn_noocodec_dag_the-archivist/node/describe-gate
urn_noocodec_dag_the-archivist/node/describe-gate["describe-gate"]
urn_noocodec_dag_the-archivist/node/describe-gate -->|pass| urn_noocodec_dag_the-archivist/node/describe-recall
urn_noocodec_dag_the-archivist/node/describe-gate -->|fail| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/describe-recall["describe-recall"]
urn_noocodec_dag_the-archivist/node/describe-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/recommend-similar["recommend-similar"]
urn_noocodec_dag_the-archivist/node/recommend-similar -->|seeded| urn_noocodec_dag_the-archivist/node/similar-search
urn_noocodec_dag_the-archivist/node/recommend-similar -->|empty| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/similar-search[["similar-search"]]
urn_noocodec_dag_the-archivist/node/similar-search -->|success| urn_noocodec_dag_the-archivist/node/compose-loop
urn_noocodec_dag_the-archivist/node/similar-search -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/compose-loop[["compose-loop"]]
urn_noocodec_dag_the-archivist/node/compose-loop -->|success| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-loop -->|error| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/respond-to-visitor["respond-to-visitor"]
urn_noocodec_dag_the-archivist/node/respond-to-visitor -->|success| urn_noocodec_dag_the-archivist/node/end
urn_noocodec_dag_the-archivist/node/memory-recall["memory-recall"]
urn_noocodec_dag_the-archivist/node/memory-recall -->|recalled| urn_noocodec_dag_the-archivist/node/compose-memory-recall
urn_noocodec_dag_the-archivist/node/compose-memory-recall["compose-memory-recall"]
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|drafted| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|retry| urn_noocodec_dag_the-archivist/node/compose-memory-recall
urn_noocodec_dag_the-archivist/node/compose-memory-recall -->|salvage| urn_noocodec_dag_the-archivist/node/compose-memory-salvage
urn_noocodec_dag_the-archivist/node/compose-memory-salvage["compose-memory-salvage"]
urn_noocodec_dag_the-archivist/node/compose-memory-salvage -->|done| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/decline-off-topic["decline-off-topic"]
urn_noocodec_dag_the-archivist/node/decline-off-topic -->|success| urn_noocodec_dag_the-archivist/node/end
urn_noocodec_dag_the-archivist/node/compose-empty["compose-empty"]
urn_noocodec_dag_the-archivist/node/compose-empty -->|drafted| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/compose-empty -->|retry| urn_noocodec_dag_the-archivist/node/compose-empty
urn_noocodec_dag_the-archivist/node/compose-empty -->|salvage| urn_noocodec_dag_the-archivist/node/compose-empty-salvage
urn_noocodec_dag_the-archivist/node/compose-empty-salvage["compose-empty-salvage"]
urn_noocodec_dag_the-archivist/node/compose-empty-salvage -->|done| urn_noocodec_dag_the-archivist/node/respond-to-visitor
urn_noocodec_dag_the-archivist/node/end((("end")))Serializing the checkpoint
ts
// Checkpoint.capture() returns a Checkpoint instance.
// cursor !== null here because we aborted mid-run.
const checkpoint = await Checkpoint.capture('urn:noocodec:dag:count', partial);
const persisted = checkpoint.toJson(); // → JSON string (store in DB, file, etc.)ckpt.toJson() is JSON.stringify(ckpt.data, null, 2). The output is a stable JSON document; persist it however the system stores other JSON: file, database column, object store, etc.
Loading and rehydrating state
ts
// Parse the persisted JSON back to an unknown value, then load into a Checkpoint.
const ckpt = Checkpoint.load(JSON.parse(persisted));
// restoreState maps the snapshot back to a typed CountingState instance.
// Consumers supply their own restore fn so the checkpoint module never
// imports domain state classes.
const { state, dagName, cursor } = ckpt.restoreState(
CheckpointRestoreAdapter.wrap((snap) => CountingState.restore(snap)), // rehydrates domain fields via restoreData()
);
process.stdout.write(` restored: count=${state.count} cursor="${cursor}"\n`);Checkpoint.load(raw) validates the unknown value against CheckpointDataSchema (Ajv 2020-12) before touching any fields. An invalid or stale payload throws ValidationError. ckpt.restoreState(adapter) accepts a CheckpointRestoreAdapter<TState>; wrap a plain factory function with CheckpointRestoreAdapter.wrap(fn) from @studnicky/dagonizer/checkpoint.
Resuming execution
ts
// Resume from cursor 'b'; only nodes b and c execute.
const resumed = await dispatcher.resume(dagName, state, cursor);
process.stdout.write(` resumed: count=${resumed.state.count} log=${JSON.stringify(resumed.state.log)}\n`);
process.stdout.write('\nLesson: cursor marks where to resume; snapshotData/restoreData\n');
process.stdout.write(' persist domain fields across the serialisation boundary.\n');
process.stdout.write(' Final count=3 and log length=3: identical to a full run.\n');dispatcher.resume continues the flow at the cursor and runs the remaining nodes. The dispatcher does not re-execute completed nodes; the recorded executedNodes and skippedNodes survive the round-trip.
Named stores ride along
Checkpoint.capture(dagIri, result, { stores, execution }) snapshots named stores into the checkpoint envelope alongside the state, and ckpt.restoreStores(map, { execution }) repopulates fresh instances on resume. Pass the DAG IRI/CURIE string as the first argument. The execution policy uses the same batch executor options as tool and node batch execution, giving remote or expensive stores explicit concurrency, throttle, and timing controls. The following shows the full abort-capture-restore-resume cycle with a MemoryStore riding along in the checkpoint:
ts
{
const logStore = new MemoryStore();
const dispatcher = new Dagonizer<NodeStateBase>();
dispatcher.registerNode(new StepANode(logStore));
dispatcher.registerNode(new StepBNode(logStore));
dispatcher.registerNode(new ChildStepNode(logStore));
dispatcher.registerDAG(childDag);
dispatcher.registerDAG(parentDag);
// Abort mid-run: abort after step-a to produce a checkpoint-worthy cursor.
const ctl = new AbortController();
const execution = dispatcher.execute('urn:noocodec:dag:main-flow', new NodeStateBase(), { "signal": ctl.signal });
let seen = 0;
for await (const _event of execution) {
seen++;
if (seen === 1) ctl.abort(new Error('checkpoint'));
}
const partial = await execution;
if (partial.cursor === null) {
process.stdout.write('\nPart 2: run completed before abort; no cursor\n');
} else {
// Capture checkpoint: snapshot the store alongside the parent state.
const ckpt = await Checkpoint.capture('urn:noocodec:dag:main-flow', partial, { "stores": { "log": logStore } });
const json = ckpt.toJson();
process.stdout.write('\nPart 2: Checkpoint captured:\n');
process.stdout.write(` cursor = "${partial.cursor}"\n`);
process.stdout.write(` log at capture = "${await logStore.get('entries') ?? ''}"\n`);
// Resume: restore store from checkpoint, then resume execution.
const freshLog = new MemoryStore();
const ckpt2 = Checkpoint.load(JSON.parse(json));
await ckpt2.restoreStores({ "log": freshLog });
const restoredEntries = await freshLog.get('entries') ?? '';
process.stdout.write(` log after restoreStores = "${restoredEntries}"\n`);
const resumeDispatcher = new Dagonizer<NodeStateBase>();
resumeDispatcher.registerNode(new StepANode(freshLog));
resumeDispatcher.registerNode(new StepBNode(freshLog));
resumeDispatcher.registerNode(new ChildStepNode(freshLog));
resumeDispatcher.registerDAG(childDag);
resumeDispatcher.registerDAG(parentDag);
const { dagName, state, cursor } = ckpt2.restoreState(
CheckpointRestoreAdapter.wrap((snap) => NodeStateBase.restore(snap)),
);
await resumeDispatcher.resume(dagName, state, cursor);
const finalEntries = await freshLog.get('entries') ?? '';
process.stdout.write(` log after resume = "${finalEntries}"\n`);
// → "step-a,child-step,step-b" (all three present, none duplicated)
}
}Both take Record<string, Snapshottable>: the capability, not the key-value Store surface. A non-KV backing (an RDF triple store, a vector index) checkpoints by implementing snapshot() / restore() only. A name present in the checkpoint but absent from the restore map throws DAGError; an extra name in the map is a no-op. See Store, Snapshottable.
NodeStateBase.snapshot() and snapshotData()
snapshot() captures metadata, warnings, and the retry budget (retries). Engine errors are intentionally excluded from the snapshot — they flow via outcome.errors as the single authoritative channel. Domain fields are excluded unless the subclass overrides snapshotData():
ts
export class CountingState extends NodeStateBase {
count = 0;
log: string[] = [];
/**
* Serialize domain fields into a plain JSON-serialisable object.
* Called by Checkpoint.capture() to capture state at the abort point.
*/
protected override snapshotData(): JsonObjectType {
return { "count": this.count, "log": [...this.log] };
}
/**
* Restore domain fields from a previously-captured snapshot.
* Called by CountingState.restore() after the parse step.
*/
protected override restoreData(snapshot: JsonObjectType): void {
const c = snapshot['count'];
if (typeof c === 'number') this.count = c;
const l = snapshot['log'];
if (Array.isArray(l)) this.log = l.filter((x): x is string => typeof x === 'string');
}
}restoreData is called by NodeStateBase.restore(snap). The static restore method is typed with this-polymorphism so subclasses return the correct instance type.
CheckpointStore: composing with persistence
CheckpointStore is the adapter contract for persistence backends. ckpt.persist(store, key) and Checkpoint.recall(store, key) compose the codec with a store so save and resume become a single call per side.
ts
const store1 = new MemoryCheckpointStore();
const ckpt = await Checkpoint.capture('urn:noocodec:dag:pipeline', partial);
await ckpt.persist(store1, CHECKPOINT_KEY);
process.stdout.write(`[checkpoint] persisted to MemoryCheckpointStore under key "${CHECKPOINT_KEY}"\n`);
process.stdout.write(`[checkpoint] store.size=${String(store1.size)}\n`);
// Print the persisted JSON for inspection
const raw = await store1.load(CHECKPOINT_KEY);
if (raw === undefined || raw === null) throw new Error(`No checkpoint data found for key "${CHECKPOINT_KEY}"`);
const parsed: unknown = JSON.parse(raw);
if (typeof parsed !== 'object' || parsed === null || !('dagName' in parsed) || !('cursor' in parsed)) {
throw new Error('Checkpoint data missing expected fields');
}
process.stdout.write(`[checkpoint] dagName="${String(parsed.dagName)}" cursor="${String(parsed.cursor)}"\n\n`);
// ── Step 4: fresh dispatcher + same store (simulates process restart) ─────────
const dispatcher2 = PipelineDispatcher.make();
// In production this would be a new store pointing at the same persistence
// backend; here we reuse store1 to avoid I/O.
// ── Step 5: recall, restore, resume ──────────────────────────────────────────
const recalled = await Checkpoint.recall(store1, CHECKPOINT_KEY);
if (recalled === null) {
throw new Error(`No checkpoint found under key "${CHECKPOINT_KEY}"`);
}
const { state, dagName, cursor } = recalled.restoreState(
CheckpointRestoreAdapter.wrap((snap) => PipelineState.restore(snap)),
);
process.stdout.write(`[resume] restored cursor="${cursor}" tally=${String(state.tally)}\n`);
process.stdout.write(`[resume] trail so far: ${JSON.stringify(state.trail)}\n`);
const resumed = await dispatcher2.resume(dagName, state, cursor);MemoryCheckpointStore is for tests and demos. Production deployments implement CheckpointStore against a database, object store, or filesystem (see persistence).
Scatter resume: per-item progress
A ScatterNode with a source records per-item progress on state.metadata so a checkpointed run does not re-execute already-completed clones on resume. This matters most for long scatter runs whose items hit external APIs or LLMs: re-running a 200-item batch from the top after a restart would burn quota and waste hours.
Reserved metadata key
SCATTER_PROGRESS_KEY is exported from @studnicky/dagonizer as the string '__dagonizer_scatter_progress__'.
Application nodes must not write to this key. It is engine-internal and may be overwritten or cleared between batch boundaries by executeScatter.
The stored shape is a record keyed by the scatter placement's name, so multiple ScatterNode placements in one DAG keep independent progress entries:
ts
// ScatterProgress is a discriminated union on `mode`.
// `retained` mode stores full per-item acked results (used by map/append/collect strategies).
// `bounded` mode stores a watermark + ahead-acked indices (used by partition/discard strategies).
declare const stored: StoredScatterProgressType;
declare const progress: ScatterProgressType;
// Fields common to both branches:
const name: string = progress.placementName;
const inbox = progress.inbox; // ScatterInboxItemType[] — items pulled but not yet acked
// Narrow on `mode` to access branch-specific fields:
if (progress.mode === 'retained') {
const results = progress.ackedResults; // ScatterAckedResultType[] — completed items
void results;
} else {
// progress.mode === 'bounded'
const mark: number = progress.watermark; // highest contiguously-acked index
const ahead = progress.aheadAcked; // { index, output }[] — acked items above watermark
const tally = progress.outcomeTally; // Record<string, number> — per-output counts
void mark; void ahead; void tally;
}
void name; void inbox;
void stored;
export {};Lifecycle
- On entry:
executeScatterreadsstate.metadata[SCATTER_PROGRESS_KEY]?.[scatter.name]. Items already recorded inackedResults(retained mode) or at or belowwatermark(bounded mode) are skipped; their recorded outputs rehydrate the gather accumulator without re-executing the body. - Per-batch write: after each
Promise.all(batchPromises)resolves, the dispatcher updates the placement's entry with the batch's completed item records. Writes happen once per batch (not per item) to keep the metadata update serialised across concurrent item promises. - Pre-gather clear: once every batch drains, the placement's entry is removed before the gather strategy runs. Gather always starts from a clean slate; subsequent re-runs of the same
ScatterNode(such as inside a loop) do not see stale bookkeeping.
Index semantics on resume
Indices refer to positions in the source array at the time of resume, not the array as it stood when the checkpoint was captured. If the application rewrites the source array between checkpoint and resume, the resumed scatter trusts the persisted indices verbatim; items 0 and 1 are skipped even when the array has been re-sliced or reordered.
Treat the scatter's source array as immutable while a scatter checkpoint is live. If the source must change between runs, clear the entry under SCATTER_PROGRESS_KEY[scatter.name] before calling dispatcher.resume() so the scatter re-executes every item against the new source.
Snapshot round-trip
The reserved key rides along with the rest of state.metadata through NodeStateBase.snapshot() and restore(). No extra plumbing in application state classes; snapshotData() overrides do not need to touch the progress key. Checkpoint.capture() and Checkpoint.load() both preserve it intact.
Related Concepts
- Persistence - wire
ckpt.persistandCheckpoint.recallto aCheckpointStore - Cancellation - abort a flow to produce a non-null cursor worth checkpointing
- Subclassing State - override
snapshotData()andrestoreData()for domain fields - Reference: Checkpoint
- Reference: Contracts
- Demo: Example 08: Checkpoint and Resume