The Archivist
What It Is
The Archivist is a runnable demo: a real browser-executed DAG application, not a decorative diagram. A bookstore help-bot powered by Dagonizer: multi-branch DAG with hard and soft gates, parallel scouts, RAG retrieval, and a bounded compose/validate retry loop. It is the running demo every Dagonizer agent example references.
Use it to see a model-driven workflow become inspectable: model calls are nodes, tool work is routed through DAG placements, retries are visible edges, and memory is a shared store rather than a hidden callback side effect.
How It Works
The runner wires real node classes, real DAG documents, and browser UI observers together. The visual panes listen to dispatcher lifecycle events, so the page shows execution rather than replaying a canned animation.
Diagrams, Examples, and Outputs
The live demo is the main diagram. Its graph, state panes, traces, memory views, backend selectors, and outputs are all evidence from the running system.
Branches and gates
Three exit conditions, each carrying a different outcome.
| Path | Trigger | Terminal node | What happens |
|---|---|---|---|
| Off-topic hard gate | classifyIntent returns off-topic | decline-off-topic | Politely redirects the visitor to a book-related question. |
| Empty soft gate | mergeCandidates produces zero candidates | compose-empty | Composes an in-character "nothing came back" message; collects a EMPTY_SHORTLIST warning. |
| Best-effort response | validateResponse exhausts MAX_COMPOSE_ATTEMPTS | respond-to-visitor | Sends the last draft anyway; the dispatcher never throws. |
| Approved response | validateResponse returns approved | respond-to-visitor | Normal happy path. |
| Retry loop | validateResponse returns retry | back to compose-response | Bounded by the retry budget on state (state.retriesFor('compose')). |
What this proves
The Archivist proves that an LLM agent application can be an inspectable DAG: model calls are nodes, tool dispatch is embedded-DAG/scatter composition, recall uses shared memory, retries are graph edges, and the final response is a lifecycle outcome instead of an opaque callback.
Try it live below; the demo runs in your browser. The browser runner instantiates a single selected backend via ProviderInstantiator.instantiate() — the picker surfaces which provider is active. Cloud-first when keys are present (Groq, Cerebras, Gemini API, Mistral, OpenRouter), local-first when reachable (Ollama on desktop), then on-device options (Gemini Nano, WebLLM). The demo only runs against a real model: when none is reachable it shows a setup gate with links to free backends rather than fabricating a response. The browser demo provisions an on-device embedder (EmbedderProvisioner — transformers.js MiniLM, with TensorFlow.js USE and WebLLM behind it); cosine recall, hybrid ranking, and vector-similarity intent classification run client-side, using Jaccard / heuristics only when no embedder probes are available. The CLI path (runArchivist.ts) uses an LlmAdapterCascade and a separate EmbedderCascade for the same vector-similarity intent classification.
The Archivist composes reusable work through one interface: a placement points at a DAG through dag, either as a literal registered DAG IRI or as a dynamic DagReference with explicit candidates. EmbeddedDAGNode invokes one selected DAG once; ScatterNode invokes the selected DAG per source item and then routes clone output into a first-class gather placement. build-book-worksets converts the decided tool plan into a bookWorksets array where each item carries a dagIri field, the scatter resolves the body DAG through the same dag reference surface, the tool-candidate-merge gather folds each clone's output into the parent candidates, and the any-success reducer routes success when at least one tool returned results. A PhaseNode (phase: 'pre', placement display name setup) runs pre-run-setup before the entrypoint: it stamps a runId on state and clears any stale draft from a prior interrupted execution. Phase nodes are out-of-band; they do not participate in output routing.
Watch the DAG pane: each node lights cyan while executing, then settles to "completed" with the taken edge highlighted. The Memory pane mirrors state.intent, state.terms, state.shortlist, and the compose retry budget (state.retriesFor('compose')) as the dispatcher mutates them. Everything is driven by the dispatcher's onFlowStart, onNodeStart, onNodeEnd, onError, onFlowEnd hooks; there is no timer-based animation, the runner is a pure observer of the state machine.
What It Lets You Do
Use the Archivist when you want to see a complete model-backed application as a graph instead of a pile of callbacks. It demonstrates classification, tool selection, scatter fan-out, RAG-style recall, composition, validation, retry, checkpointing, and response delivery in one inspectable run.
For application teams, this page answers a practical question: what does a real Dagonizer agent look like when it has to remember, recover, route, and explain itself?
What to try
Ask for a book recommendation, an author lookup, a review-oriented query, or an off-topic question. Watch the active backend selector, the DAG pane, and the Memory pane while the same JSON-LD graph routes the turn through classification, search, compose, retry, and response paths.
Code Samples
The Archivist source is intentionally visible because this demo is the reference point for most numbered examples. Start with the top-level DAG, then drill into the reusable embedded DAGs, state, prompts, and memory model.
Compositional embedded-DAG sub-DAGs
The Archivist's DAG is composed of two reusable sub-DAGs that ship as independent components. Each is a DAG value any application can import, register, and reference via .embed(placementIri, dagIri, routes, options).
book-search-scatter: extract-query → decide-tools → recall-candidates → build-book-worksets → scatter overbookWorksetswith a dynamicDagReferencebody (tool-registry dispatch, concurrency 4,tool-candidate-mergegather,any-successreducer) → rank-candidates → merge-candidates → record-findings → has-citations-gate → recall-past-visits. Used in three intent branches (on-topic-search,author-search,similar-search); one definition, three embedded-DAG placements.compose-retry-loop: compose-response and validate-response, with a bounded retry edge back to compose and acompose-salvagerecovery node. The sub-DAG producesstate.draftand exits withsuccess; the parent DAG owns the sharedrespond-to-visitorterminal. Every successful search branch funnels through this one shared cluster.
The renderer expands both sub-DAGs inline in the diagram. Compound-graph children render inside the embedded-DAG placement box so the full topology is visible. No opaque boxes.
Reviews and describe branches are inlined in the parent DAG because they substitute rankByRating and pickBestMatch for rankCandidates respectively; the structural variation is explicit rather than hidden behind a sub-DAG parameter.
BookSearchScatterDAG
ts
/**
* BookSearchScatterDAG: reusable query-extract + tool-registry scatter cluster.
*
* Internal flow:
*
* extract-query
* └─ success ──► decide-tools
* decide-tools
* └─ (tools | no-tools) ──► recall-candidates
* recall-candidates
* └─ recalled ──► build-book-worksets
* build-book-worksets
* └─ ready ──► book-search-scatter (scatter over bookWorksets, concurrency 4)
* body: DagReference(item.dagIri) (resolves declared tool DAG IRI per item)
* book-search-gather: tool-candidate-merge (reads clone output via accessor, no cast)
* reducer: any-success (routes 'success' if any tool found results)
* └─ rank-candidates
* └─ merge-candidates
* ├─ ranked ──► record-findings
* └─ empty ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
* └─ record-findings
* └─ has-citations-gate
* ├─ pass ──► recall-past-visits ──► END (success)
* └─ fail ──► no-results (TerminalNode(failed) → parent EmbeddedDAGNode routes parent error)
*
* Outputs:
* success: query extracted, candidates found, ranked, recorded, and recalled
* error: no candidates after merge, or citations gate failed;
* signalled by the no-results TerminalNode(failed) placement so
* the parent EmbeddedDAGNode routes the parent placement to its
* 'error' branch
*
* Molecular import pattern:
* import { bookSearchScatterDAG } from './embedded-dags/BookSearchScatterDAG.ts';
* const nodes = ArchivistNodes.build(services);
* dispatcher.registerBundle(toolRegistry.bundle<ArchivistServices>());
* dispatcher.registerBundle({ nodes: nodes.bookSearchScatterNodes, dags: [bookSearchScatterDAG] });
*
* The sub-DAG reads `state.query` directly (no input stateMapping; the field
* names already align with the parent). Each parent placement supplies an
* `outputs` stateMapping that copies the fields the sub-DAG writes:
* `terms`, `toolPlan`, `candidates`, `shortlist`, `priorContext`,
* `failureCause` back onto the parent state so the downstream compose,
* group-by-year, and recall steps can read them.
*
* Three EmbeddedDAGNode placements in the parent `the-archivist` DAG reference
* this one definition. One definition, three usages:
* on-topic-search: general web book search
* author-search: author body-of-work search
* similar-search: recommend-similar search
*
* Reviews and describe branches are inlined in the parent because they use
* distinct post-scout steps (rankByRating and pickBestMatch respectively).
*/
import type { ArchivistState } from '../ArchivistState.ts';
import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';
const BOOK_SEARCH_SCATTER_DAG_IRI = 'urn:noocodec:dag:book-search-scatter';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(BOOK_SEARCH_SCATTER_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });
const BOOK_SEARCH_TOOL_DAGS = [
'urn:noocodec:tool:web_search_books',
'urn:noocodec:tool:google_books_search',
'urn:noocodec:tool:subject_search',
'urn:noocodec:tool:wikipedia_summary',
] as const;
const extractQuery = new PlaceholderNode<ArchivistState, 'success' | 'retry' | 'salvage'>('urn:noocodec:node:extract-query', ['success', 'retry', 'salvage']);
const extractQuerySalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:extract-query-salvage', ['done']);
const decideTools = new PlaceholderNode<ArchivistState, 'tools' | 'no-tools' | 'retry' | 'salvage'>('urn:noocodec:node:decide-tools', ['tools', 'no-tools', 'retry', 'salvage']);
const decideToolsSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:decide-tools-salvage', ['done']);
const recallCandidates = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-candidates', ['recalled']);
const buildBookWorksets = new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:build-book-worksets', ['ready']);
const rankCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'retry' | 'salvage'>('urn:noocodec:node:rank-candidates', ['ranked', 'retry', 'salvage']);
const rankCandidatesSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:rank-candidates-salvage', ['done']);
const mergeCandidates = new PlaceholderNode<ArchivistState, 'ranked' | 'empty'>('urn:noocodec:node:merge-candidates', ['ranked', 'empty']);
const recordFindings = new PlaceholderNode<ArchivistState, 'recorded'>('urn:noocodec:node:record-findings', ['recorded']);
const hasCitationsGate = new PlaceholderNode<ArchivistState, 'pass' | 'fail'>('urn:noocodec:node:has-citations-gate', ['pass', 'fail']);
const recallPastVisits = new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-past-visits', ['recalled']);
export const bookSearchScatterDAG: DAGType = new DAGBuilder(BOOK_SEARCH_SCATTER_DAG_IRI, '1.0', display('book-search-scatter'))
// ── 1. extract-query ─────────────────────────────────────────────────────
// LLM parses the raw visitor question into structured search terms.
// Writes state.terms for the scouts and decide-tools to consume.
// 'retry' loops back (bounded by the state retry budget); 'salvage' routes to
// a deterministic recovery node; never a fabricated term list on the node.
// #region retry-salvage-wiring
.node(placement('extract-query'), extractQuery, {
'success': placement('decide-tools'),
'retry': placement('extract-query'), // flow-shape retry loop (self-edge)
'salvage': placement('extract-query-salvage'), // recovery route
}, display('extract-query'))
.node(placement('extract-query-salvage'), extractQuerySalvage, {
'done': placement('decide-tools'), // deterministic recovery rejoins the happy path
}, display('extract-query-salvage'))
// #endregion retry-salvage-wiring
// ── 2. decide-tools ──────────────────────────────────────────────────────
// LLM decides which external sources to invoke. Both outputs route into
// recall-candidates so prior memory is loaded before scouts fire.
// 'retry' loops back (bounded); 'salvage' routes to the minimal-plan node.
.node(placement('decide-tools'), decideTools, {
'tools': placement('recall-candidates'),
'no-tools': placement('recall-candidates'),
'retry': placement('decide-tools'),
'salvage': placement('decide-tools-salvage'),
}, display('decide-tools'))
.node(placement('decide-tools-salvage'), decideToolsSalvage, {
'done': placement('recall-candidates'),
}, display('decide-tools-salvage'))
// ── 2b. recall-candidates ────────────────────────────────────────────────
// Pre-loads state.priorCandidates from memory: shortlisted books from prior
// runs whose visitor query has Jaccard >= 0.35 overlap with the current
// query. Cap 10. Always routes 'recalled', even when no prior runs match.
.node(placement('recall-candidates'), recallCandidates, {
'recalled': placement('build-book-worksets'),
}, display('recall-candidates'))
// ── 2c. build-book-worksets ──────────────────────────────────────────────
// Converts state.toolPlan into a bookWorksets array where each entry
// carries { dagIri: 'urn:noocodec:tool:<name>', arguments: {...} }. The scatter
// placement reads dagIri through an item-scoped DagReference to resolve
// the body DAG at runtime.
.node(placement('build-book-worksets'), buildBookWorksets, {
'ready': placement('book-search-scatter'),
}, display('build-book-worksets'))
// ── 3. book-search-scatter ───────────────────────────────────────────────
// Tool-registry scatter: bookWorksets items fan out concurrently. Each item
// carries its own tool DAG IRI via dagIri; the DagReference
// resolves the body DAG at runtime from the item. ToolInvokeNode reads the
// item's arguments field and calls the bound tool. The following GatherNode
// reads each clone's ToolInvocationState.output (via accessor, no cast)
// and folds the CandidateType[] into the parent state's candidates.
// any-success reducer: 'success' → rank-candidates when at least one tool hit;
// 'error' → rank-candidates to allow graceful empty-candidates handling.
.scatter(placement('book-search-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('book-search-gather'),
'error': placement('book-search-gather'),
'empty': placement('rank-candidates'),
}, {
'name': 'book-search-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('book-search-gather'), { [placement('book-search-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('rank-candidates'),
'error': placement('rank-candidates'),
'empty': placement('rank-candidates'),
}, display('book-search-gather'))
// ── 4. rank-candidates ───────────────────────────────────────────────────
// LLM-driven relevance scoring. Routes 'ranked' on success (an empty set is
// still a valid ranking, so merge can soft-gate on zero candidates).
// 'retry' loops back (bounded); 'salvage' passes candidates through unranked
// via a dedicated node rather than emitting them as if they were ranked.
.node(placement('rank-candidates'), rankCandidates, {
'ranked': placement('merge-candidates'),
'retry': placement('rank-candidates'),
'salvage': placement('rank-candidates-salvage'),
}, display('rank-candidates'))
.node(placement('rank-candidates-salvage'), rankCandidatesSalvage, {
'done': placement('merge-candidates'),
}, display('rank-candidates-salvage'))
// ── 5. merge-candidates ──────────────────────────────────────────────────
// Cross-source dedupe via CanonicalId, top-5. Routes 'empty' to
// no-results (TerminalNode(failed)) so the parent EmbeddedDAGNode's
// terminal outcome routes the parent placement to its 'error' branch.
.node(placement('merge-candidates'), mergeCandidates, {
'ranked': placement('record-findings'),
'empty': placement('no-results'),
}, display('merge-candidates'))
// ── 6. record-findings ───────────────────────────────────────────────────
// Deterministic RDF write: same input always produces the same triples.
.node(placement('record-findings'), recordFindings, {
'recorded': placement('has-citations-gate'),
}, display('record-findings'))
// ── 7. has-citations-gate ────────────────────────────────────────────────
// SPARQL ASK over the per-run state graph. Symbolic fence for the LLM.
// 'fail' routes to no-results (TerminalNode(failed)) so the parent
// EmbeddedDAGNode routes the parent placement to 'error'.
.node(placement('has-citations-gate'), hasCitationsGate, {
'pass': placement('recall-past-visits'),
'fail': placement('no-results'),
}, display('has-citations-gate'))
// ── 8. recall-past-visits ────────────────────────────────────────────────
// Injects prior-session context (prior queries + shortlisted titles) into
// state.priorContext, then routes to the canonical `found` TerminalNode
// (completed) so the parent EmbeddedDAGNode resolves its 'success' branch.
.node(placement('recall-past-visits'), recallPastVisits, {
'recalled': placement('found'),
}, display('recall-past-visits'))
// ── 9. Terminal nodes ────────────────────────────────────────────────────
// Both sub-DAG exits are canonical TerminalNode placements (no bare null
// routes): `found` (completed) drives the parent EmbeddedDAGNode's 'success'
// branch; `no-results` (failed) drives its 'error' branch.
.terminal(placement('found'), { outcome: 'completed', name: 'found' })
.terminal(placement('no-results'), { outcome: 'failed', name: 'no-results' })
.build();ComposeRetryLoopDAG
ts
/**
* ComposeRetryLoopDAG: reusable compose / validate / retry loop.
*
* Internal flow:
*
* compose-response
* └─ drafted ──► validate-response
* ├─ approved ──► END (success) ─► parent: respond-to-visitor
* ├─ retry ──► compose-response (bounded by the retry budget on state (retriesFor('compose')))
* └─ exhausted ──► END (success) ─► parent: respond-to-visitor
*
* Outputs:
* success: draft composed (approved or best-effort); parent routes to
* the shared respond-to-visitor terminal.
* error: clone-state errors accumulated (propagated by the parent
* ScatterNode to the parent's 'error' branch)
*
* Convergence policy: this sub-DAG does NOT contain respondToVisitor. It is a
* pure compose/validate unit that produces state.draft and exits. The
* single shared respond-to-visitor placement lives at the parent DAG level
* so that every converging branch strikes exactly one terminal node per run.
*
* Molecular import pattern:
* import { composeRetryLoopDAG } from './embedded-dags/ComposeRetryLoopDAG.ts';
* const nodes = ArchivistNodes.build(services);
* dispatcher.registerBundle({ nodes: nodes.composeRetryLoopNodes, dags: [composeRetryLoopDAG] });
*
* The sub-DAG operates on the parent's state directly (no projection / gather
* needed); it reads `state.shortlist` / `state.intent` / `state.priorContext`
* and writes `state.draft` / `state.approved`, which the parent DAG already
* manages. Every intent branch funnels through this one composed loop rather
* than each branch owning its own compose→validate chain.
*/
import type { ArchivistState } from '../ArchivistState.ts';
import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';
const COMPOSE_RETRY_LOOP_DAG_IRI = 'urn:noocodec:dag:compose-retry-loop';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(COMPOSE_RETRY_LOOP_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });
const composeResponse = new PlaceholderNode<ArchivistState, 'drafted' | 'retry' | 'salvage'>('urn:noocodec:node:compose-response', ['drafted', 'retry', 'salvage']);
const composeResponseSalvage = new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:compose-salvage', ['done']);
const validateResponse = new PlaceholderNode<ArchivistState, 'approved' | 'retry' | 'exhausted'>('urn:noocodec:node:validate-response', ['approved', 'retry', 'exhausted']);
export const composeRetryLoopDAG: DAGType = new DAGBuilder(COMPOSE_RETRY_LOOP_DAG_IRI, '1.1', display('compose-retry-loop'))
// ── 1. compose-response ──────────────────────────────────────────────────
// Writes state.draft via an intent-specific compose method. A transient LLM
// failure routes 'retry' (loops back, bounded by the shared 'compose' budget)
// or 'salvage' once spent; no in-node RetryPolicy. 'drafted' goes on to the
// quality gate, which adds its own retry edge for low-quality drafts.
.node(placement('compose-response'), composeResponse, {
'drafted': placement('validate-response'),
'retry': placement('compose-response'),
'salvage': placement('compose-salvage'),
}, display('compose-response'))
.node(placement('compose-salvage'), composeResponseSalvage, {
'done': placement('composed'),
}, display('compose-salvage'))
// ── 2. validate-response ─────────────────────────────────────────────────
// Quality gate: length, citations, tone. On 'retry', routes back to
// compose (bounded by MAX_COMPOSE_ATTEMPTS via state.retriesFor('compose')).
// 'approved' and 'exhausted' both exit via the canonical `composed`
// TerminalNode (completed), so the parent EmbeddedDAGNode resolves 'success'
// and routes to respond-to-visitor.
.node(placement('validate-response'), validateResponse, {
'approved': placement('composed'),
'retry': placement('compose-response'),
'exhausted': placement('composed'),
}, display('validate-response'))
// ── 3. composed ──────────────────────────────────────────────────────────
// Canonical TerminalNode(completed): the single explicit exit of the compose
// loop. No bare null end-of-flow routes.
.terminal(placement('composed'), { outcome: 'completed', name: 'composed' })
.build();Source
JSON-LD as the canonical DAG format
The DAG is JSON-LD natively. DAGBuilder.build() returns a plain JavaScript object whose wire shape is JSON-LD 1.1; every placement carries a typed IRI under @type. DAGDocument.serialize(dag) produces the JSON string; DAGDocument.load(json) parses and validates it back to an equivalent typed DAG.
There is no separate projection layer or dual configuration. The object DAGBuilder.build() returns is the same object the engine consumes and the same object that serializes to JSON-LD. Load a DAG from JSON, register it, execute it: one surface throughout.
ts
import { DAGDocument } from '@studnicky/dagonizer/dag';
import { Validator } from '@studnicky/dagonizer/validation';
import { archivistDAG } from './dag.ts';
const json = DAGDocument.serialize(archivistDAG);
const reloaded = DAGDocument.load(json);
console.assert(reloaded.name === archivistDAG.name, 'name mismatch');
console.assert(JSON.stringify(reloaded.entrypoints) === JSON.stringify(archivistDAG.entrypoints), 'entrypoints mismatch');
console.assert(reloaded.nodes.length === archivistDAG.nodes.length, 'node count mismatch');
const validated = Validator.dag.validate(reloaded);
console.log('dag-roundtrip: ok');
console.log(` name: ${validated.name}`);
console.log(` entrypoints: ${JSON.stringify(validated.entrypoints)}`);
console.log(` nodes: ${validated.nodes.length}`);Embedded-DAG placements in the JSON-LD output look like:
json
{
"@type": "EmbeddedDAGNode",
"name": "on-topic-search",
"dag": "book-search-scatter",
"outputs": { "success": "compose-loop", "error": "compose-empty" }
}DAG topology
ts
/**
* The Archivist: canonical DAG, built with DAGBuilder. Version 6.0.
*
* Molecular composition: the parent DAG is composed of two reusable
* sub-DAGs that ship as independent components and are imported as
* `.embed(name, dagIri, routes)` placements. The sub-DAGs are registered
* separately and referenced by canonical IRI; the parent DAG never knows
* their internals.
*
* recall-context
* └─ recalled ──► classify-intent
*
* classify-intent
* ├─ off-topic ──► decline-off-topic ──► END
* │
* ├─ on-topic ──► [book-search-scatter] (extract+decide+4scouts+rank+merge+record+gate+recall)
* │ ├─ success ──► [compose-retry-loop] (compose+validate+retry)
* │ └─ error ──► compose-empty ──┐
* │ │
* ├─ lookup-author ──► [book-search-scatter] │
* │ ├─ success ──► group-by-year ──► [compose-retry-loop]
* │ └─ error ──► compose-empty ──┐
*
* │ ▼
* ├─ find-reviews ──► reviews-extract ──► [compose-retry-loop] (success) ──► respond-to-visitor ──► END
* │ (inline: decide+4scouts+rankByRating+merge+record+gate+recall) ▲
* │ ▲
* ├─ describe-book ──► describe-extract ──► [compose-retry-loop]
* │ (inline: decide+4scouts+pickBestMatch+merge+record+gate+recall)
* │
* ├─ recommend ──► recommend-extract ──► [compose-retry-loop] (success) ──► respond-to-visitor ──► END
* │ (inline: decide+4scouts+rankByRating+merge+record+gate+recall) ▲
* │ ▲
* ├─ recall-memories ──► memory-recall ──► compose-memory-recall ──────────────────────────────┐
* │ ▼
* └─ recommend-similar ──► recommend-similar-gate respond-to-visitor ──► END
* ├─ seeded ──► [book-search-scatter] ▲
* │ ├─ success ──► [compose-retry-loop] (success) ──────┘
* │ └─ error ──► compose-empty ──────────────────────►┘
* └─ empty ──► compose-empty ───────────────────────────────────────►┘
*
* Convergence policy (v6.0): all response-producing branches converge into ONE
* shared `respond-to-visitor` terminal at this (parent) level. The
* compose-retry-loop embedded-DAG exits with `success` after producing state.draft
* and does NOT contain respondToVisitor internally. This ensures exactly one
* terminal node fires per run with the full converged state.draft.
*
* Embedded-DAGs (molecular components):
* book-search-scatter: extract-query + decide-tools + 4-source parallel scouts
* (OpenLibrary, Google Books, Subject, Wikipedia) + rankCandidates
* + mergeCandidates + recordFindings + hasCitationsGate +
* recallPastVisits. Three placements in this DAG:
* on-topic-search, author-search, similar-search.
*
* compose-retry-loop: composeResponse + validateResponse (with bounded retry loop)
* + respondToVisitor. Four placements in this DAG:
* compose-loop (shared by all four convergent branches).
*
* Inlined branches (reviews, describe, recommend-top-rated):
* Reviews and recommend-top-rated both use `rankByRating` (deterministic,
* rating-weighted) instead of `rankCandidates` (LLM-driven); recommend-top-rated
* is the structural sibling of find-reviews, reusing the same node objects at
* `recommend-*` placements, for the vague "good book / good story" recommend
* intent that has no topic to rank by relevance. Describe uses `pickBestMatch`
* to narrow to the top-3 title-similar candidates before merge. All three are
* structurally identical to book-search-scatter except for the post-scout
* ranking step; keeping them inline makes the intentional distinction explicit
* rather than hiding it behind a embedded-DAG parameter.
*
* Empty-result handling (v5.2):
* Empty results route through `compose-empty` → `respond-to-visitor`.
* `compose-empty` calls the LLM with `state.failureCause` (accumulated by
* scouts) to produce an in-character message that acknowledges what was
* searched and offers a concrete next step.
*
* Builder output shape:
* DAGBuilder.node(name, dagNode, routes) emits a
* { type: 'single', name, node: dagNode.name, outputs: routes }
* object. build() returns a plain DAG passed straight to
* DAGDocument.load().
*
* DAG containment (WorkerThreadContainer) — why the archivist stays in-process:
* The container/worker feature is most natural for CPU-bound, self-contained
* per-item work: pure data transforms that only read from state and write a
* result back (see the-cartographer, which routes its canonical-event enrichment
* sub-DAG through a WorkerThreadContainer — haversine, GDPR redaction, pricing
* and ETA are pure arithmetic on serialisable data).
*
* The archivist's scatter items (the four scout providers) are LLM / network
* bound: each clone calls a live language model and external book APIs. Workers
* cannot share the LLM provider instance (it is not serialisable), and each
* scatter item's payload — prompt context, live API credentials, LLM adapter
* state — crosses the worker message channel as structured-clone, which drops
* functions, closures, class instances, and streams.
*
* Worker containment suits CPU / data DAGs. LLM-cascade DAGs — where nodes
* carry network clients, streaming responses, and rich provider objects — run
* in-process and rely on async concurrency (Promise.all, scatter concurrency)
* rather than OS-level thread isolation for parallelism.
*/
import type { ArchivistState } from './ArchivistState.ts';
import './nodes/scouts.ts'; // registers 'tool-candidate-merge' gather strategy
import { DAGBuilder, DAGIdentity, PlaceholderNode } from '@studnicky/dagonizer';
import type { DAGType } from '@studnicky/dagonizer';
// #region dispatcher-bundle
//
// IRI identity: DAGBuilder embeds the canonical DAG_CONTEXT in every built
// DAG's `@context` field. The archivist uses explicit placement IRIs via
// DAGIdentity.placementId(dagIri, placementIdentifier) so route targets,
// gather sources, and entrypoint wiring all stay on canonical IRIs while the
// placement `name` field remains display-only.
//
// @id values on each node placement follow the urn:noocodec:dag:<dagName>/node/<placementName>
// convention produced by DAGIdentity.placementId(), e.g.:
// urn:noocodec:dag:the-archivist/node/recall-context
//
// A plugin shipping nodes under its own namespace would declare a prefix in
// the bundle's `context` field — e.g. { context: { archivist: 'https://archivist.example.com/' } }
// — and use prefixed names like 'archivist:recallContext' to prevent collisions
// with other plugins that might register a node named 'recallContext'.
// See docs/guide/iri-identity.md for the full expansion rule set.
const BOOK_SEARCH_TOOL_DAGS = [
'urn:noocodec:tool:web_search_books',
'urn:noocodec:tool:google_books_search',
'urn:noocodec:tool:subject_search',
'urn:noocodec:tool:wikipedia_summary',
] as const;
const ARCHIVIST_DAG_IRI = 'urn:noocodec:dag:the-archivist';
const BOOK_SEARCH_SCATTER_DAG_IRI = 'urn:noocodec:dag:book-search-scatter';
const COMPOSE_RETRY_LOOP_DAG_IRI = 'urn:noocodec:dag:compose-retry-loop';
const placement = (placementIdentifier: string): string => DAGIdentity.placementId(ARCHIVIST_DAG_IRI, placementIdentifier);
const display = <T extends string>(name: T): { name: T } => ({ name });
const nodes = {
'preRunSetup': new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:pre-run-setup', ['ready']),
'parkForInput': new PlaceholderNode<ArchivistState, 'parked' | 'resumed'>('urn:noocodec:node:park-for-input', ['parked', 'resumed']),
'recallContext': new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-context', ['recalled']),
'classifyIntent': new PlaceholderNode<ArchivistState, 'lookup-author' | 'find-reviews' | 'describe-book' | 'recommend-similar' | 'recall-memories' | 'on-topic' | 'recommend-top-rated' | 'off-topic' | 'retry' | 'salvage'>('urn:noocodec:node:classify-intent', ['lookup-author', 'find-reviews', 'describe-book', 'recommend-similar', 'recall-memories', 'on-topic', 'recommend-top-rated', 'off-topic', 'retry', 'salvage']),
'classifyIntentSalvage': new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:classify-intent-salvage', ['done']),
'extractQuery': new PlaceholderNode<ArchivistState, 'success' | 'retry' | 'salvage'>('urn:noocodec:node:extract-query', ['success', 'retry', 'salvage']),
'extractQuerySalvage': new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:extract-query-salvage', ['done']),
'decideTools': new PlaceholderNode<ArchivistState, 'tools' | 'no-tools' | 'retry' | 'salvage'>('urn:noocodec:node:decide-tools', ['tools', 'no-tools', 'retry', 'salvage']),
'decideToolsSalvage': new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:decide-tools-salvage', ['done']),
'buildBookWorksets': new PlaceholderNode<ArchivistState, 'ready'>('urn:noocodec:node:build-book-worksets', ['ready']),
'rankByRating': new PlaceholderNode<ArchivistState, 'ranked'>('urn:noocodec:node:rank-by-rating', ['ranked']),
'pickBestMatch': new PlaceholderNode<ArchivistState, 'picked'>('urn:noocodec:node:pick-best-match', ['picked']),
'mergeCandidates': new PlaceholderNode<ArchivistState, 'ranked' | 'empty'>('urn:noocodec:node:merge-candidates', ['ranked', 'empty']),
'recordFindings': new PlaceholderNode<ArchivistState, 'recorded'>('urn:noocodec:node:record-findings', ['recorded']),
'hasCitationsGate': new PlaceholderNode<ArchivistState, 'pass' | 'fail'>('urn:noocodec:node:has-citations-gate', ['pass', 'fail']),
'groupByYear': new PlaceholderNode<ArchivistState, 'ordered'>('urn:noocodec:node:group-by-year', ['ordered']),
'recallPastVisits': new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-past-visits', ['recalled']),
'recommendSimilar': new PlaceholderNode<ArchivistState, 'seeded' | 'empty'>('urn:noocodec:node:recommend-similar', ['seeded', 'empty']),
'recallMemories': new PlaceholderNode<ArchivistState, 'recalled'>('urn:noocodec:node:recall-memories', ['recalled']),
'composeMemoryResponse': new PlaceholderNode<ArchivistState, 'drafted' | 'retry' | 'salvage'>('urn:noocodec:node:compose-memory-response', ['drafted', 'retry', 'salvage']),
'composeMemoryResponseSalvage': new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:compose-memory-salvage', ['done']),
'respondToVisitor': new PlaceholderNode<ArchivistState, 'success'>('urn:noocodec:node:respond-to-visitor', ['success']),
'declineOffTopic': new PlaceholderNode<ArchivistState, 'success'>('urn:noocodec:node:decline-off-topic', ['success']),
'composeEmptyResponse': new PlaceholderNode<ArchivistState, 'drafted' | 'retry' | 'salvage'>('urn:noocodec:node:compose-empty', ['drafted', 'retry', 'salvage']),
'composeEmptyResponseSalvage': new PlaceholderNode<ArchivistState, 'done'>('urn:noocodec:node:compose-empty-salvage', ['done']),
} as const;
export const archivistDAG: DAGType = new DAGBuilder(ARCHIVIST_DAG_IRI, '6.0', display('the-archivist'))
// ── pre-phase: setup ─────────────────────────────────────────────────────
// Stamps state.runId and clears any stale draft before the main loop starts.
// PhaseNode placement: runs before the entrypoint node; errors abort the run.
// No routing: phase placements are out-of-band and never set the entrypoint.
.phase(placement('setup'), 'pre', nodes.preRunSetup, display('setup'))
// ── 0. park-for-input (HITL gate) ────────────────────────────────────────
// First added → auto-entrypoint. Parks the flow when state.query is empty,
// waiting for the human to supply input via the browser HITL banner. On
// resume, `state.query` is set by the caller before `dispatcher.resume()`;
// the node then routes `'resumed'` and proceeds to `recall-context`.
// The `'parked'` output routes to the engine park machinery (null wiring).
.node(placement('park-for-input'), nodes.parkForInput, {
'parked': placement('park-for-input'),
'resumed': placement('recall-context'),
}, display('park-for-input'))
// ── 1. recall-context ────────────────────────────────────────────────────
// Runs before classifyIntent so the classifier can benefit from
// prior-session continuity hints.
.node(placement('recall-context'), nodes.recallContext, {
'recalled': placement('classify-intent'),
}, display('recall-context'))
// #region intent-routes
// ── 1. classify-intent ───────────────────────────────────────────────────
// Wide output union routes to seven branches. EmbeddedDAG placements and inline
// branches share the same shared terminal: compose-loop and compose-empty.
// recall-memories routes directly to memory-recall → compose-memory-recall
// → memory-respond (no search needed; the memory store is the source).
.node(placement('classify-intent'), nodes.classifyIntent, {
'lookup-author': placement('author-search'),
'find-reviews': placement('reviews-extract'),
'describe-book': placement('describe-extract'),
'recommend-similar': placement('recommend-similar'),
'recall-memories': placement('memory-recall'),
'on-topic': placement('on-topic-search'),
'recommend-top-rated': placement('recommend-extract'),
'off-topic': placement('decline-off-topic'),
// Own timeout / classifier failure → retry budget decides. 'retry' loops
// back; 'salvage' defaults to the broadest on-topic search via a node.
'retry': placement('classify-intent'),
'salvage': placement('classify-intent-salvage'),
}, display('classify-intent'))
.node(placement('classify-intent-salvage'), nodes.classifyIntentSalvage, {
'done': placement('on-topic-search'),
}, display('classify-intent-salvage'))
// #endregion intent-routes
// #region embedded-dag-placements
// ── on-topic branch ──────────────────────────────────────────────────────
// EmbeddedDAGNode: book-search-scatter handles extract-query, decide-tools,
// all four scouts, rank-candidates, merge, record, gate, and recall.
// One packaged cluster; first of three placements of the same sub-DAG.
// gather.map copies the fields the sub-DAG writes back to the parent state
// so compose-loop and group-by-year can read them.
.embed(placement('on-topic-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('compose-loop'),
'error': placement('compose-empty'),
}, {
'name': 'on-topic-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// ── lookup-author branch ─────────────────────────────────────────────────
// EmbeddedDAGNode: same book-search-scatter cluster, second placement.
// After success, group-by-year sorts results chronologically before the
// compose loop; author surveys read better in publication-timeline order.
.embed(placement('author-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('group-by-year'),
'error': placement('compose-empty'),
}, {
'name': 'author-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// group-by-year is author-branch-specific: sorts shortlist chronologically.
.node(placement('group-by-year'), nodes.groupByYear, {
'ordered': placement('compose-loop'),
}, display('group-by-year'))
// ── find-reviews branch ───────────────────────────────────────────────────
// Inlined. Uses rankByRating (deterministic, rating-weighted) in place of
// rankCandidates (LLM-driven). The Google Books scout carries notes.rating /
// notes.ratingsCount; rankByRating weights those for reviews-style output.
.node(placement('reviews-extract'), nodes.extractQuery, {
'success': placement('reviews-decide-tools'),
'retry': placement('reviews-extract'),
'salvage': placement('reviews-extract-salvage'),
}, display('reviews-extract'))
.node(placement('reviews-extract-salvage'), nodes.extractQuerySalvage, {
'done': placement('reviews-decide-tools'),
}, display('reviews-extract-salvage'))
.node(placement('reviews-decide-tools'), nodes.decideTools, {
'tools': placement('reviews-build-worksets'),
'no-tools': placement('reviews-build-worksets'),
'retry': placement('reviews-decide-tools'),
'salvage': placement('reviews-decide-tools-salvage'),
}, display('reviews-decide-tools'))
.node(placement('reviews-decide-tools-salvage'), nodes.decideToolsSalvage, {
'done': placement('reviews-build-worksets'),
}, display('reviews-decide-tools-salvage'))
// Build scatter worksets: converts toolPlan into bookWorksets items so the
// scatter can dispatch to each declared tool DAG IRI.
.node(placement('reviews-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('reviews-scatter'),
}, display('reviews-build-worksets'))
// Tool-registry scatter: each bookWorksets item carries its own tool DAG
// IRI. The following GatherNode reads each clone's output via
// accessor (no cast) and folds CandidateType[] into parent candidates.
.scatter(placement('reviews-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('reviews-gather'),
'error': placement('reviews-gather'),
'empty': placement('reviews-rank'),
}, {
'name': 'reviews-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('reviews-gather'), { [placement('reviews-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('reviews-rank'),
'error': placement('reviews-rank'),
'empty': placement('reviews-rank'),
}, display('reviews-gather'))
.node(placement('reviews-rank'), nodes.rankByRating, { 'ranked': placement('reviews-merge') }, display('reviews-rank'))
.node(placement('reviews-merge'), nodes.mergeCandidates, { 'ranked': placement('reviews-record'), 'empty': placement('compose-empty') }, display('reviews-merge'))
.node(placement('reviews-record'), nodes.recordFindings, { 'recorded': placement('reviews-gate') }, display('reviews-record'))
.node(placement('reviews-gate'), nodes.hasCitationsGate, { 'pass': placement('reviews-recall'), 'fail': placement('compose-empty') }, display('reviews-gate'))
.node(placement('reviews-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('reviews-recall'))
// ── recommend-top-rated branch ───────────────────────────────────────────
// Inlined, structural sibling of find-reviews. Reuses rankByRating
// (deterministic, rating-weighted) instead of rankCandidates (LLM-driven)
// because a vague "good book / good story" request carries no topic for
// relevance ranking — rating is the only signal that makes sense.
.node(placement('recommend-extract'), nodes.extractQuery, {
'success': placement('recommend-decide-tools'),
'retry': placement('recommend-extract'),
'salvage': placement('recommend-extract-salvage'),
}, display('recommend-extract'))
.node(placement('recommend-extract-salvage'), nodes.extractQuerySalvage, {
'done': placement('recommend-decide-tools'),
}, display('recommend-extract-salvage'))
.node(placement('recommend-decide-tools'), nodes.decideTools, {
'tools': placement('recommend-build-worksets'),
'no-tools': placement('recommend-build-worksets'),
'retry': placement('recommend-decide-tools'),
'salvage': placement('recommend-decide-tools-salvage'),
}, display('recommend-decide-tools'))
.node(placement('recommend-decide-tools-salvage'), nodes.decideToolsSalvage, {
'done': placement('recommend-build-worksets'),
}, display('recommend-decide-tools-salvage'))
// Build scatter worksets: converts toolPlan into bookWorksets items so the
// scatter can dispatch to each declared tool DAG IRI.
.node(placement('recommend-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('recommend-scatter'),
}, display('recommend-build-worksets'))
// Tool-registry scatter: each bookWorksets item carries its own tool DAG
// IRI. The following GatherNode reads each clone's output via
// accessor (no cast) and folds CandidateType[] into parent candidates.
.scatter(placement('recommend-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('recommend-gather'),
'error': placement('recommend-gather'),
'empty': placement('recommend-rank'),
}, {
'name': 'recommend-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('recommend-gather'), { [placement('recommend-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('recommend-rank'),
'error': placement('recommend-rank'),
'empty': placement('recommend-rank'),
}, display('recommend-gather'))
.node(placement('recommend-rank'), nodes.rankByRating, { 'ranked': placement('recommend-merge') }, display('recommend-rank'))
.node(placement('recommend-merge'), nodes.mergeCandidates, { 'ranked': placement('recommend-record'), 'empty': placement('compose-empty') }, display('recommend-merge'))
.node(placement('recommend-record'), nodes.recordFindings, { 'recorded': placement('recommend-gate') }, display('recommend-record'))
.node(placement('recommend-gate'), nodes.hasCitationsGate, { 'pass': placement('recommend-recall'), 'fail': placement('compose-empty') }, display('recommend-gate'))
.node(placement('recommend-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('recommend-recall'))
// ── describe-book branch ─────────────────────────────────────────────────
// Inlined. Uses pickBestMatch to narrow multi-hit results to the top-3
// title-similar candidates before merge. Ensures the composer receives the
// specific book the visitor named, not arbitrary top-5 hits.
.node(placement('describe-extract'), nodes.extractQuery, { 'success': placement('describe-decide-tools'), 'retry': placement('describe-extract'), 'salvage': placement('describe-extract-salvage') }, display('describe-extract'))
.node(placement('describe-extract-salvage'), nodes.extractQuerySalvage, { 'done': placement('describe-decide-tools') }, display('describe-extract-salvage'))
.node(placement('describe-decide-tools'), nodes.decideTools, { 'tools': placement('describe-build-worksets'), 'no-tools': placement('describe-build-worksets'), 'retry': placement('describe-decide-tools'), 'salvage': placement('describe-decide-tools-salvage') }, display('describe-decide-tools'))
.node(placement('describe-decide-tools-salvage'), nodes.decideToolsSalvage, { 'done': placement('describe-build-worksets') }, display('describe-decide-tools-salvage'))
// Build scatter worksets before dispatch.
.node(placement('describe-build-worksets'), nodes.buildBookWorksets, {
'ready': placement('describe-scatter'),
}, display('describe-build-worksets'))
// Tool-registry scatter: DagReference resolves body DAG from each item's dagName.
// any-success reducer: 'success' → describe-pick, 'error' → compose-empty.
// 'error' fires when all tool scouts return empty.
.scatter(placement('describe-scatter'), 'bookWorksets', { 'dag': { 'from': 'item', 'path': 'dagIri', 'candidates': BOOK_SEARCH_TOOL_DAGS } }, {
'success': placement('describe-gather'),
'error': placement('compose-empty'),
'empty': placement('compose-empty'),
}, {
'name': 'describe-scatter',
'execution': { 'mode': 'item', 'concurrency': 4 },
'reducer': 'any-success',
})
.gather(placement('describe-gather'), { [placement('describe-scatter')]: {} }, { 'strategy': 'tool-candidate-merge' }, {
'success': placement('describe-pick'),
'error': placement('compose-empty'),
'empty': placement('compose-empty'),
}, display('describe-gather'))
.node(placement('describe-pick'), nodes.pickBestMatch, { 'picked': placement('describe-merge') }, display('describe-pick'))
.node(placement('describe-merge'), nodes.mergeCandidates, { 'ranked': placement('describe-record'), 'empty': placement('compose-empty') }, display('describe-merge'))
.node(placement('describe-record'), nodes.recordFindings, { 'recorded': placement('describe-gate') }, display('describe-record'))
.node(placement('describe-gate'), nodes.hasCitationsGate, { 'pass': placement('describe-recall'), 'fail': placement('compose-empty') }, display('describe-gate'))
.node(placement('describe-recall'), nodes.recallPastVisits, { 'recalled': placement('compose-loop') }, display('describe-recall'))
// ── recommend-similar branch ─────────────────────────────────────────────
// recommendSimilar seeds state.terms from prior-run shortlist memory.
// 'seeded' routes to the book-search-scatter sub-DAG; third placement of
// the same packaged cluster. 'empty' routes to the compose-empty terminal.
.node(placement('recommend-similar'), nodes.recommendSimilar, {
'seeded': placement('similar-search'),
'empty': placement('compose-empty'),
}, display('recommend-similar'))
// EmbeddedDAGNode: same book-search-scatter, third and final placement.
.embed(placement('similar-search'), BOOK_SEARCH_SCATTER_DAG_IRI, {
'success': placement('compose-loop'),
'error': placement('compose-empty'),
}, {
'name': 'similar-search',
'outputs': {
'terms': 'terms',
'toolPlan': 'toolPlan',
'candidates': 'candidates',
'shortlist': 'shortlist',
'priorContext': 'priorContext',
'failureCause': 'failureCause',
},
})
// ── compose-loop: shared compose/validate sub-DAG ──────────────────────────
// All branches that successfully find candidates converge here.
// composeResponse → validateResponse (retry loop, bounded by the retry budget on state (retriesFor('compose'))).
// One sub-DAG definition serves all four convergent branches.
// stateMapping.outputs copies the compose loop's writes back to the parent.
//
// Convergence policy: 'success' routes to the shared respond-to-visitor terminal
// at the parent level; the sub-DAG produces state.draft and exits cleanly;
// exactly ONE respond-to-visitor fires per run regardless of branch count.
// 'error' (retry budget exhausted) falls through to compose-empty so the
// visitor always receives an in-character response rather than a silent drop.
.embed(placement('compose-loop'), COMPOSE_RETRY_LOOP_DAG_IRI, {
'success': placement('respond-to-visitor'),
'error': placement('compose-empty'),
}, {
'name': 'compose-loop',
'outputs': {
'draft': 'draft',
'approvalState': 'approvalState',
},
})
// #endregion embedded-dag-placements
// ── respond-to-visitor: single shared happy-path terminal ───────────────
// Every branch that successfully composes a response converges here.
// compose-loop (success) and both memory + empty-result paths all route
// through this one placement. Convergence policy: exactly ONE respond-to-visitor
// fires per run with the full converged state.draft in context. Success routes
// to the canonical `end` TerminalNode rather than a bare null end-of-flow.
.node(placement('respond-to-visitor'), nodes.respondToVisitor, { 'success': placement('end') }, display('respond-to-visitor'))
// ── recall-memories branch ───────────────────────────────────────────────
// No search needed; the memory store is queried directly.
// recallMemories → composeMemoryResponse → respond-to-visitor (shared terminal).
.node(placement('memory-recall'), nodes.recallMemories, { 'recalled': placement('compose-memory-recall') }, display('memory-recall'))
.node(placement('compose-memory-recall'), nodes.composeMemoryResponse, {
'drafted': placement('respond-to-visitor'),
'retry': placement('compose-memory-recall'),
'salvage': placement('compose-memory-salvage'),
}, display('compose-memory-recall'))
.node(placement('compose-memory-salvage'), nodes.composeMemoryResponseSalvage, { 'done': placement('respond-to-visitor') }, display('compose-memory-salvage'))
// #region terminal-placements
// ── Terminal nodes ───────────────────────────────────────────────────────
.node(placement('decline-off-topic'), nodes.declineOffTopic, { 'success': placement('end') }, display('decline-off-topic'))
.node(placement('compose-empty'), nodes.composeEmptyResponse, {
'drafted': placement('respond-to-visitor'),
'retry': placement('compose-empty'),
'salvage': placement('compose-empty-salvage'),
}, display('compose-empty'))
.node(placement('compose-empty-salvage'), nodes.composeEmptyResponseSalvage, { 'done': placement('respond-to-visitor') }, display('compose-empty-salvage'))
// Canonical end-of-flow: every completed path (a composed answer or an
// off-topic decline) routes to this one `TerminalNode(completed)` instead of
// a bare `null` route. The flow ends explicitly, not by absence of a route.
.terminal(placement('end'), { outcome: 'completed', name: 'end' })
// #endregion terminal-placements
.build();
// #endregion dispatcher-bundleState
ts
/**
* ArchivistState: the clipboard the Archivist's nodes mutate.
*
* Carries the visitor's question, the parsed intent, scout candidates,
* the merged shortlist, the draft response, and per-execution counters.
* Extends `NodeStateBase` so the dispatcher owns the lifecycle FSM and
* `snapshot()` round-trips for `Checkpoint.capture` / `ckpt.restoreState`.
*/
import type { CandidateType } from './entities/Book.ts';
import type { BookWorksetItemType } from './nodes/buildBookWorksets.ts';
import { NodeStateBase } from '@studnicky/dagonizer';
import type { JsonObjectType, StateFieldsType } from '@studnicky/dagonizer/types';
import type { ReasoningStepType } from '@studnicky/dagonizer';
import { Validator } from '@studnicky/dagonizer/validation';
import { CandidateSchema } from '@studnicky/dagonizer-book-entities';
/**
* A single turn in the visitor–archivist conversation.
* Stored on `ArchivistState.conversation` and injected into LLM prompts
* so the model can resolve pronouns and follow-ups across turns.
*/
export interface ConversationTurn {
readonly role: 'visitor' | 'archivist';
readonly text: string;
readonly ts: number;
}
/**
* A roll-up of everything the Archivist has accumulated in its memory
* store across all prior runs, produced by `recallMemories` and consumed
* by `composeMemoryResponse`.
*/
export interface MemoryDigest {
/** Total distinct books recorded across all runs. */
readonly bookCount: number;
/** Total visitor queries issued across all runs. */
readonly queryCount: number;
/** Up to the last 10 distinct shortlisted books (most-recent first). */
readonly recentBooks: ReadonlyArray<{ readonly title: string; readonly author: string }>;
/** Intent distribution: how many times each intent was classified. */
readonly intentBreakdown: ReadonlyArray<{ readonly intent: string; readonly count: number }>;
/** 1–2 sentence LLM-ready summary of the digest. */
readonly summary: string;
}
/**
* Prior-context facts recalled from the memory graph before classification.
* `summary` is an LLM-ready 1–2 sentence hint; the structured arrays are
* available directly on `state.recalledContext` for downstream nodes.
*/
export interface RecalledContext {
/** Intents the classifier returned for similar prior queries. */
readonly priorIntents: ReadonlyArray<{
readonly query: string;
readonly intent: string;
readonly ts: string;
}>;
/** Books seen in recent state graphs (shortlisted candidates). */
readonly recentCandidates: ReadonlyArray<CandidateType>;
/** Prior queries that overlap with the current query text. */
readonly similarPriorQueries: ReadonlyArray<{
readonly query: string;
readonly ts: string;
}>;
/** Reasoning steps recalled from prior runs' PROV graphs. */
readonly priorReasoning: ReadonlyArray<{ readonly text: string; readonly kind: string }>;
/** 1–2 sentence LLM-ready hint; empty string when nothing was recalled. */
readonly summary: string;
}
/** What the visitor asked the Archivist to do. */
export type ArchivistIntent =
| 'lookup-author' // visitor named an author and wants their body of work
| 'find-reviews' // visitor wants opinions / reviews / what readers think
| 'describe-book' // visitor named a specific title and wants a description
| 'recommend-similar' // visitor wants something like a previous read
| 'recall-memories' // visitor asked what the agent has seen / remembered
| 'search' // visitor named a title / author / ISBN (generic search)
| 'describe' // visitor described a book without naming it
| 'recommend' // visitor asked for a generic recommendation
| 'off-topic'; // visitor wandered: not a book query and not memory-related
export class ArchivistState extends NodeStateBase {
private static readonly candidateValidator = Validator.compile<CandidateType>(CandidateSchema);
/**
* Declared scalar fields for schema-driven snapshot/restore.
* Complex fields (arrays with item type-guards, nested objects,
* discriminated-union fields) are handled manually in
* `snapshotData` / `restoreData`.
*/
static readonly FIELDS: StateFieldsType = {
'query': 'string',
'userLanguage': 'string',
'draft': 'string',
'failureCause': 'string',
'runId': 'string',
};
/** Raw question the visitor submitted. */
query = '';
/**
* Visitor's device language as an ISO 639-1 code (e.g. `'en'`,
* `'ja'`). Drives every LLM prompt's response-language directive
* and the language filter scouts apply to upstream results. Set by
* the entrypoint from `UserLanguage.detect()` (or a URL override);
* defaulted to `'en'` so existing call sites stay correct.
*/
userLanguage: string = 'en';
/** Parsed intent; set by `classifyIntent`. */
intent: ArchivistIntent = 'search';
/** Structured query terms; set by `extractQuery`. */
terms: readonly string[] = [];
/** Candidates returned by each scout, partitioned by source. */
candidates: readonly CandidateType[] = [];
/** Final shortlist after merge + dedupe + rank. */
shortlist: readonly CandidateType[] = [];
/** The Archivist's draft response. */
draft = '';
/**
* Validation lifecycle state for the current draft.
* 'pending' — not yet validated (initial state, reset by preRunSetup)
* 'approved' — LLM validator accepted the draft
* 'rejected' — validator rejected (retry or salvage path follows)
*/
approvalState: 'pending' | 'approved' | 'rejected' = 'pending';
/**
* ToolInterface plan emitted by the LLM via `decideTools`. The DAG inspects
* this to gate the optional scouts (web search runs only when the
* LLM asked for it). Empty = no tools needed.
*/
toolPlan: ReadonlyArray<{ readonly name: string; readonly arguments: Record<string, unknown> }> = [];
/**
* Per-run identifier. Used to subject every triple we write so the
* recall node can `SELECT` other runs' facts without re-reading the
* current run's findings.
*/
runId: string = '';
/**
* Sanitized one-liner description of why the search produced no
* results. Accumulated by scouts and gate nodes; consumed by
* `composeEmptyResponse` to craft an in-character failure message.
* Empty string when no failure has been recorded.
*/
failureCause = '';
/**
* Prior-context facts the recall node SELECTs out of memory before
* compose. Each entry has a `variant` (e.g. 'prior-query',
* 'prior-recommendation') and free-text content the LLM can cite.
*/
priorContext: ReadonlyArray<{ readonly variant: string; readonly text: string }> = [];
/**
* Structured context recalled from the unified memory graph by
* `recallContext` (runs before `classifyIntent`). The `summary` field
* is injected into the classifier prompt; all fields are available to
* downstream nodes (decideTools, composeResponse).
*/
recalledContext: RecalledContext = {
'priorIntents': [],
'recentCandidates': [],
'similarPriorQueries': [],
'priorReasoning': [],
'summary': '',
};
/**
* The N most recent turns of the conversation (visitor + archivist),
* sliced from the runner's display buffer and injected here before each
* run. The runner controls the window size; nodes read this to thread
* prior context into LLM prompts for pronoun resolution and continuity.
* Always initialised to `[]`; never undefined (V8 shape stability).
*/
conversation: readonly ConversationTurn[] = [];
/**
* Prior shortlisted candidates loaded from memory by `recallContext`
* (cap 5, low Jaccard) and overridden by `recallCandidates` inside the
* `book-search-scatter` embedded-DAG (cap 10, Jaccard >= 0.35).
* `mergeCandidates` uses this pool when live scouts return zero.
* Always initialized; never undefined (V8 shape stability).
*/
priorCandidates: readonly CandidateType[] = [];
/**
* Scatter workset built by BuildBookWorksetsNode before each scatter fan-out.
* Each entry carries a registered tool DAG IRI and the call
* arguments to pass to it. The scatter placement reads `dagIri` through an
* item-scoped DagReference to resolve the body DAG at runtime.
* Written fresh before every scatter; always array-typed (never undefined).
*/
bookWorksets: ReadonlyArray<BookWorksetItemType> = [];
/**
* The agent's own reasoning steps, accumulated across the current run via
* `ReasoningStep.create(...)`. Each step is provenance-linked by
* `RdfProvObserver.recordReasoning` into the PROV graph. Always
* initialized; never undefined (V8 shape stability).
*/
reasoning: readonly ReasoningStepType[] = [];
/**
* Memory roll-up produced by `recallMemories` for the `recall-memories`
* intent. Empty/zero-valued when the intent is not `recall-memories`.
*/
memoryDigest: MemoryDigest = {
'bookCount': 0,
'queryCount': 0,
'recentBooks': [],
'intentBreakdown': [],
'summary': '',
};
// #region clone
override clone(): this {
const copy = super.clone(); // new Constructor() + _metadata copy from base
copy.query = this.query;
copy.userLanguage = this.userLanguage;
copy.intent = this.intent;
copy.terms = [...this.terms];
copy.candidates = [...this.candidates];
copy.shortlist = [...this.shortlist];
copy.draft = this.draft;
copy.approvalState = this.approvalState;
copy.toolPlan = [...this.toolPlan];
copy.runId = this.runId;
copy.failureCause = this.failureCause;
copy.priorContext = [...this.priorContext];
copy.recalledContext = {
'priorIntents': [...this.recalledContext.priorIntents],
'recentCandidates': [...this.recalledContext.recentCandidates],
'similarPriorQueries': [...this.recalledContext.similarPriorQueries],
'priorReasoning': [...this.recalledContext.priorReasoning],
'summary': this.recalledContext.summary,
};
copy.conversation = [...this.conversation];
copy.priorCandidates = [...this.priorCandidates];
copy.bookWorksets = [...this.bookWorksets];
copy.reasoning = [...this.reasoning];
copy.memoryDigest = {
'bookCount': this.memoryDigest.bookCount,
'queryCount': this.memoryDigest.queryCount,
'recentBooks': [...this.memoryDigest.recentBooks],
'intentBreakdown': [...this.memoryDigest.intentBreakdown],
'summary': this.memoryDigest.summary,
};
return copy;
}
// #endregion clone
// #region snapshot-restore
protected override snapshotData(): JsonObjectType {
return {
...NodeStateBase.snapshotFields(this, ArchivistState.FIELDS),
"intent": this.intent,
"terms": [...this.terms],
"candidates": this.candidates.map(ArchivistState.candidateToJson),
"shortlist": this.shortlist.map(ArchivistState.candidateToJson),
"approvalState": this.approvalState,
"recalledContext": {
"priorIntents": this.recalledContext.priorIntents.map(ArchivistState.priorIntentToJson),
"recentCandidates": this.recalledContext.recentCandidates.map(ArchivistState.candidateToJson),
"similarPriorQueries": this.recalledContext.similarPriorQueries.map(ArchivistState.priorQueryToJson),
"priorReasoning": this.recalledContext.priorReasoning.map(ArchivistState.priorReasoningToJson),
"summary": this.recalledContext.summary,
},
"priorCandidates": this.priorCandidates.map(ArchivistState.candidateToJson),
"conversation": this.conversation.map(ArchivistState.turnToJson),
"bookWorksets": this.bookWorksets.map((w) => ({ "dagIri": w.dagIri, "arguments": w.arguments })),
"reasoning": this.reasoning.map(ArchivistState.reasoningStepToJson),
"memoryDigest": {
"bookCount": this.memoryDigest.bookCount,
"queryCount": this.memoryDigest.queryCount,
"recentBooks": this.memoryDigest.recentBooks.map((b) => ({ "title": b.title, "author": b.author })),
"intentBreakdown": this.memoryDigest.intentBreakdown.map((i) => ({ "intent": i.intent, "count": i.count })),
"summary": this.memoryDigest.summary,
},
};
}
// #region snapshot-helpers
private static candidateToJson(c: CandidateType): JsonObjectType {
const book: JsonObjectType = {
"isbn": c.book.identity.isbn,
"title": c.book.identity.title,
"authors": [...c.book.identity.authors],
"price": { "amount": c.book.availability.price.amount, "currency": c.book.availability.price.currency },
// Null-sentinel fields are omitted when null, so the wire shape carries a
// key only when a real value exists (not an explicit `null`).
...(c.book.publication.summary !== null ? { "summary": c.book.publication.summary } : {}),
...(c.book.publication.firstPublishYear !== null ? { "firstPublishYear": c.book.publication.firstPublishYear } : {}),
...(c.book.publication.subjects.length > 0 ? { "subjects": [...c.book.publication.subjects] } : {}),
...(c.book.publication.publishers.length > 0 ? { "publishers": [...c.book.publication.publishers] } : {}),
...(c.book.availability.inStock !== null ? { "inStock": c.book.availability.inStock } : {}),
...(c.book.publication.languages.length > 0 ? { "languages": [...c.book.publication.languages] } : {}),
};
// notes values are Record<string, unknown>; serialize only JSON-safe primitives.
const notesOut: JsonObjectType = {};
if (c.notes !== undefined) {
for (const [k, v] of Object.entries(c.notes)) {
if (v === null || typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean') {
notesOut[k] = v;
}
}
}
return {
"book": book,
"score": c.score,
"source": c.source,
...(c.reason !== undefined ? { "reason": c.reason } : {}),
...(c.notes !== undefined ? { "notes": notesOut } : {}),
};
}
private static priorIntentToJson(p: RecalledContext['priorIntents'][number]): JsonObjectType {
return { "query": p.query, "intent": p.intent, "ts": p.ts };
}
private static priorQueryToJson(q: RecalledContext['similarPriorQueries'][number]): JsonObjectType {
return { "query": q.query, "ts": q.ts };
}
private static turnToJson(t: ConversationTurn): JsonObjectType {
return { "role": t.role, "text": t.text, "ts": t.ts };
}
private static priorReasoningToJson(p: RecalledContext['priorReasoning'][number]): JsonObjectType {
return { "text": p.text, "kind": p.kind };
}
/**
* `ReasoningStepType.action.args` is `Record<string, unknown>` at the
* construction boundary; serialize only JSON-safe primitives, mirroring
* `candidateToJson`'s `notesOut` sanitizer.
*/
private static reasoningStepToJson(step: ReasoningStepType): JsonObjectType {
if (step.kind === 'action') {
const argsOut: JsonObjectType = {};
for (const [k, v] of Object.entries(step.args)) {
if (v === null || typeof v === 'string' || typeof v === 'number' || typeof v === 'boolean') {
argsOut[k] = v;
}
}
return { "kind": step.kind, "tool": step.tool, "args": argsOut };
}
if (step.kind === 'observation') {
return { "kind": step.kind, "output": step.output };
}
return { "kind": step.kind, "text": step.text };
}
// #endregion snapshot-helpers
protected override restoreData(snap: JsonObjectType): void {
NodeStateBase.restoreFields(this, snap, ArchivistState.FIELDS);
const rawIntent = snap['intent'];
if (ArchivistState.isIntent(rawIntent)) this.intent = rawIntent;
const approvalSnap = snap['approvalState'];
if (approvalSnap === 'pending' || approvalSnap === 'approved' || approvalSnap === 'rejected') {
this.approvalState = approvalSnap;
}
const rawTerms = snap['terms'];
if (Array.isArray(rawTerms) && rawTerms.every((x): x is string => typeof x === 'string')) {
this.terms = rawTerms;
}
const rawCandidates = snap['candidates'];
if (Array.isArray(rawCandidates)) {
this.candidates = ArchivistState.filterCandidates(rawCandidates);
}
const rawShortlist = snap['shortlist'];
if (Array.isArray(rawShortlist)) {
this.shortlist = ArchivistState.filterCandidates(rawShortlist);
}
const rc = snap['recalledContext'];
if (rc !== null && rc !== undefined && typeof rc === 'object' && !Array.isArray(rc)) {
const rawPriorIntents = rc['priorIntents'];
const rawRecentCandidates = rc['recentCandidates'];
const rawSimilarPriorQueries = rc['similarPriorQueries'];
const rawPriorReasoning = rc['priorReasoning'];
this.recalledContext = {
'priorIntents': Array.isArray(rawPriorIntents)
? ArchivistState.filterPriorIntents(rawPriorIntents)
: [],
'recentCandidates': Array.isArray(rawRecentCandidates)
? ArchivistState.filterCandidates(rawRecentCandidates)
: [],
'similarPriorQueries': Array.isArray(rawSimilarPriorQueries)
? ArchivistState.filterSimilarPriorQueries(rawSimilarPriorQueries)
: [],
'priorReasoning': Array.isArray(rawPriorReasoning)
? ArchivistState.filterPriorReasoning(rawPriorReasoning)
: [],
'summary': typeof rc['summary'] === 'string' ? rc['summary'] : '',
};
}
const rawPriorCandidates = snap['priorCandidates'];
if (Array.isArray(rawPriorCandidates)) {
this.priorCandidates = ArchivistState.filterCandidates(rawPriorCandidates);
}
const rawConversation = snap['conversation'];
if (Array.isArray(rawConversation)) {
this.conversation = ArchivistState.filterConversationTurns(rawConversation);
}
const rawBookWorksets = snap['bookWorksets'];
if (Array.isArray(rawBookWorksets)) {
this.bookWorksets = ArchivistState.filterBookWorksetItems(rawBookWorksets);
}
const rawReasoning = snap['reasoning'];
if (Array.isArray(rawReasoning)) {
this.reasoning = ArchivistState.filterReasoningSteps(rawReasoning);
}
const md = snap['memoryDigest'];
if (md !== null && md !== undefined && typeof md === 'object' && !Array.isArray(md)) {
const rawRecentBooks = md['recentBooks'];
const rawIntentBreakdown = md['intentBreakdown'];
this.memoryDigest = {
'bookCount': typeof md['bookCount'] === 'number' ? md['bookCount'] : 0,
'queryCount': typeof md['queryCount'] === 'number' ? md['queryCount'] : 0,
'recentBooks': Array.isArray(rawRecentBooks)
? ArchivistState.filterRecentBooks(rawRecentBooks)
: [],
'intentBreakdown': Array.isArray(rawIntentBreakdown)
? ArchivistState.filterIntentBreakdown(rawIntentBreakdown)
: [],
'summary': typeof md['summary'] === 'string' ? md['summary'] : '',
};
}
}
// #region type-guards
private static filterCandidates(arr: unknown[]): CandidateType[] {
const out: CandidateType[] = [];
for (const item of arr) {
if (ArchivistState.isCandidate(item)) out.push(item);
}
return out;
}
private static filterPriorIntents(arr: unknown[]): RecalledContext['priorIntents'] {
const out: RecalledContext['priorIntents'][number][] = [];
for (const item of arr) {
if (ArchivistState.isPriorIntent(item)) out.push(item);
}
return out;
}
private static filterSimilarPriorQueries(arr: unknown[]): RecalledContext['similarPriorQueries'] {
const out: RecalledContext['similarPriorQueries'][number][] = [];
for (const item of arr) {
if (ArchivistState.isSimilarPriorQuery(item)) out.push(item);
}
return out;
}
private static filterConversationTurns(arr: unknown[]): ConversationTurn[] {
const out: ConversationTurn[] = [];
for (const item of arr) {
if (ArchivistState.isConversationTurn(item)) out.push(item);
}
return out;
}
private static filterBookWorksetItems(arr: unknown[]): BookWorksetItemType[] {
const out: BookWorksetItemType[] = [];
for (const item of arr) {
if (ArchivistState.isBookWorksetItem(item)) out.push(item);
}
return out;
}
private static filterPriorReasoning(arr: unknown[]): RecalledContext['priorReasoning'] {
const out: RecalledContext['priorReasoning'][number][] = [];
for (const item of arr) {
if (ArchivistState.isPriorReasoning(item)) out.push(item);
}
return out;
}
private static filterReasoningSteps(arr: unknown[]): ReasoningStepType[] {
const out: ReasoningStepType[] = [];
for (const item of arr) {
if (ArchivistState.isReasoningStep(item)) out.push(item);
}
return out;
}
private static filterRecentBooks(arr: unknown[]): MemoryDigest['recentBooks'] {
const out: MemoryDigest['recentBooks'][number][] = [];
for (const item of arr) {
if (ArchivistState.isRecentBook(item)) out.push(item);
}
return out;
}
private static filterIntentBreakdown(arr: unknown[]): MemoryDigest['intentBreakdown'] {
const out: MemoryDigest['intentBreakdown'][number][] = [];
for (const item of arr) {
if (ArchivistState.isIntentBreakdownEntry(item)) out.push(item);
}
return out;
}
private static isIntent(v: unknown): v is ArchivistIntent {
return v === 'lookup-author'
|| v === 'find-reviews'
|| v === 'describe-book'
|| v === 'recommend-similar'
|| v === 'recall-memories'
|| v === 'search'
|| v === 'describe'
|| v === 'recommend'
|| v === 'off-topic';
}
private static isCandidate(v: unknown): v is CandidateType {
return ArchivistState.candidateValidator.is(v);
}
private static isPriorIntent(v: unknown): v is RecalledContext['priorIntents'][number] {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('query' in v && 'intent' in v && 'ts' in v)) return false;
return typeof v.query === 'string'
&& typeof v.intent === 'string'
&& typeof v.ts === 'string';
}
private static isSimilarPriorQuery(v: unknown): v is RecalledContext['similarPriorQueries'][number] {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('query' in v && 'ts' in v)) return false;
return typeof v.query === 'string' && typeof v.ts === 'string';
}
private static isConversationTurn(v: unknown): v is ConversationTurn {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('role' in v && 'text' in v && 'ts' in v)) return false;
return (v.role === 'visitor' || v.role === 'archivist')
&& typeof v.text === 'string'
&& typeof v.ts === 'number';
}
private static isBookWorksetItem(v: unknown): v is BookWorksetItemType {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('dagIri' in v && 'arguments' in v)) return false;
return typeof v.dagIri === 'string'
&& typeof v.arguments === 'object'
&& v.arguments !== null
&& !Array.isArray(v.arguments);
}
private static isRecentBook(v: unknown): v is MemoryDigest['recentBooks'][number] {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('title' in v && 'author' in v)) return false;
return typeof v.title === 'string' && typeof v.author === 'string';
}
private static isIntentBreakdownEntry(v: unknown): v is MemoryDigest['intentBreakdown'][number] {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('intent' in v && 'count' in v)) return false;
return typeof v.intent === 'string' && typeof v.count === 'number';
}
private static isPriorReasoning(v: unknown): v is RecalledContext['priorReasoning'][number] {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('text' in v && 'kind' in v)) return false;
return typeof v.text === 'string' && typeof v.kind === 'string';
}
private static isReasoningStep(v: unknown): v is ReasoningStepType {
if (typeof v !== 'object' || v === null || Array.isArray(v)) return false;
if (!('kind' in v)) return false;
if (v.kind === 'thought' || v.kind === 'final') {
return 'text' in v && typeof v.text === 'string';
}
if (v.kind === 'action') {
return 'tool' in v && typeof v.tool === 'string'
&& 'args' in v && typeof v.args === 'object' && v.args !== null && !Array.isArray(v.args);
}
if (v.kind === 'observation') {
return 'output' in v && typeof v.output === 'string';
}
return false;
}
// #endregion type-guards
}Prompts (composable directives)
ts
/**
* prompts.ts: every prompt the Archivist sends, composed from small
* directive primitives.
*
* Directive = one short positive instruction (an "attractor")
* Prompt = a list of directives + slots, joined deterministically
* Schema = the data contract that pairs with a prompt
*
* Rules of the road:
* • Every prompt is built here. No other module assembles natural-language.
* • Directives state what to DO, not what to avoid (attractors beat repulsors).
* • Builder bodies contain ONLY directive references + slot interpolations
* + paragraph-break empty strings. Every static instructional line is a
* named primitive in the `directives` registry.
* • Examples in schemas describe SHAPE, never real-world content,
* so models can't quote example data back into the conversation.
* • Persistent memory is INERT context; the directive only encourages
* citation when the visitor explicitly references their past.
*/
import type { ConversationTurn, MemoryDigest } from '../ArchivistState.ts';
import type { CandidateType } from '../entities/Book.ts';
import { UserLanguage } from '../language/UserLanguage.ts';
// ── Directive primitives ────────────────────────────────────────────────
/** Composable directive lines. Keep them positive, terse, and orthogonal. */
export const directives = {
// ── Persona ──────────────────────────────────────────────────────────
// All directives are positive imperatives: tell the model what the
// Archivist DOES and HOW it speaks. Attractors bind tighter than
// repulsors. Describe the role, the data sources, and the response
// shape; the model will inhabit that frame rather than fight it.
"persona": 'You are the Archivist, a research librarian. You have global catalog access through OpenLibrary, Google Books, and Wikipedia and you can look up, describe, and discuss any published work.',
"scope": 'Help the visitor find, describe, and compare books, working from the catalog records you just retrieved (listed below).',
"catalogAuthority": 'When a visitor names or cites a title, treat it as a catalog reference. Pull the title, author, publication year, subjects, and any available summary or notes from the records below and weave them into your reply.',
"speakAsLibrarian": 'Speak as a librarian who has just consulted the catalog: cite what the records say, summarise themes and reception when notes are present, and invite the visitor to explore adjacent records.',
"specialty": 'Your particular depth is science fiction and philosophy. You are equally fluent in any genre when the catalog returns it.',
"declineOffTopic": 'For questions unrelated to books or reading, redirect to a literary topic the visitor has shown interest in.',
// ── Response style ───────────────────────────────────────────────────
"beTerse": 'Reply in 2–3 sentences.',
"conversational": 'Reply in flowing prose as a librarian speaking aloud. Weave the catalog records you cite into your sentences naturally: title-case the work, mention the author when it matters, drop the rest. Headings, bullet lists, and numbered enumerations belong in a printed bibliography, not in conversation.',
"citeShortlist": 'Every title you cite comes from the catalog records below; they are the records the visitor is asking about.',
"groundInShortlist":'Build each sentence from the metadata in the records below: title, author, year, subjects, notes, source.',
"clarifyOnDoubt": 'If no records were returned or the question is ambiguous, ask a single clarifying question.',
"memoryAsContext": 'Treat persistent memory as background only. Mention it when the visitor says "last time" / "earlier" / "I mentioned before".',
"emitJsonOnly": 'Return JSON that satisfies the supplied schema. No surrounding prose.',
"pickTerseQuery": 'Pick a terse search query: title, author, ISBN, or two-to-five topic keywords.',
"chronological": 'Present the works in publication order, oldest first.',
"weightRatings": 'Weight ratings (notes.rating + notes.ratingsCount) when scoring; high counts of high ratings boost score.',
"describeOnly": 'Describe the book in two sentences using the catalog metadata supplied. Stay on the cited title.',
"authorSurvey": 'Treat the records below as one author\'s body of work; sketch its arc, not a single recommendation.',
"similarToPrior": 'Frame each suggestion as "similar to <prior title>" using the persistent-memory facts as the anchor.',
"weighOpinions": 'Quote average ratings and ratings counts when present; explain what readers seem to feel about each title.',
"continuityHint": 'Use the recent context if it suggests a likely intent or recurring interest.',
"recallMemories": 'When the visitor asks what you remember, what books you have seen, or what they have asked before, give a warm roll-up of your memory.',
"ownTheGap": 'Acknowledge which sources were searched. Explain in one sentence why nothing matched. Offer one concrete alternative angle the visitor could try. Never refuse or decline; you are a helpful librarian, not a content filter. Acknowledge that no exact match was found, then offer to recommend something or ask the visitor for a genre or author they enjoy.',
// ── Slot labels ──────────────────────────────────────────────────────
"visitorQuestionLabel": 'Visitor question:',
"recentContextLabel": 'Recent context:',
"conversationContextLabel":'Conversation context:',
"searchNotesLabel": 'Search notes:',
"shortlistedTitlesLabel": 'Catalog records returned:',
"draftLabel": 'Draft:',
"candidatesPlainHeader": 'Catalog records:',
"matchedBooksHeader": 'Catalog records for this title:',
// ── Intent classification ────────────────────────────────────────────
"intentEnumerationHeader": 'Classify the visitor question as exactly one of the following intents:',
"intentEnumeration": [
' lookup-author : the visitor named an author and wants their body of work',
' find-reviews : the visitor wants opinions, reviews, or what readers think',
' describe-book : the visitor named a specific existing title by name and wants a description of that exact book',
' recommend-similar : the visitor wants something like a previous read',
' recall-memories : the visitor asks about your own memory or history: what books you have looked up, what they have asked before, what has been recommended; any meta-question about your past activity',
' search : the visitor named a topic / title / ISBN (no clear sub-case)',
' describe : the visitor described a book without naming it',
' recommend : the visitor asked for a good book or a good story to read without naming a title or genre (a generic recommendation)',
' off-topic : the visitor asked something unrelated to books and unrelated to your memory',
].join('\n'),
"intentExamplesHeader": 'Examples:',
"intentExamples": [
' "do you have anything exploring the ethics of AI, maybe with a sci-fi bent?" → search',
' "what should I read after Project Hail Mary?" → recommend-similar',
' "recommend something similar to Dune" → recommend-similar',
' "tell me about The Sun Also Rises" → describe-book',
' "what did Murakami write?" → lookup-author',
' "anything good in cosy fantasy?" → recommend',
' "tell me a good story" → recommend',
' "what\'s a good book?" → recommend',
' "recommend a good read" → recommend',
' "what was that book I asked about last week?" → recall-memories',
' "use the web search tools to find me a book" → search',
' "search the web for books about stoicism" → search',
' "what time is it?" → off-topic',
' "what is the weather like?" → off-topic',
' "try again" / "another one" / "different" / "no" → REUSE THE PRIOR INTENT from recent context if any, otherwise default to `search`',
].join('\n'),
"intentRules": 'Rules: prefer the most specific intent. Treat short follow-up phrases ("try again", "next", "no", "different") as continuations of the previous intent; never classify them as off-topic. If the visitor explicitly asks for tools, web search, lookups, or external sources, classify as `search`, NEVER `off-topic`. Off-topic is ONLY for queries clearly unrelated to books or reading (weather, sports scores, jokes, recipes, news). Anything book-adjacent, tool-related, or meta about the assistant is on-topic.',
"intentResponseFormat": 'Respond with the single token only.',
// ── Term extraction ──────────────────────────────────────────────────
"extractTermsTask": [
'Distill the visitor question into 2-4 catalog-searchable domain keywords.',
'Strip filler words (do, you, have, any, tell, me, about, like, want, looking, for, please, thanks).',
'Strip generic nouns like "book(s)", "novel(s)", "title(s)", "question(s)"; those don\'t narrow a catalog search.',
'Normalize abbreviations: "sci-fi" → "science fiction", "AI" → "artificial intelligence".',
'Keep proper nouns intact (author names, book titles).',
'',
'Examples:',
' "Do you have any sci-fi novels that grapple with existential questions?"',
' → ["existentialism", "science fiction"]',
' "I like robots and singularity"',
' → ["robots", "singularity"]',
' "Yea tell me about Neuromancer"',
' → ["Neuromancer"]',
' "Recommend a book by Ursula K. Le Guin about morality"',
' → ["Ursula K. Le Guin", "morality"]',
' "What books did Philip K. Dick write about androids?"',
' → ["Philip K. Dick", "androids"]',
].join('\n'),
"jsonArrayOnly": 'Return ONLY a JSON array of strings.',
// ── ToolInterface decision ────────────────────────────────────────────────────
"callAllToolsForAuthor": 'For any visitor question that names an author or describes a book to find, call ALL of the available tools; do not omit any source.',
"shortKeywordQuery": 'Use a short, keyword-only query (no surrounding quotes, no filler phrases).',
// ── Compose-side candidate headers ───────────────────────────────────
"candidatesHeader": 'Catalog records (cite in flowing prose; the order reflects ranking):',
"candidatesHeaderChronological": 'Catalog records (cite in flowing prose; the order is chronological):',
"candidatesHeaderRated": 'Catalog records (cite in flowing prose; the order reflects reader ratings):',
"persistentMemoryHeader": 'PERSISTENT MEMORY — your own findings from earlier sessions, not the visitor\'s words (background only; cite only on explicit recall request):',
"persistentMemoryAnchorHeader": 'PERSISTENT MEMORY — your own findings from earlier sessions, not the visitor\'s words (anchor; cite explicitly as the basis for similarity):',
// ── Validation ───────────────────────────────────────────────────────
"validateApprovalRule": 'Approve if the draft (a) cites a title from the catalog records and (b) reads as a polite on-topic reply.',
"validateResponseFormat": 'Reply with the single token "yes" or "no".',
// ── Starter / greeting / visitor-reply suggestion ────────────────────
"starterGenrePool": 'Pick one acclaimed work or author from science fiction or philosophy at random. Examples of the genre frame: Liu Cixin\'s Three Body Problem, William Gibson\'s Neuromancer, Ursula K. Le Guin, Stanisław Lem, Ted Chiang, Jorge Luis Borges, Albert Camus, Michel Foucault, Gilles Deleuze, Ludwig Wittgenstein. Pick something in that vein but vary your selection.',
"starterPhraseInstruction":'Phrase ONE short curious question a first-time visitor to a bookstore might ask about it.',
"starterLengthLimit": 'The question must be under 20 words.',
"starterReturnFormat": 'Return just the question, with no preamble, no quotation marks, and no explanation.',
// Visitor persona: the leading system message for the bootstrap suggestion
// calls. A `role: 'system'` message makes `BaseAdapter.#withDefaultSystemPrompt`
// skip its default Archivist directive injection, so a weak model writes as the visitor
// rather than echoing a librarian greeting.
"visitorPersona": 'You are a curious visitor approaching The Archivist with book questions. Generate one short, natural visitor message as directed.',
"greetingInstruction": 'Write ONE fresh opening greeting for a new visitor walking into the shop.',
"greetingTone": 'The greeting must be warm, curious, and invite a book question.',
"greetingLengthLimit": 'Keep it under 30 words.',
"greetingReturnFormat": 'Return just the greeting, with no preamble, no quotation marks, and no explanation.',
"visitorReplyContextLine": 'A bookshop visitor has just received this greeting from the Archivist:',
"visitorReplyInterest": 'The visitor is interested in science fiction and philosophy.',
"visitorReplyInstruction": 'Write ONE natural first message the visitor might send in reply.',
"visitorReplyContent": 'The reply must be a book question or request that follows naturally from the greeting.',
"visitorReplyLengthLimit": 'Keep it under 30 words.',
"visitorReplyReturnFormat": 'Return just the visitor message, with no preamble, no quotation marks, and no explanation.',
// ── ToolInterface explanation ─────────────────────────────────────────────────
"explainToolPersona": 'You are a librarian explaining a backend tool to a curious visitor.',
"explainToolInstruction": 'Explain in 2-3 plain-English sentences:',
"explainToolPoint1": '1. What the tool does',
"explainToolPoint2": '2. Why it matters',
"explainToolPoint3": '3. One concrete example use-case',
"explainToolTone": 'Keep it warm and clear. No jargon. Under 80 words.',
"explainToolReturnFormat": 'Return just the explanation, no preamble.',
// ── Memory recall ────────────────────────────────────────────────────
"memoryEmptyStatus": 'Memory status: my shelves are fresh. No books have been recorded yet this session.',
// ── Prior memory hint ────────────────────────────────────────────────
/**
* Injected when any candidate in the shortlist carries
* `notes.fromPriorMemory: true`. Instructs the model to phrase those
* recalls as "I recall from earlier" rather than "I just searched".
*/
"priorMemoryHint": 'Some of these books come from prior sessions where you discussed similar queries; phrase them as "I recall" or "from earlier we found" rather than "I just searched".',
// ── Compose repair ───────────────────────────────────────────────────
/**
* Injected when a previous compose attempt returned raw JSON instead
* of prose (a failure mode of weak on-device models like Gemini Nano).
* Instructs the model to write only flowing natural language.
*/
"repairJson": 'Write only flowing prose. Do not output JSON, code blocks, or any structured data format.',
} as const;
// ── Shared system message, composed from persona directives ────────────
// Positive imperatives only. Describe what the Archivist DOES; the model
// inhabits that frame rather than negotiating around prohibitions.
const SYSTEM = [
directives.persona,
directives.scope,
directives.catalogAuthority,
directives.speakAsLibrarian,
directives.specialty,
directives.declineOffTopic,
directives.beTerse,
directives.conversational,
directives.citeShortlist,
directives.groundInShortlist,
directives.clarifyOnDoubt,
directives.memoryAsContext,
].join(' ');
// ── Output schemas (the data contract, paired with prompts) ───────────
//
// Index-pointer schemas: the LLM emits flat integer arrays that point at
// items in the pre-numbered prompt lists. Deterministic code in
// `BaseLlmClient` materialises the full records from those pointers.
//
// This shape is dramatically faster for slow constrained-output backends
// (Gemini Nano, WebLLM) because `responseConstraint` only validates a
// short int array instead of every field of every record.
export const schemas = {
"rankCandidates": {
'type': 'object',
'description': 'Order candidates best-to-worst by 1-based index into the candidate list.',
'additionalProperties': false,
'properties': {
'order': {
'type': 'array',
'description': 'Indices (1-based) into the candidate list above, in best-to-worst order. Each value 1 <= n <= N. No duplicates.',
'items': {
'type': 'integer',
'minimum': 1,
},
},
},
'required': ['order'],
} satisfies Record<string, unknown>,
"decideTools": {
'type': 'object',
'description': 'Pick tools by 1-based index into the numbered tool list.',
'additionalProperties': false,
'properties': {
'tools': {
'type': 'array',
'description': 'Indices (1-based) into the numbered tool list above, in any order. Empty array means no tools.',
'items': {
'type': 'integer',
'minimum': 1,
},
},
},
'required': ['tools'],
} satisfies Record<string, unknown>,
};
// ── Prompt builders ────────────────────────────────────────────────────
/** Helpers expose only the builders; nodes never assemble prose themselves. */
export const prompts = {
/**
* The language-independent shared system prompt: persona, scope, catalog
* authority, librarian voice, specialty, response style, shortlist grounding,
* and memory-as-context rules — the whole standing frame, not just a persona
* line. The adapter injects it as a leading system message via the
* `BaseAdapter.systemPrompt` seam, so pipeline bodies no longer prepend it
* inline; pass this value to the adapter constructor once and it arrives as a
* real leading system turn on every backend.
*/
systemPrompt(): string { return SYSTEM; },
classifyIntent(language: string, query: string, recalledSummary?: string, conversation: readonly ConversationTurn[] = []): string {
const contextBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: [
'',
`${directives.recentContextLabel} ${recalledSummary} ${directives.continuityHint}`,
].join('\n');
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const body = [
directives.intentEnumerationHeader,
directives.intentEnumeration,
'',
directives.intentExamplesHeader,
directives.intentExamples,
'',
directives.intentRules,
directives.intentResponseFormat,
contextBlock,
conversationBlock,
'',
`${directives.visitorQuestionLabel} ${query}`,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
extractTerms(language: string, query: string): string {
const body = [
directives.extractTermsTask,
directives.jsonArrayOnly,
'',
`${directives.visitorQuestionLabel} ${query}`,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
decideTools(
language: string,
query: string,
available: readonly { name: string; description: string }[],
): string {
// Index-pointer schema: the LLM picks tools by 1-based index into a
// numbered list rendered in the prompt. ToolInterface arguments are
// synthesised deterministically by `BaseLlmClient.decideTools` from
// `state.query` and `state.userLanguage`; the model never touches
// arguments. Massive token savings vs the per-call adapter tools
// channel on Nano / WebLLM.
const toolList = available
.map((t, i) => ` ${String(i + 1)}. ${t.name}: ${t.description}`)
.join('\n');
const body = [
directives.emitJsonOnly,
'',
'Available tools:',
toolList,
'',
`Reply with {"tools": [n, n, ...]} where each n is a tool number from the list above. Include every tool you want to call (use all that apply). Use [] for no tools.`,
'',
`${directives.visitorQuestionLabel} ${query}`,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
rankCandidates(language: string, query: string, candidates: readonly CandidateType[]): string {
const rows = candidates.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const body = [
directives.emitJsonOnly,
'',
`${directives.visitorQuestionLabel} ${query}`,
'',
directives.candidatesPlainHeader,
rows,
'',
`Reply with {"order": [n, n, n, ...]} where each n is a candidate number from the list above, ordered best to worst. No duplicates, no other fields.`,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
compose(
language: string,
query: string,
shortlist: readonly CandidateType[],
priorContext?: readonly { variant: string; text: string }[],
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
repairHint = '',
): string {
const rows = shortlist.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const contextBlock = (priorContext === undefined || priorContext.length === 0)
? ''
: [
'',
directives.persistentMemoryHeader,
...priorContext.map((p) => `- [${p.variant}] ${p.text}`),
].join('\n');
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const memoryHint = PromptFormat.priorMemoryHintLine(shortlist);
const repairLines: string[] = repairHint.length > 0
? [
'',
directives.repairJson,
`A previous attempt returned raw JSON instead of prose. Do NOT output JSON or code blocks. Write flowing prose only. Data in plain text: ${repairHint}`,
]
: [];
const body = [
directives.beTerse,
directives.citeShortlist,
...(memoryHint.length > 0 ? [memoryHint] : []),
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
contextBlock,
...repairLines,
'',
directives.candidatesHeader,
rows,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
composeAuthor(
language: string,
query: string,
shortlist: readonly CandidateType[],
priorContext?: readonly { variant: string; text: string }[],
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
repairHint = '',
): string {
const rows = shortlist.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const contextBlock = (priorContext === undefined || priorContext.length === 0)
? ''
: [
'',
directives.persistentMemoryHeader,
...priorContext.map((p) => `- [${p.variant}] ${p.text}`),
].join('\n');
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const memoryHintAuthor = PromptFormat.priorMemoryHintLine(shortlist);
const repairLines: string[] = repairHint.length > 0
? [
'',
directives.repairJson,
`A previous attempt returned raw JSON instead of prose. Do NOT output JSON or code blocks. Write flowing prose only. Data in plain text: ${repairHint}`,
]
: [];
const body = [
directives.beTerse,
directives.citeShortlist,
directives.chronological,
directives.authorSurvey,
...(memoryHintAuthor.length > 0 ? [memoryHintAuthor] : []),
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
contextBlock,
...repairLines,
'',
directives.candidatesHeaderChronological,
rows,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
composeReviews(
language: string,
query: string,
shortlist: readonly CandidateType[],
priorContext?: readonly { variant: string; text: string }[],
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
repairHint = '',
): string {
const rows = shortlist.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const contextBlock = (priorContext === undefined || priorContext.length === 0)
? ''
: [
'',
directives.persistentMemoryHeader,
...priorContext.map((p) => `- [${p.variant}] ${p.text}`),
].join('\n');
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const memoryHintReviews = PromptFormat.priorMemoryHintLine(shortlist);
const repairLines: string[] = repairHint.length > 0
? [
'',
directives.repairJson,
`A previous attempt returned raw JSON instead of prose. Do NOT output JSON or code blocks. Write flowing prose only. Data in plain text: ${repairHint}`,
]
: [];
const body = [
directives.beTerse,
directives.citeShortlist,
directives.weightRatings,
directives.weighOpinions,
...(memoryHintReviews.length > 0 ? [memoryHintReviews] : []),
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
contextBlock,
...repairLines,
'',
directives.candidatesHeaderRated,
rows,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
describeBook(
language: string,
query: string,
shortlist: readonly CandidateType[],
priorContext?: readonly { variant: string; text: string }[],
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
repairHint = '',
): string {
const rows = shortlist.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const contextBlock = (priorContext === undefined || priorContext.length === 0)
? ''
: [
'',
directives.persistentMemoryHeader,
...priorContext.map((p) => `- [${p.variant}] ${p.text}`),
].join('\n');
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const memoryHintDescribe = PromptFormat.priorMemoryHintLine(shortlist);
const repairLines: string[] = repairHint.length > 0
? [
'',
directives.repairJson,
`A previous attempt returned raw JSON instead of prose. Do NOT output JSON or code blocks. Write flowing prose only. Data in plain text: ${repairHint}`,
]
: [];
const body = [
directives.describeOnly,
directives.citeShortlist,
directives.groundInShortlist,
...(memoryHintDescribe.length > 0 ? [memoryHintDescribe] : []),
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
contextBlock,
...repairLines,
'',
directives.matchedBooksHeader,
rows,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
composeSimilar(
language: string,
query: string,
shortlist: readonly CandidateType[],
priorContext?: readonly { variant: string; text: string }[],
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
repairHint = '',
): string {
const rows = shortlist.map((c, i) => PromptFormat.formatCandidateRow(i + 1, c)).join('\n');
const contextBlock = (priorContext === undefined || priorContext.length === 0)
? ''
: [
'',
directives.persistentMemoryAnchorHeader,
...priorContext.map((p) => `- [${p.variant}] ${p.text}`),
].join('\n');
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const memoryHintSimilar = PromptFormat.priorMemoryHintLine(shortlist);
const repairLines: string[] = repairHint.length > 0
? [
'',
directives.repairJson,
`A previous attempt returned raw JSON instead of prose. Do NOT output JSON or code blocks. Write flowing prose only. Data in plain text: ${repairHint}`,
]
: [];
const body = [
directives.beTerse,
directives.citeShortlist,
directives.similarToPrior,
...(memoryHintSimilar.length > 0 ? [memoryHintSimilar] : []),
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
contextBlock,
...repairLines,
'',
directives.candidatesHeader,
rows,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
composeEmptyResponse(language: string, query: string, failureCause: string, conversation: readonly ConversationTurn[] = []): string {
const causeBlock = failureCause.trim().length > 0
? `\n${directives.searchNotesLabel} ${failureCause.trim()}`
: '';
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const body = [
directives.ownTheGap,
directives.beTerse,
'',
`${directives.visitorQuestionLabel} ${query}`,
conversationBlock,
causeBlock,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
validate(language: string, draft: string, shortlist: readonly CandidateType[]): string {
const titles = shortlist.map((c) => c.book.identity.title).join(' | ');
const body = [
directives.validateApprovalRule,
directives.validateResponseFormat,
'',
`${directives.shortlistedTitlesLabel} ${titles}`,
'',
`${directives.draftLabel} ${draft}`,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
/** System directive for the visitor-role bootstrap calls (starter query, visitor reply). */
visitorPersona(): string { return directives.visitorPersona; },
suggestStarterQuery(language: string): string {
// Runs under the visitorPersona() system message, so the Archivist directive
// directives are intentionally omitted here; starterGenrePool supplies the
// full genre frame.
const body = [
directives.starterGenrePool,
directives.starterPhraseInstruction,
directives.starterLengthLimit,
directives.starterReturnFormat,
].join(' ');
return PromptFormat.withLanguagePreamble(language, body);
},
suggestGreeting(language: string): string {
const body = [
directives.persona,
directives.specialty,
directives.greetingInstruction,
directives.greetingTone,
directives.greetingLengthLimit,
directives.greetingReturnFormat,
].join(' ');
return PromptFormat.withLanguagePreamble(language, body);
},
suggestVisitorReplyTo(language: string, greeting: string): string {
const body = [
directives.visitorReplyContextLine,
`"${greeting}"`,
directives.visitorReplyInterest,
directives.visitorReplyInstruction,
directives.visitorReplyContent,
directives.visitorReplyLengthLimit,
directives.visitorReplyReturnFormat,
].join(' ');
return PromptFormat.withLanguagePreamble(language, body);
},
explainTool(language: string, name: string, context: string): string {
const body = [
directives.explainToolPersona,
`The tool is called "${name}".`,
`Here is what it does: ${context}`,
directives.explainToolInstruction,
directives.explainToolPoint1,
directives.explainToolPoint2,
directives.explainToolPoint3,
directives.explainToolTone,
directives.explainToolReturnFormat,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
composeMemoryRecall(
language: string,
query: string,
digest: MemoryDigest,
recalledSummary?: string,
conversation: readonly ConversationTurn[] = [],
): string {
const continuityBlock = (recalledSummary === undefined || recalledSummary.length === 0)
? ''
: `\n${directives.conversationContextLabel} ${recalledSummary}`;
const conversationBlock = PromptFormat.formatConversationBlock(conversation);
const digestBlock = digest.bookCount === 0
? directives.memoryEmptyStatus
: [
`Memory status: ${String(digest.bookCount)} distinct book${digest.bookCount === 1 ? '' : 's'} recorded, ${String(digest.queryCount)} visitor ${digest.queryCount === 1 ? 'query' : 'queries'} seen.`,
digest.recentBooks.length > 0
? `Recent titles: ${digest.recentBooks.map((b) => `"${b.title}"${b.author !== undefined ? ` by ${b.author}` : ''}`).join('; ')}.`
: '',
digest.intentBreakdown.length > 0
? `Intent breakdown: ${digest.intentBreakdown.map((e) => `${e.intent} (${String(e.count)})`).join(', ')}.`
: '',
].filter(Boolean).join(' ');
const body = [
directives.recallMemories,
directives.beTerse,
'',
`${directives.visitorQuestionLabel} ${query}`,
continuityBlock,
conversationBlock,
'',
digestBlock,
].join('\n');
return PromptFormat.withLanguagePreamble(language, body);
},
};
// ── Internals ──────────────────────────────────────────────────────────
/** Prompt formatting utilities: language preamble, conversation blocks, candidate rows. */
export class PromptFormat {
/**
* Prepend a single language directive to every prompt body. Single
* source of truth for the language instruction so we can evolve the
* exact phrasing in one place.
*
* The directive instructs the model to:
* • respond in the user's device language;
* • use that language for every natural-language field in any JSON
* output (descriptions, ranking reasons, draft responses);
* • not echo translations of the input; respond directly in the
* target language.
*/
static withLanguagePreamble(language: string, body: string): string {
const code = UserLanguage.normalize(language);
const name = UserLanguage.displayName(code);
const preamble = [
`You communicate in ${name} (${code}). Every word you output, including`,
'JSON field values that contain natural language (book descriptions, ranking',
`reasons, draft responses), MUST be in ${name}. Do not output translations`,
`or transliterations of the user's input. Respond directly in ${name}.`,
].join('\n');
return `${preamble}\n\n${body}`;
}
/**
* Returns the `priorMemoryHint` directive line when any candidate in the
* shortlist carries `notes.fromPriorMemory: true`. Returns empty string
* otherwise so callers can splice it directly into the prompt body array.
*/
static priorMemoryHintLine(shortlist: readonly CandidateType[]): string {
const hasPriorMemory = shortlist.some((c) => c.notes?.['fromPriorMemory'] === true);
return hasPriorMemory ? directives.priorMemoryHint : '';
}
/** Format prior conversation turns as a terse "Conversation so far" block. */
static formatConversationBlock(turns: readonly ConversationTurn[]): string {
if (turns.length === 0) return '';
// Attribute each line to its speaker explicitly. "Visitor" is the person
// the Archivist is helping; "You" is the Archivist's own earlier words. A
// weak model otherwise reads its own prior `archivist:` line and echoes the
// title back as "you mentioned <title>", misattributing the source of data.
const lines = turns
.map((t) => (t.role === 'visitor' ? ` Visitor said: ${t.text}` : ` You (the Archivist) said: ${t.text}`))
.join('\n');
return `\nConversation so far (most recent last); attribute each line to its speaker:\n${lines}`;
}
static formatCandidateRow(n: number, c: CandidateType): string {
const parts: string[] = [];
parts.push(`${String(n)}. isbn=${c.book.identity.isbn}`);
parts.push(`"${c.book.identity.title}"`);
parts.push(`by ${c.book.identity.authors.join(', ') || '<unknown author>'}`);
if (c.book.publication.firstPublishYear !== null) parts.push(`(${String(c.book.publication.firstPublishYear)})`);
if (c.book.publication.subjects.length > 0) {
parts.push(`subjects: ${c.book.publication.subjects.slice(0, 5).join(', ')}`);
}
if (c.book.publication.publishers.length > 0) {
parts.push(`pub: ${c.book.publication.publishers[0]}`);
}
if (c.book.publication.summary !== null && c.book.publication.summary.length > 0) {
parts.push(`summary: ${c.book.publication.summary}`);
}
if (c.reason !== undefined && c.reason.length > 0) {
parts.push(`[rank-reason: ${c.reason}]`);
}
return parts.join(' | ');
}
}Classification node
ts
/**
* classifyIntent: entry node. Asks the LLM to classify the visitor's
* question, then routes one of seven on-topic branches plus the
* off-topic exit:
*
* lookup-author → `lookup-author-web-search` (chronological author survey)
* find-reviews → `find-reviews` (ratings tool branch)
* describe-book → `describe-web-search` (one-hit description branch)
* recommend-similar → `recommend-similar` (prior-shortlist seeding branch)
* recommend → `recommend-top-rated` (rating-ranked branch for vague "good book" asks)
* search | describe → `extract-query` (general on-topic pipeline)
* off-topic → `decline-off-topic`
*
* Demonstrates: a wide narrowly-typed output union and dispatch into
* embedded-DAG branches based on classifier output.
*/
// #region node-class
import type { ArchivistState } from '../ArchivistState.ts';
import type { ArchivistServices, ClassifiedIntent } from '../services.ts';
import { Batch, MonadicNode, NodeOutput, ReasoningStep, RoutedBatch } from '@studnicky/dagonizer';
import type { ItemType, NodeContextType, SchemaObjectType } from '@studnicky/dagonizer';
import { Signal } from '@studnicky/signal';
type IntentOutput =
| 'lookup-author'
| 'find-reviews'
| 'describe-book'
| 'recommend-similar'
| 'recall-memories'
| 'on-topic'
| 'recommend-top-rated'
| 'off-topic'
| 'retry'
| 'salvage';
/** Per-node timeout: generous for Gemini Nano's constrained-output path (20-60 s typical). */
const NODE_TIMEOUT_MS = 30_000;
/** Total attempts (initial + retries) before routing to salvage. */
const RETRY_BUDGET = 2;
export class ClassifyIntentNode extends MonadicNode<ArchivistState, IntentOutput> {
private readonly services: ArchivistServices;
readonly name = 'classify-intent';
readonly '@id' = 'urn:noocodec:node:classify-intent';
constructor(services: ArchivistServices) {
super();
this.services = services;
}
readonly outputs = ['lookup-author', 'find-reviews', 'describe-book', 'recommend-similar', 'recall-memories', 'on-topic', 'recommend-top-rated', 'off-topic', 'retry', 'salvage'] as const;
override get outputSchema(): Record<'lookup-author' | 'find-reviews' | 'describe-book' | 'recommend-similar' | 'recall-memories' | 'on-topic' | 'recommend-top-rated' | 'off-topic' | 'retry' | 'salvage', SchemaObjectType> {
return {
'lookup-author': { 'type': 'object' },
'find-reviews': { 'type': 'object' },
'describe-book': { 'type': 'object' },
'recommend-similar': { 'type': 'object' },
'recall-memories': { 'type': 'object' },
'on-topic': { 'type': 'object' },
'recommend-top-rated': { 'type': 'object' },
'off-topic': { 'type': 'object' },
'retry': { 'type': 'object' },
'salvage': { 'type': 'object' },
};
}
override async execute(batch: Batch<ArchivistState>, context: NodeContextType) {
const buckets = new Map<IntentOutput, ItemType<ArchivistState>[]>();
for (const output of this.outputs) buckets.set(output, []);
for (const item of batch) {
const { state } = item;
const summary = state.recalledContext.summary.length > 0
? state.recalledContext.summary
: undefined;
const conversation = state.conversation.length > 0 ? state.conversation : undefined;
const signal = Signal.compose({
'deadlineMs': this.services.nodeTimeouts[context.nodeName] ?? NODE_TIMEOUT_MS,
'signal': context.signal,
});
try {
const intent = await this.services.llm.classifyIntent(state.query, summary, conversation, signal);
// Guard: empty or unrecognised intent is a classification failure; the
// retry/salvage flow decides the path.
if (intent.length === 0) {
if (state.withinRetryBudget(context.nodeName, RETRY_BUDGET)) {
const result = NodeOutput.create('retry');
for (const error of result.errors) state.collectError(error);
buckets.get(result.output)?.push(item);
} else {
state.clearAttempts(context.nodeName);
const result = NodeOutput.create('salvage');
for (const error of result.errors) state.collectError(error);
buckets.get(result.output)?.push(item);
}
continue;
}
state.intent = intent;
state.reasoning = [...state.reasoning, ReasoningStep.create({ 'kind': 'thought', 'text': `classified intent as '${intent}'` })];
state.clearAttempts(context.nodeName);
// Map every ClassifiedIntent variant to its node output port.
// 'search', 'describe' are general on-topic intents that route through
// the main pipeline (extract-query -> decide-tools -> ...). 'recommend' is
// a vague "good book / good story" ask: it routes through the dedicated
// rating-ranked branch instead of the LLM-relevance-ranked one.
const intentDispatch: Record<ClassifiedIntent, IntentOutput> = {
'off-topic': 'off-topic',
'lookup-author': 'lookup-author',
'find-reviews': 'find-reviews',
'describe-book': 'describe-book',
'recommend-similar': 'recommend-similar',
'recall-memories': 'recall-memories',
'search': 'on-topic',
'describe': 'on-topic',
'recommend': 'recommend-top-rated',
};
const result = NodeOutput.create(intentDispatch[intent]);
for (const error of result.errors) state.collectError(error);
buckets.get(result.output)?.push(item);
} catch (err) {
// External cancellation / run deadline propagates unchanged.
if (context.signal.aborted) throw err;
// Node-local timeout or LLM failure -> retry budget decides the flow. The
// classifier never fabricates an intent it didn't receive.
if (state.withinRetryBudget(context.nodeName, RETRY_BUDGET)) {
const result = NodeOutput.create('retry');
for (const error of result.errors) state.collectError(error);
buckets.get(result.output)?.push(item);
} else {
state.clearAttempts(context.nodeName);
const result = NodeOutput.create('salvage');
for (const error of result.errors) state.collectError(error);
buckets.get(result.output)?.push(item);
}
}
}
const routes: Array<readonly [IntentOutput, Batch<ArchivistState>]> = [];
for (const output of this.outputs) {
const items = buckets.get(output) ?? [];
if (items.length > 0) routes.push([output, Batch.from(items)]);
}
return RoutedBatch.create(routes);
}
}
// #endregion node-classPre-phase setup node
ts
import { MonadicNode, RoutedBatch } from '@studnicky/dagonizer';
import type { Batch, NodeContextType, SchemaObjectType } from '@studnicky/dagonizer';
import type { ArchivistState } from '../ArchivistState.ts';
export class PreRunSetupNode extends MonadicNode<ArchivistState, 'ready'> {
readonly name = 'pre-run-setup';
readonly '@id' = 'urn:noocodec:node:pre-run-setup';
readonly outputs = ['ready'] as const;
override get outputSchema(): Record<'ready', SchemaObjectType> {
return {
'ready': { 'type': 'object' },
};
}
override async execute(batch: Batch<ArchivistState>, _context: NodeContextType) {
for (const { state } of batch) {
// Stamp a per-run identifier that downstream memory-write nodes key their
// named graph on. Format: ISO timestamp with milliseconds, URL-safe.
// crypto.randomUUID() would be stronger but wall-clock is deterministic
// across replays (same input → same id), which matters for snapshot tests.
const runId = new Date().toISOString().replace(/[:.]/g, '-');
state.runId = runId;
// Clear any draft from a prior interrupted execution so a resumed run
// does not accidentally serve stale content.
state.draft = '';
state.approvalState = 'pending';
}
return RoutedBatch.create('ready', batch);
}
}Services
ts
export interface ArchivistServices {
readonly webSearch: WebSearchTool;
readonly googleBooks: GoogleBooksToolContract;
readonly wikipediaSummary: WikipediaSummaryToolContract;
readonly subjectSearch: SubjectSearchToolContract;
readonly llm: LlmClientInterface;
/**
* RDF triple store (n3.js in-memory). Per-run scratchpad: memory
* nodes write findings; gate nodes ASK the store; the live UI panel
* mirrors the triples so the visitor can watch the graph grow.
*/
readonly memory: MemoryStore;
/**
* Optional embedder service for cosine-similarity recall and hybrid
* ranking. Resolved at runtime via `EmbedderCascade.select()`. Set to
* `null` when no embedder is reachable (browser without Ollama, no
* API keys, etc.); every consumer is required to handle this
* gracefully and use deterministic Jaccard / heuristics.
* Explicit-null sentinel (not optional) keeps V8 hidden-class stability.
*/
readonly embedder: EmbedderInterface | null;
/**
* Per-placement deadline overrides in milliseconds, keyed by node/placement
* name (`context.nodeName`). Empty map ⇒ every node uses its built-in
* default. The live demo wires this from the TimeoutPane sliders.
*/
readonly nodeTimeouts: Readonly<Record<string, number>>;
}Memory + ontology
ts
/**
* MemoryStore: browser-runnable RDF quad store for the Archivist.
*
* Wraps `n3.Store` (pure JS, ~30KB gzipped, identical surface on Node
* and in the browser) and exposes a named-graph-aware surface:
*
* assert(s, p, o, graph?) write one quad
* ask({ s?, p?, o?, graph? }) boolean existence check
* select({ s?, p?, o?, graph? }) list bound rows (vars start with ?)
* triplesIn(graph) iterate quads in one graph
* triples() iterate every quad
*
* Four named graphs are reserved by convention:
*
* urn:dagonizer:ontology TBox schema (classes, properties, domains, ranges);
* loaded once on mount via loadOntology()
* urn:dagonizer:memory persistent cross-run facts
* (books, sources, scores; survives reloads)
* urn:dagonizer:state:<runId> per-run typed-state mirror
* (ArchivistState fields → triples on every node end)
* urn:dagonizer:prov:<runId> PROV-O activity log
* (which node did what when, attributed to which agent)
*
* Pattern surface intentionally mirrors SPARQL's basic graph pattern
* (`{ ?s <pred> ?o }`) without a full SPARQL engine. For richer query
* shapes (UNION, FILTER, paths) swap in `@comunica/query-sparql`.
*/
import { DataFactory, Parser, Store, Writer } from 'n3';
import type { Literal, NamedNode, Quad, Quad_Graph, Quad_Object, Quad_Predicate, Quad_Subject, Term } from 'n3';
import type { SnapshottableInterface, StoreSnapshotEntryType, StoreSnapshotType } from '@studnicky/dagonizer/contracts';
const { namedNode, literal, quad, defaultGraph } = DataFactory;
/** Stable identifier + version for `MemoryStore` snapshots; resume refuses anything else. */
const MEMORY_SNAPSHOT_TYPE = 'archivist-memory-v1';
const MEMORY_SNAPSHOT_VERSION = 1;
/** Single snapshot entry key: the whole quad store serialised as N-Quads. */
const MEMORY_SNAPSHOT_KEY = 'nquads';
export const DAG_NS = 'https://noocodec.dev/ontology/dagonizer/';
export const BOOK_NS = 'urn:dagonizer:book:';
export const RUN_NS = 'urn:dagonizer:run:';
/** Named-graph IRIs reserved by the Archivist demo. */
export const GRAPH_ONTOLOGY = namedNode('urn:dagonizer:ontology');
export const GRAPH_MEMORY = namedNode('urn:dagonizer:memory');
export const STATE_GRAPH_PREFIX = 'urn:dagonizer:state:';
export const PROV_GRAPH_PREFIX = 'urn:dagonizer:prov:';
/**
* One bound row from `select()`. Keys are pattern variable names without
* the leading `?`. Values are the raw n3 terms (NamedNode | Literal | …).
*/
export type Binding = Readonly<Record<string, Term>>;
interface SlotPattern {
readonly subject?: Term | string;
readonly predicate?: Term | string;
readonly object?: Term | string;
readonly graph?: Term | string;
}
export class MemoryStore implements SnapshottableInterface {
readonly #store = new Store();
/** Total quad count; useful for the live UI counter. */
get size(): number { return this.#store.size; }
/** Pre-bake a named-node IRI for the `dag:` vocabulary. */
static dagIri(local: string): NamedNode { return namedNode(`${DAG_NS}${local}`); }
/** Pre-bake a named-node IRI for a candidate book by ISBN. */
static bookIri(isbn: string): NamedNode { return namedNode(`${BOOK_NS}${isbn}`); }
/** Per-run subject IRI. */
static runIri(id: string): NamedNode { return namedNode(`${RUN_NS}${id}`); }
/** Make any IRI. */
static iri(value: string): NamedNode { return namedNode(value); }
/** Named-graph IRI for the per-run typed-state mirror graph. */
static stateGraphIri(runId: string): NamedNode { return namedNode(`${STATE_GRAPH_PREFIX}${runId}`); }
/** Named-graph IRI for the per-run PROV-O activity log. */
static provGraphIri(runId: string): NamedNode { return namedNode(`${PROV_GRAPH_PREFIX}${runId}`); }
/** Literal helpers: typed XSD where it matters for SPARQL FILTER. */
static lit = {
str(value: string): Literal { return literal(value); },
num(value: number): Literal { return literal(String(value), namedNode('http://www.w3.org/2001/XMLSchema#double')); },
int(value: number): Literal { return literal(String(value), namedNode('http://www.w3.org/2001/XMLSchema#integer')); },
bool(value: boolean): Literal { return literal(String(value), namedNode('http://www.w3.org/2001/XMLSchema#boolean')); },
dateTime(value: Date): Literal { return literal(value.toISOString(), namedNode('http://www.w3.org/2001/XMLSchema#dateTime')); },
};
/**
* Load the TBox ontology into `urn:dagonizer:ontology`.
*
* Accepts the `ONTOLOGY_NTRIPLES` array from `ArchivistOntology.ts`.
* Idempotent: clears the graph before writing so repeated calls on
* mount are safe. The `typeof` guard lets tests supply any string[].
*/
loadOntology(ntriples: readonly string[]): void {
this.#store.removeQuads(this.#store.getQuads(null, null, null, GRAPH_ONTOLOGY));
const parser = new Parser({ 'format': 'N-Triples' });
const joined = ntriples.join('\n');
const parsed = parser.parse(joined);
for (const q of parsed) {
this.#store.addQuad(
quad(q.subject, q.predicate, q.object, GRAPH_ONTOLOGY),
);
}
}
/** Write one quad. `graph` defaults to the default graph. */
assert(s: Quad_Subject, p: Quad_Predicate, o: Quad_Object, graph?: Quad_Graph): void {
this.#store.addQuad(quad(s, p, o, graph ?? defaultGraph()));
}
/** Write many quads. Each quad carries its own graph. */
assertAll(quads: readonly Quad[]): void {
for (const q of quads) this.#store.addQuad(q);
}
/** ASK: true when at least one quad matches the pattern. */
ask(pattern: SlotPattern): boolean {
return this.#store.getQuads(
MemoryStore.asTerm(pattern.subject) ?? null,
MemoryStore.asTerm(pattern.predicate) ?? null,
MemoryStore.asTerm(pattern.object) ?? null,
MemoryStore.asTerm(pattern.graph) ?? null,
).length > 0;
}
/**
* SELECT: list bound rows. Variables: pass a string `?name` in any
* slot and it becomes a binding key; concrete terms filter.
*/
select(pattern: SlotPattern): Binding[] {
const subject = MemoryStore.asTerm(pattern.subject) ?? null;
const predicate = MemoryStore.asTerm(pattern.predicate) ?? null;
const object = MemoryStore.asTerm(pattern.object) ?? null;
const graph = MemoryStore.asTerm(pattern.graph) ?? null;
const quads = this.#store.getQuads(subject, predicate, object, graph);
return quads.map((q) => {
const row: Record<string, Term> = {};
if (MemoryStore.isVar(pattern.subject)) row[MemoryStore.stripQuestion(pattern.subject)] = q.subject;
if (MemoryStore.isVar(pattern.predicate)) row[MemoryStore.stripQuestion(pattern.predicate)] = q.predicate;
if (MemoryStore.isVar(pattern.object)) row[MemoryStore.stripQuestion(pattern.object)] = q.object;
if (MemoryStore.isVar(pattern.graph)) row[MemoryStore.stripQuestion(pattern.graph)] = q.graph;
return row;
});
}
/** Count matching quads. */
count(pattern: SlotPattern): number {
return this.#store.getQuads(
MemoryStore.asTerm(pattern.subject) ?? null,
MemoryStore.asTerm(pattern.predicate) ?? null,
MemoryStore.asTerm(pattern.object) ?? null,
MemoryStore.asTerm(pattern.graph) ?? null,
).length;
}
/** Empty the entire store. */
clear(): void {
this.#store.removeQuads(this.#store.getQuads(null, null, null, null));
}
/** Drop every quad in one named graph (useful when a run resets). */
clearGraph(graph: Term): void {
this.#store.removeQuads(this.#store.getQuads(null, null, null, graph));
}
/**
* Drop every quad in `urn:dagonizer:memory` whose subject is typed as
* `dag:Book` (i.e. has a `rdf:type dag:Book` triple). Safe to call
* before re-seeding so the library stays idempotent across reloads.
*/
clearBooks(): void {
const rdfType = namedNode('http://www.w3.org/1999/02/22-rdf-syntax-ns#type');
const dagBook = namedNode(`${DAG_NS}Book`);
// Collect all book subject IRIs in GRAPH_MEMORY.
const bookSubjects = this.#store
.getQuads(null, rdfType, dagBook, GRAPH_MEMORY)
.map((q) => q.subject.value);
// Remove every quad whose subject is one of those book IRIs.
for (const subjectValue of bookSubjects) {
const subject = namedNode(subjectValue);
this.#store.removeQuads(this.#store.getQuads(subject, null, null, GRAPH_MEMORY));
}
}
/**
* Capture the entire quad store (all named graphs) as a `StoreSnapshotType`.
*
* Satisfies the `SnapshottableInterface` contract so the store can ride along in
* `Checkpoint.capture(dag, result, { stores: { memory } })`. The whole
* store serialises to one N-Quads string entry; N-Quads carries the
* graph term per quad, so ontology / memory / per-run graphs all round-trip.
*/
// #region snapshottable-impl
async snapshot(): Promise<StoreSnapshotType> {
const nquads = await this.#serializeNquads();
return {
'version': MEMORY_SNAPSHOT_VERSION,
'type': MEMORY_SNAPSHOT_TYPE,
'entries': [{ 'key': MEMORY_SNAPSHOT_KEY, 'value': nquads }],
};
}
/**
* Repopulate from a `StoreSnapshotType` produced by `snapshot()`. Clears the
* current store first so restore is a full replace, not a merge. Refuses a
* snapshot whose `type` / `version` doesn't match this store's format.
*/
async restore(snapshot: StoreSnapshotType): Promise<void> {
if (snapshot.type !== MEMORY_SNAPSHOT_TYPE) {
throw new Error(`MemoryStore.restore: incompatible snapshot type '${snapshot.type}' (expected '${MEMORY_SNAPSHOT_TYPE}')`);
}
if (snapshot.version !== MEMORY_SNAPSHOT_VERSION) {
throw new Error(`MemoryStore.restore: incompatible snapshot version ${String(snapshot.version)} (expected ${String(MEMORY_SNAPSHOT_VERSION)})`);
}
const entry = snapshot.entries.find((e) => e.key === MEMORY_SNAPSHOT_KEY);
const nquads = typeof entry?.value === 'string' ? entry.value : '';
this.#store.removeQuads(this.#store.getQuads(null, null, null, null));
if (nquads.length > 0) {
const parser = new Parser({ 'format': 'N-Quads' });
for (const q of parser.parse(nquads)) this.#store.addQuad(q);
}
}
/**
* Stream the entire quad store as a sequence of `StoreSnapshotEntryType` values.
* Emits exactly one entry containing the full N-Quads serialization.
*/
async *snapshotStream(): AsyncIterable<StoreSnapshotEntryType> {
const nquads = await this.#serializeNquads();
yield { 'key': MEMORY_SNAPSHOT_KEY, 'value': nquads };
}
/**
* Restore state from a stream of `StoreSnapshotEntryType` values.
* Clears the current store and repopulates from the N-Quads entry.
*/
async restoreStream(entries: AsyncIterable<StoreSnapshotEntryType>): Promise<void> {
this.#store.removeQuads(this.#store.getQuads(null, null, null, null));
for await (const entry of entries) {
if (entry.key === MEMORY_SNAPSHOT_KEY && typeof entry.value === 'string' && entry.value.length > 0) {
const parser = new Parser({ 'format': 'N-Quads' });
for (const q of parser.parse(entry.value)) this.#store.addQuad(q);
}
}
}
// #endregion snapshottable-impl
/** Serialise every quad in every graph to an N-Quads string. Promisified `Writer.end`. */
#serializeNquads(): Promise<string> {
return new Promise<string>((resolve, reject) => {
const writer = new Writer({ 'format': 'N-Quads' });
writer.addQuads(this.#store.getQuads(null, null, null, null));
writer.end((err, result) => {
if (err === null || err === undefined) resolve(result);
else reject(err instanceof Error ? err : new Error(String(err)));
});
});
}
/** Iterate every quad in every graph. */
*triples(): IterableIterator<Quad> {
for (const q of this.#store.getQuads(null, null, null, null)) yield q;
}
/** Iterate every quad in a single named graph. */
*triplesIn(graph: Term): IterableIterator<Quad> {
for (const q of this.#store.getQuads(null, null, null, graph)) yield q;
}
/** Distinct graph IRIs the store currently knows about. */
graphs(): readonly Term[] {
const seen = new Map<string, Term>();
for (const q of this.#store.getQuads(null, null, null, null)) {
if (q.graph.termType === 'DefaultGraph') continue;
if (!seen.has(q.graph.value)) seen.set(q.graph.value, q.graph);
}
return [...seen.values()];
}
private static isVar(slot: Term | string | undefined): slot is string {
return typeof slot === 'string' && slot.startsWith('?');
}
private static stripQuestion(name: string): string {
return name.startsWith('?') ? name.slice(1) : name;
}
private static asTerm(slot: Term | string | undefined): Term | null {
if (slot === undefined) return null;
if (MemoryStore.isVar(slot)) return null;
if (typeof slot === 'string') return null;
return slot;
}
}Ontology (TBox + ABox)
ts
/**
* ArchivistOntology: TBox (schema) for the Archivist's RDF memory.
*
* Defines the class and property vocabulary under the `dag:` namespace
* (`https://noocodec.dev/ontology/dagonizer/`). Every ABox write in
* `recordFindings.ts` and `StateProjection.ts` uses these same IRIs so
* SPARQL queries span the TBox (`urn:dagonizer:ontology`) and ABox
* (`urn:dagonizer:memory`, `urn:dagonizer:state:<runId>`) uniformly.
*
* Exported surfaces:
* - `ArchivistOntologyJsonLd`: canonical JSON-LD document (docs / tooling)
* - `ONTOLOGY_NTRIPLES`: N-Triples ready to load via `MemoryStore.loadOntology()`
*
* Classes (7):
* dag:Book, dag:Author, dag:Subject, dag:Run, dag:Activity,
* dag:Source, dag:Score
*
* Object properties (7):
* dag:hasAuthor, dag:hasSubject, dag:fromSource, dag:queriedIn,
* dag:shortlisted, dag:about, dag:publishedBy
*
* Datatype properties (9):
* dag:title, dag:isbn, dag:summary, dag:firstPublishYear,
* dag:rating, dag:score, dag:visitorQuery, dag:runTimestamp, dag:inShortlist
*
* Cross-source query surface (with TBox + ABox co-loaded):
* • JOIN on dag:title across catalog, web-search, wiki records (same predicate)
* • Enumerate all books from a Run: ?run dag:candidate ?book
* • Rank by score across sources: ?book dag:score ?s ORDER BY DESC(?s)
* • Trace lineage: ?run dag:queriedIn / dag:fromSource ?src
* • Schema reflection: ask what class/domain/range a predicate has
*/
/** @internal Namespace abbreviation. */
const DAG = 'https://noocodec.dev/ontology/dagonizer/';
const RDFS = 'http://www.w3.org/2000/01/rdf-schema#';
const OWL = 'http://www.w3.org/2002/07/owl#';
const XSD = 'http://www.w3.org/2001/XMLSchema#';
const PROV = 'http://www.w3.org/ns/prov#';
// ── JSON-LD context ─────────────────────────────────────────────────────────
const CONTEXT = {
'@vocab': DAG,
'dag': DAG,
'rdfs': RDFS,
'owl': OWL,
'xsd': XSD,
'prov': PROV,
'subClassOf': { '@id': `${RDFS}subClassOf`, '@type': '@id' },
'domain': { '@id': `${RDFS}domain`, '@type': '@id' },
'range': { '@id': `${RDFS}range`, '@type': '@id' },
'label': { '@id': `${RDFS}label`, '@language': 'en' },
'comment': { '@id': `${RDFS}comment`, '@language': 'en' },
'type': '@type',
'Class': `${OWL}Class`,
'ObjectProperty': `${OWL}ObjectProperty`,
'DatatypeProperty': `${OWL}DatatypeProperty`,
'Ontology': `${OWL}Ontology`,
};
// ── JSON-LD document ────────────────────────────────────────────────────────
/** Canonical JSON-LD ontology document. Use for tooling, docs, and exports. */
export const ArchivistOntologyJsonLd: Record<string, unknown> = {
'@context': CONTEXT,
'@graph': [
// Ontology header
{
'@id': `${DAG}`,
'type': 'Ontology',
'label': 'Dagonizer Archivist Ontology',
'comment': 'TBox vocabulary for the Archivist demo RDF memory store',
},
// ── Classes ────────────────────────────────────────────────────────────
{
'@id': `${DAG}Book`,
'type': 'Class',
'label': 'Book',
'comment': 'A bibliographic record: catalog entry, web-search result, or wiki article.',
},
{
'@id': `${DAG}Author`,
'type': 'Class',
'label': 'Author',
'comment': 'A person or organisation responsible for a Book.',
},
{
'@id': `${DAG}Subject`,
'type': 'Class',
'label': 'Subject',
'comment': 'A thematic topic or classification applied to a Book.',
},
{
'@id': `${DAG}Run`,
'type': 'Class',
'label': 'Run',
'comment': 'One top-level Archivist execution, keyed by runId.',
'subClassOf': `${PROV}Activity`,
},
{
'@id': `${DAG}Activity`,
'type': 'Class',
'label': 'Activity',
'comment': 'An Archivist-domain prov:Activity (node execution, tool call, LLM call).',
'subClassOf': `${PROV}Activity`,
},
{
'@id': `${DAG}Source`,
'type': 'Class',
'label': 'Source',
'comment': 'A data source from which Book records are fetched (catalog, web, wiki, reviews).',
},
{
'@id': `${DAG}Score`,
'type': 'Class',
'label': 'Score',
'comment': 'A ranked relevance score in [0, 1] assigned to a Book by the ranking node.',
},
// ── Object properties ──────────────────────────────────────────────────
{
'@id': `${DAG}hasAuthor`,
'type': 'ObjectProperty',
'label': 'hasAuthor',
'comment': 'Relates a Book to an Author.',
'domain': `${DAG}Book`,
'range': `${DAG}Author`,
},
{
'@id': `${DAG}hasSubject`,
'type': 'ObjectProperty',
'label': 'hasSubject',
'comment': 'Relates a Book to a Subject.',
'domain': `${DAG}Book`,
'range': `${DAG}Subject`,
},
{
'@id': `${DAG}fromSource`,
'type': 'ObjectProperty',
'label': 'fromSource',
'comment': 'Relates a Book record to the Source it was retrieved from.',
'domain': `${DAG}Book`,
'range': `${DAG}Source`,
},
{
'@id': `${DAG}queriedIn`,
'type': 'ObjectProperty',
'label': 'queriedIn',
'comment': 'Relates a Source to the Run it was consulted in.',
'domain': `${DAG}Source`,
'range': `${DAG}Run`,
},
{
'@id': `${DAG}shortlisted`,
'type': 'ObjectProperty',
'label': 'shortlisted',
'comment': 'Relates a Run to a Book that was placed on the shortlist.',
'domain': `${DAG}Run`,
'range': `${DAG}Book`,
},
{
'@id': `${DAG}about`,
'type': 'ObjectProperty',
'label': 'about',
'comment': 'Relates a Book to a Subject it is about.',
'domain': `${DAG}Book`,
'range': `${DAG}Subject`,
},
{
'@id': `${DAG}publishedBy`,
'type': 'ObjectProperty',
'label': 'publishedBy',
'comment': 'Relates a Book to its publisher (as a named node or literal).',
'domain': `${DAG}Book`,
},
// ── Datatype properties ────────────────────────────────────────────────
{
'@id': `${DAG}title`,
'type': 'DatatypeProperty',
'label': 'title',
'comment': 'Human-readable title of a Book.',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}isbn`,
'type': 'DatatypeProperty',
'label': 'isbn',
'comment': 'ISBN-13, ISBN-10, or opaque source key identifying a Book.',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}summary`,
'type': 'DatatypeProperty',
'label': 'summary',
'comment': 'Editorial description or summary of a Book.',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}firstPublishYear`,
'type': 'DatatypeProperty',
'label': 'firstPublishYear',
'comment': 'Year the Book was first published.',
'domain': `${DAG}Book`,
'range': `${XSD}integer`,
},
{
'@id': `${DAG}rating`,
'type': 'DatatypeProperty',
'label': 'rating',
'comment': 'Reader rating of a Book in [0, 5].',
'domain': `${DAG}Book`,
'range': `${XSD}double`,
},
{
'@id': `${DAG}score`,
'type': 'DatatypeProperty',
'label': 'score',
'comment': 'Relevance score in [0, 1] assigned to a Book for a given query.',
'domain': `${DAG}Book`,
'range': `${XSD}double`,
},
{
'@id': `${DAG}visitorQuery`,
'type': 'DatatypeProperty',
'label': 'visitorQuery',
'comment': 'Raw question string submitted by the visitor in a Run.',
'domain': `${DAG}Run`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}runTimestamp`,
'type': 'DatatypeProperty',
'label': 'runTimestamp',
'comment': 'Unix timestamp (ms) when the Run was recorded.',
'domain': `${DAG}Run`,
'range': `${XSD}double`,
},
{
'@id': `${DAG}inShortlist`,
'type': 'DatatypeProperty',
'label': 'inShortlist',
'comment': 'True when a Book was selected onto the shortlist for the current Run.',
'domain': `${DAG}Book`,
'range': `${XSD}boolean`,
},
{
'@id': `${DAG}source`,
'type': 'DatatypeProperty',
'label': 'source',
'comment': 'String identifier of the source a Book record was retrieved from (e.g. "web-search").',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}author`,
'type': 'DatatypeProperty',
'label': 'author',
'comment': 'String name of an author of a Book (literal form of hasAuthor).',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}subject`,
'type': 'DatatypeProperty',
'label': 'subject',
'comment': 'String label of a subject/topic of a Book (literal form of hasSubject).',
'domain': `${DAG}Book`,
'range': `${XSD}string`,
},
{
'@id': `${DAG}candidate`,
'type': 'ObjectProperty',
'label': 'candidate',
'comment': 'Relates a Run to a Book that was a candidate in that run.',
'domain': `${DAG}Run`,
'range': `${DAG}Book`,
},
{
'@id': `${DAG}shortlistedTitle`,
'type': 'DatatypeProperty',
'label': 'shortlistedTitle',
'comment': 'Title string of a Book shortlisted in a Run (literal convenience predicate).',
'domain': `${DAG}Run`,
'range': `${XSD}string`,
},
],
};
// ── N-Triples serialisation ─────────────────────────────────────────────────
//
// Pre-baked so `MemoryStore.loadOntology()` can parse them without a
// JSON-LD library. Generated once from the JSON-LD graph above; kept
// in sync manually (or via build tooling) since the ontology is stable.
/** Turtle/N-Triples serialization helpers. */
class Turtle {
private static iri(s: string): string { return `<${s}>`; }
private static lit(s: string): string { return `"${s.replace(/\\/g, '\\\\').replace(/"/g, '\\"').replace(/\n/g, '\\n')}"@en`; }
static triple(s: string, p: string, o: string): string {
return `${Turtle.iri(s)} ${Turtle.iri(p)} ${Turtle.iri(o)} .`;
}
static tripleL(s: string, p: string, o: string): string {
return `${Turtle.iri(s)} ${Turtle.iri(p)} ${Turtle.lit(o)} .`;
}
}
const RDF_TYPE = 'http://www.w3.org/1999/02/22-rdf-syntax-ns#type';
const OWL_CLASS = `${OWL}Class`;
const OWL_OP = `${OWL}ObjectProperty`;
const OWL_DP = `${OWL}DatatypeProperty`;
const RDFS_SUB = `${RDFS}subClassOf`;
const RDFS_DOMAIN = `${RDFS}domain`;
const RDFS_RANGE = `${RDFS}range`;
const RDFS_LABEL = `${RDFS}label`;
const RDFS_COMMENT = `${RDFS}comment`;
/** N-Triple strings ready to load into the ontology named graph. */
export const ONTOLOGY_NTRIPLES: readonly string[] = [
// ── Classes
Turtle.triple(`${DAG}Book`, RDF_TYPE, OWL_CLASS),
Turtle.tripleL(`${DAG}Book`, RDFS_LABEL, 'Book'),
Turtle.tripleL(`${DAG}Book`, RDFS_COMMENT, 'A bibliographic record: catalog entry, web-search result, or wiki article.'),
Turtle.triple(`${DAG}Author`, RDF_TYPE, OWL_CLASS),
Turtle.tripleL(`${DAG}Author`, RDFS_LABEL, 'Author'),
Turtle.tripleL(`${DAG}Author`, RDFS_COMMENT, 'A person or organisation responsible for a Book.'),
Turtle.triple(`${DAG}Subject`, RDF_TYPE, OWL_CLASS),
Turtle.tripleL(`${DAG}Subject`, RDFS_LABEL, 'Subject'),
Turtle.tripleL(`${DAG}Subject`, RDFS_COMMENT, 'A thematic topic or classification applied to a Book.'),
Turtle.triple(`${DAG}Run`, RDF_TYPE, OWL_CLASS),
Turtle.triple(`${DAG}Run`, RDFS_SUB, `${PROV}Activity`),
Turtle.tripleL(`${DAG}Run`, RDFS_LABEL, 'Run'),
Turtle.tripleL(`${DAG}Run`, RDFS_COMMENT, 'One top-level Archivist execution, keyed by runId.'),
Turtle.triple(`${DAG}Activity`, RDF_TYPE, OWL_CLASS),
Turtle.triple(`${DAG}Activity`, RDFS_SUB, `${PROV}Activity`),
Turtle.tripleL(`${DAG}Activity`, RDFS_LABEL, 'Activity'),
Turtle.tripleL(`${DAG}Activity`, RDFS_COMMENT, 'An Archivist-domain prov:Activity (node execution, tool call, LLM call).'),
Turtle.triple(`${DAG}Source`, RDF_TYPE, OWL_CLASS),
Turtle.tripleL(`${DAG}Source`, RDFS_LABEL, 'Source'),
Turtle.tripleL(`${DAG}Source`, RDFS_COMMENT, 'A data source from which Book records are fetched (catalog, web, wiki, reviews).'),
Turtle.triple(`${DAG}Score`, RDF_TYPE, OWL_CLASS),
Turtle.tripleL(`${DAG}Score`, RDFS_LABEL, 'Score'),
Turtle.tripleL(`${DAG}Score`, RDFS_COMMENT, 'A ranked relevance score in [0, 1] assigned to a Book by the ranking node.'),
// ── Object properties
Turtle.triple(`${DAG}hasAuthor`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}hasAuthor`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}hasAuthor`, RDFS_RANGE, `${DAG}Author`),
Turtle.tripleL(`${DAG}hasAuthor`, RDFS_LABEL, 'hasAuthor'),
Turtle.triple(`${DAG}hasSubject`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}hasSubject`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}hasSubject`, RDFS_RANGE, `${DAG}Subject`),
Turtle.tripleL(`${DAG}hasSubject`, RDFS_LABEL, 'hasSubject'),
Turtle.triple(`${DAG}fromSource`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}fromSource`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}fromSource`, RDFS_RANGE, `${DAG}Source`),
Turtle.tripleL(`${DAG}fromSource`, RDFS_LABEL, 'fromSource'),
Turtle.triple(`${DAG}queriedIn`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}queriedIn`, RDFS_DOMAIN, `${DAG}Source`),
Turtle.triple(`${DAG}queriedIn`, RDFS_RANGE, `${DAG}Run`),
Turtle.tripleL(`${DAG}queriedIn`, RDFS_LABEL, 'queriedIn'),
Turtle.triple(`${DAG}shortlisted`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}shortlisted`, RDFS_DOMAIN, `${DAG}Run`),
Turtle.triple(`${DAG}shortlisted`, RDFS_RANGE, `${DAG}Book`),
Turtle.tripleL(`${DAG}shortlisted`, RDFS_LABEL, 'shortlisted'),
Turtle.triple(`${DAG}about`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}about`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}about`, RDFS_RANGE, `${DAG}Subject`),
Turtle.tripleL(`${DAG}about`, RDFS_LABEL, 'about'),
Turtle.triple(`${DAG}publishedBy`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}publishedBy`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.tripleL(`${DAG}publishedBy`, RDFS_LABEL, 'publishedBy'),
// ── Datatype properties
Turtle.triple(`${DAG}title`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}title`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}title`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}title`, RDFS_LABEL, 'title'),
Turtle.triple(`${DAG}isbn`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}isbn`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}isbn`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}isbn`, RDFS_LABEL, 'isbn'),
Turtle.triple(`${DAG}summary`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}summary`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}summary`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}summary`, RDFS_LABEL, 'summary'),
Turtle.triple(`${DAG}firstPublishYear`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}firstPublishYear`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}firstPublishYear`, RDFS_RANGE, `${XSD}integer`),
Turtle.tripleL(`${DAG}firstPublishYear`, RDFS_LABEL, 'firstPublishYear'),
Turtle.triple(`${DAG}rating`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}rating`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}rating`, RDFS_RANGE, `${XSD}double`),
Turtle.tripleL(`${DAG}rating`, RDFS_LABEL, 'rating'),
Turtle.triple(`${DAG}score`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}score`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}score`, RDFS_RANGE, `${XSD}double`),
Turtle.tripleL(`${DAG}score`, RDFS_LABEL, 'score'),
Turtle.triple(`${DAG}visitorQuery`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}visitorQuery`, RDFS_DOMAIN, `${DAG}Run`),
Turtle.triple(`${DAG}visitorQuery`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}visitorQuery`, RDFS_LABEL, 'visitorQuery'),
Turtle.triple(`${DAG}runTimestamp`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}runTimestamp`, RDFS_DOMAIN, `${DAG}Run`),
Turtle.triple(`${DAG}runTimestamp`, RDFS_RANGE, `${XSD}double`),
Turtle.tripleL(`${DAG}runTimestamp`, RDFS_LABEL, 'runTimestamp'),
Turtle.triple(`${DAG}inShortlist`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}inShortlist`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}inShortlist`, RDFS_RANGE, `${XSD}boolean`),
Turtle.tripleL(`${DAG}inShortlist`, RDFS_LABEL, 'inShortlist'),
Turtle.triple(`${DAG}source`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}source`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}source`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}source`, RDFS_LABEL, 'source'),
Turtle.triple(`${DAG}author`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}author`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}author`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}author`, RDFS_LABEL, 'author'),
Turtle.triple(`${DAG}subject`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}subject`, RDFS_DOMAIN, `${DAG}Book`),
Turtle.triple(`${DAG}subject`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}subject`, RDFS_LABEL, 'subject'),
Turtle.triple(`${DAG}candidate`, RDF_TYPE, OWL_OP),
Turtle.triple(`${DAG}candidate`, RDFS_DOMAIN, `${DAG}Run`),
Turtle.triple(`${DAG}candidate`, RDFS_RANGE, `${DAG}Book`),
Turtle.tripleL(`${DAG}candidate`, RDFS_LABEL, 'candidate'),
Turtle.triple(`${DAG}shortlistedTitle`, RDF_TYPE, OWL_DP),
Turtle.triple(`${DAG}shortlistedTitle`, RDFS_DOMAIN, `${DAG}Run`),
Turtle.triple(`${DAG}shortlistedTitle`, RDFS_RANGE, `${XSD}string`),
Turtle.tripleL(`${DAG}shortlistedTitle`, RDFS_LABEL, 'shortlistedTitle'),
];Details for Nerds
Backends
The Archivist runs against a real model in any of these environments. detectBackends() probes each and pickBestBackend() selects the highest-priority runnable backend. On mobile devices, Gemini Nano and WebLLM are excluded from auto-selection (both require desktop Chrome or a WebGPU-capable device). Cloud backends work on every device.
| Priority | Backend | What it needs |
|---|---|---|
| 1 | Groq (cloud, free tier) | Free key from console.groq.com/keys. selectChatModel() discovers the available chat catalogue. ~30 RPM on the free tier. Works on any device. |
| 2 | Cerebras (cloud, free tier) | Free key from cloud.cerebras.ai. selectChatModel() confirms the live chat model before registration. Works on any device. |
| 3 | Gemini API (Google AI Studio free tier) | Paste-into-form (browser). Free tier access, CORS open from any origin, and model selection through Gemini's ListModels response. Works on any device. |
| 4 | Mistral (cloud, free tier) | Free key from console.mistral.ai/api-keys/. selectChatModel() discovers the live catalogue. Works on any device. |
| 5 | OpenRouter (cloud, free tier) | Free key from openrouter.ai/keys. Routes through OpenRouter's model catalogue. Works on any device. |
| 6 | Browser built-in model (local, via window.LanguageModel) | Chrome 138+ or Edge. No key, no network, ~2 GB one-shot model download. Desktop only. |
| 7 | WebLLM (in-browser, WebGPU) | Browser with navigator.gpu. Lazy-loads @mlc-ai/web-llm + Phi-3.5 mini (~780 MB) on first use; cached after. Desktop only. |
When none of these is reachable, the runner renders a no-model gate (with links to the free cloud keys above) instead of running.
Cross-agent memory and live model swapping
The Archivist demonstrates two capabilities that extend beyond single-turn, single-model interaction: persistent cross-agent memory and live model swapping mid-conversation.
Persistent cross-agent memory
A single MemoryStore instance is created when the runner component initializes and lives for the entire browser session. It is not scoped to a run or a backend — it accumulates across every turn, regardless of which model composed the response. Three named graphs partition the data:
urn:dagonizer:memory— the durable cross-run graph.record-findingswrites every shortlisted book here as RDF triples:<book> dag:title / dag:source / dag:score / dag:inShortlist, and<run> dag:shortlisted <book>linking the run to each book it shortlisted.urn:dagonizer:state:<runId>— a per-run mirror ofArchivistState, written byStateProjection.project()after every node end.recall-contextqueries these graphs to surface prior intents, recently-seen candidates, and Jaccard-similar prior queries.urn:dagonizer:prov:<runId>— the per-run PROV-O activity graph written byRdfProvObserver(covered below).
recall-context executes first in the DAG, before classify-intent. It SPARQL-queries the accumulated state graphs for prior visitor queries, intents, and shortlisted books, and injects a plain-text summary into state.recalledContext. Every downstream LLM node — classification, tool selection, composition — receives the recalled context in its prompt. This means the second turn knows what the first turn found, and the third turn knows what the first two found, without the visitor having to restate prior topics.
Provenance: which agent wrote what
Each run stamps a dispatcherAgentId of the form dispatcher:<providerId> on the RdfProvObserver. The observer writes one prov:Activity per node execution into urn:dagonizer:prov:<runId> and types dispatcher:<providerId> as a prov:SoftwareAgent. Each activity is prov:wasAssociatedWith that agent (dispatcher:groq, dispatcher:anthropic, dispatcher:gemini-api, etc.), and activities chain via prov:wasInformedBy. When a visitor changes backends between turns, the accumulated provenance graphs record findings from multiple agents, each distinguishable by its IRI. The Memory tab's graph view makes this visible: provenance edges connect each run's activities to the agent that performed them.
Live model swapping
The BackendPicker component emits update:active-id events; the runner wires @update:active-id="activeBackend = $event" so the activeBackend ref updates immediately. The makeLlm() call inside ask() reads activeBackend.value at run time, so the very next run after a picker change uses the newly selected backend.
A backend swap only updates the activeBackend ref (and persists it to localStorage). It does not clear conversation, memory, or trace state — those are component-level and outlive any single run. The picker is disabled while a run is in flight (:disabled="isRunning"), so a swap always takes effect on the next turn, never mid-run.
| What a backend swap changes | What it leaves intact |
|---|---|
The active LLM client (currentLlm re-derives via makeLlm()) | conversation (full turn history) |
The persisted dagonizer-active-backend preference | memoryStore (all RDF triples, all named graphs) |
trace and logEvents | |
Checkpoint state (lastResult, checkpointNode) |
This is the core point of the demo, not an incidental feature: a visitor can start a session on Gemini Nano, switch to a cloud Groq key when they want faster responses, and continue on Anthropic — every backend reads and writes the same shared MemoryStore, each run's provenance is recorded under its own dispatcher:<providerId> agent, and recall-context feeds each backend the findings from all prior backends.
Seed library
On mount, 18 sci-fi and philosophy titles are pre-loaded into urn:dagonizer:memory so the Memory tab has content from first paint. The seed covers:
- Science fiction: Liu Cixin, William Gibson, Ursula K. Le Guin (×2), Stanisław Lem, Ted Chiang, Jeff VanderMeer, Dan Simmons, Vernor Vinge, the Strugatsky brothers.
- Philosophy and philosophical literature: Borges, Wittgenstein, Camus, Foucault, Deleuze, Hofstadter, Marcus Aurelius, Hegel.
SeedLibrary.loadInto(memoryStore) clears urn:dagonizer:memory and reasserts all 18 books as RDF triples using the same dag:title, dag:author, dag:subject, dag:firstPublishYear, dag:summary, and rdf:type dag:Book predicates that StateProjection uses for run candidates. Because the vocabulary is shared, the MemoryGraph renders seed books and run candidates uniformly.
Every backend receives the pre-seeded triples through the recall-memories node's SPARQL digest; the library is a shared starting point for every run. reset() restores the seed alongside the TBox ontology so a manual reset never leaves the Memory tab empty.
Intent classification (vector-similarity)
The CLI runner builds an EmbedderCascade alongside the LLM cascade: Ollama (loopback) → Gemini API → Mistral. The browser runner provisions one through EmbedderProvisioner.provision(), a memoized cascade over on-device browser embedders: transformers.js MiniLM (WASM, always available) → TensorFlow.js Universal Sentence Encoder → WebLLM (WebGPU). Whichever path supplies the embedder, IntentClassifier.create(embedder) precomputes label embeddings once; classifyIntent then routes by cosine similarity against the visitor's query in O(labels). Should provisioning fail (no candidate probes available, CDN import error), the provisioner returns embedder: null and the node delegates to the LLM classifier directly (same routing, slower path).
Visitor language
UserLanguage.detect() reads the device locale (navigator.language in the browser, LANG / LC_ALL env vars on the CLI), normalises it to an IETF tag, and threads it into the system prompt. The composer drafts the response in the visitor's language without an explicit toggle.
Conversational composition
Drafts ship as conversational prose. The composer prompt forbids markdown headings, bullet lists, and structured layout: the response reads like a knowledgeable shop assistant talking out loud, not a search result page. The validator rejects drafts that leak markup back into the conversation.
Mobile detection
MobileDetection.isLikelyMobile() triangulates three signals: touch points (navigator.maxTouchPoints > 1), coarse pointer media query ((pointer: coarse)), and narrow viewport (innerWidth < 900). All three must indicate mobile; a single signal is not enough. A "Treat as desktop" link in the mobile banner lets tablet visitors opt out of mobile detection and stores the override in localStorage (dagonizer-device-override).
The on-device and WebGPU backends are desktop-only, so on mobile the demo needs a cloud API key (Groq, Cerebras, Gemini API, Mistral, or OpenRouter). Until one is set, the no-model gate is shown with links to free keys; the demo does not run without a real backend. Once a key is entered the mobile banner reads "using cloud backend [name]", and adding any cloud key causes pickBestBackend to re-rank and swap the active backend automatically.
Enable the browser built-in model + tool calling
The Archivist asks the LLM to invoke tools (currently web_search_books, backed by openlibrary.org). Gemini API uses native functionDeclarations; the browser built-in model honours the same plan via the Prompt API's responseConstraint JSON-schema field, which arrived behind feature flags.
Open
chrome://flagsand enable each of:#prompt-api-for-gemini-nano→ Enabled#prompt-api-for-gemini-nano-multimodal-input→ Enabled (newer flag name in some channels)#optimization-guide-on-device-model→ EnabledBypassPerfRequirement
Restart Chrome.
Trigger the download. Visit any page that calls
LanguageModel.create()(this demo will, but you can also paste the snippet below into DevTools):jsawait LanguageModel.create();Chrome downloads ~2 GB. Status is visible at
chrome://components; look for Optimization Guide On Device Model. The widget on this page also surfacesavailability()as "downloading…" until ready.Reload this page. The backend banner should now read Browser built-in LanguageModel (on-device).
If the model is still downloadable rather than available after the steps above, leave Chrome open for a few minutes; the download runs in the background and is gated by the browser's network-condition heuristics.
Bring-your-own Gemini API key
When Gemini Nano is unavailable, the next-best option is the Google AI Studio free tier:
- Go to aistudio.google.com/apikey and click Create API key. The free tier covers 15 requests/min and 1500 requests/day on the free tier. Plenty for the demo.
- Paste the key into the Bring your own Gemini API key drawer below the backend picker. It's stored in
localStorageonly; the request itself goes straight from your browser to Google. - The runner picks
gemini-apiautomatically once a key is present.
CORS is open on the Gemini REST endpoint, so this works from GitHub Pages or any other static host without a proxy.
bash
# CLI: cascade is Ollama (localhost) → Gemini API → Cerebras → Groq; first reachable wins.
# Throws NO_ADAPTER_AVAILABLE if none is reachable.
npx tsx examples/the-archivist/runArchivist.ts
# Force Gemini REST with your key:
GEMINI_API_KEY=AIza... npx tsx examples/the-archivist/runArchivist.tsWhat the first examples cover
The first eight example pages isolate one Dagonizer feature against the Archivist domain:
| Example | Feature | Page |
|---|---|---|
| 01 | Linear intake + terminal routing | Example 01: Linear Intake |
| 02 | DAGBuilder authoring | Example 02: DAGBuilder |
| 03 | Tool schema design (JSON Schema 2020-12 inputSchema) | Example 03: Tool Schemas |
| 04 | Scatter scout with partition gather | Example 04: Scatter Scout |
| 05 | EmbeddedDAGNode composition | Example 05: Embedded DAGs |
| 06 | Abortable visitor request | Example 06: Cancellation |
| 07 | Retry as a flow shape (retry/salvage loop) | Example 07: Retry Flow |
| 08 | Checkpoint mid-draft and resume | Example 08: Checkpoint and Resume |
Every page starts from the same ArchivistState + services + node set; only the DAG variation and the registered subset change.
Related Concepts
Read these next when you want to unpack the Archivist into vocabulary, architecture, visualization, persistence, and domain schema pieces.
- Concepts - Dagonizer vocabulary the Archivist exercises
- Architecture - three-tier interface taxonomy
- Visualization - render the Archivist DAG with
MermaidRenderer.render(dag) - Persistence - wire
ckpt.persist/Checkpoint.recallto aCheckpointStore - json-tology Bookstore domain - the schema vocabulary the Archivist's
Bookentity mirrors
Archivist Feature Map
These numbered examples are owned by the Archivist domain because the live demo exposes the same principle in its runnable DAG:
| Example | Principle in the runnable Archivist |
|---|---|
| Example 22: Retry Timing and Salvage | The compose-retry-loop DAG shows retry/salvage routing; the example page isolates the timing policy that controls retry waits. |
| Example 24: LLM Adapter | Provider selection happens before compose-response / intent-classification nodes call the active adapter. |
| Example 25: Embedder | Semantic recall and intent support use the same embedder-registry surface against book and memory text. |
| Example 26: Tool Use | book-search-scatter turns tool decisions into bookWorksets; each workset selects a registered tool DAG through a dynamic DagReference. |
| Example 29: Agent DAG with JSON-LD | The Archivist is the full in-browser agent application: request classification, model/tool work, memory recall, and response composition are all DAG placements. |