Dagonizer
⦿ What problem it solves
When work has multiple steps that depend on each other — classify, then fetch, then compose, then save — you need a way to express those dependencies, track shared state as work moves through them, stop safely when something goes wrong, and pick up where you left off if the process crashes. @studnicky/dagonizer is that infrastructure. You declare each step as a typed node, place those nodes inside a JSON-LD DAG, and register the DAGs and nodes the dispatcher may run. The dispatcher follows placement IRIs, routes by typed outputs, and handles retries, cancellation, and checkpoint/resume without your nodes carrying orchestration code.
A DAG is therefore a graph of placements where each placement's output drives the routing decision for the next placement. Non-technical readers can think of it as a flowchart where each box is a typed function or registered sub-DAG, the arrows are labeled outcomes, and every box has a canonical IRI under the hood. The eye of the graph is the IRI; the display name is just the label etched on the box.
⦿ One engine, two applications
@studnicky/dagonizer is a single type-safe, resumable, abortable DAG/workflow engine. LLM-agent orchestration and data-orchestration / ETL run on the identical core — only the node domain differs. Three runnable in-browser demos prove it: The Archivist (LLM agents — a bibliographic assistant), The Dispatcher (LLM agents with a human in the loop — warm-handoff support), and The Cartographer (streaming multi-format satellite tracking feeds, geo-resolution, GDPR redaction, continent-level insights — no LLM).
⦿ What it is
A node is a typed, stateless unit of work that receives a batch of state items and a context (including an AbortSignal) and returns a routed batch — each item mapped to a named output port. Nodes receive external dependencies through their constructors. The dispatcher routes items to the next placement based on the output port. Extend MonadicNode<TState, TOutput> or implement NodeInterface<TState, TOutput> directly; per-item behavior lives inside the node's own execute(batch, context) loop. Six placement kinds cover the composition space.
| Kind | What it does |
|---|---|
single | One registered node; output name selects the next placement IRI |
scatter | Isolate one state clone per source item, run a registered node or DAG body in each clone, and emit per-item records for downstream fan-in |
gather | Join records from producer placement or entrypoint IRIs, apply a gather strategy, and route when the fan-in policy is satisfied |
embedded | Invoke a registered sub-DAG exactly once (cardinality 1) in an isolated state; optional stateMapping seeds the child and copies fields back; route on the child's terminal outcome |
terminal | Named end state for explicit completion or failure; use when a flow has more than one "done" semantics |
phase | Lifecycle-attached single-node placement: pre runs before the entrypoint, post runs after the main loop drains on every exit path |
⦿ FSM-driven lifecycle
Every execution runs through DAGLifecycleMachine: pending → running → completed | failed | cancelled | timed_out. Terminal states are sticky. Every transition is timestamped with monotonic milliseconds. The lifecycle state travels on NodeStateInterface through every node in the graph.
pending ──start──▶ running ──succeed──▶ completed
│
├──fail(error)──▶ failed
├──cancel(reason)▶ cancelled
└──timeout──────▶ timed_out⦿ No mandatory external runtime
Dagonizer runs in-process by default. No queue, scheduler, external state store, or daemon is required to get a graph moving. DAG definitions are plain JSON-LD documents: store the serialized JSON in files, databases, or configuration services, load it at runtime via DAGDocument.load(json), then register with dispatcher.registerDAG(dag). When you do need remote or worker execution, the same DAG boundary travels through the container/worker contract; no second composition model crawls out of the deep.
⦿ See it in action
Three demos, one engine — each a different role the dispatcher can run, all live in the browser with no server. Two exercise the LLM-agent surface; the third is pure deterministic ETL. Same core, same DAG topology — only the node domain differs.
The Archivist — the cataloguer. LLM agents. A bibliographic-assistant pipeline: classify intent, scatter scout nodes over source arrays, embedded search and compose sub-DAGs, retry with decorrelated-jitter backoff, checkpoint, provenance. Exercises the full LLM-agent composition surface.
The Dispatcher — the router. LLM agents with a human in the loop. A warm-handoff support pipeline: a classifier routes each message, the AI either composes a reply instantly or the flow parks and waits for a human operator, then resumes from checkpoint on their response. A deterministic "trolley switch" can force human routing on top of the LLM decision. Demonstrates HITL Park-and-Correlate and checkpoint/resume.
The Cartographer — the mapmaker. Data orchestration / ETL / streaming. Multiple source entrypoints feed a first-class intake gather, then scatter through typed event pipelines into geo-resolution, GDPR redaction, and continent-level insights. It demonstrates open intake, explicit gather barriers, worker/container roles, and plugin-shaped DAG parts. No LLM. Runs entirely in the browser.
⦿ Why "Dagonizer"
The name compresses the three ideas the project is built on.
The structure — a DAG. The engine executes a Directed Acyclic Graph: steps joined by forward-only edges, with no cycles, so the steps always admit a well-defined execution order. Engineers compose DAGs constantly — build graphs, task schedulers, spreadsheet recalculation, linker symbol resolution, and now agent tool-call chains — often without naming the structure as such. Dagonizer makes the DAG the explicit, type-safe unit of composition.
The role — an orchestrator. In H. P. Lovecraft's fiction, Dagon is the primordial deity that presides over the submerged multitudes of the Deep Ones — first evoked in the 1919 short story of the same name. The image fits an engine whose job is to marshal many small autonomous workers — LLM agents, ETL stages — through one coordinated flow. The workers are the multitude; Dagonizer is what directs them.
The shape — ports and adapters. Backends plug into Dagonizer through adapter contracts — LlmAdapterInterface, StoreInterface, ClockProviderInterface, and the rest — never through callbacks or function-passing. That is the hexagonal "ports and adapters" architecture described by Alistair Cockburn: capabilities snap together at the boundary like interchangeable parts, and the core stays closed to modification.
Read together, Dagonizer is "the orchestrator of the DAGs." Spoken aloud it also resolves to dag-on-eyes-er — a deliberate nod to the Eye of Dagon, and to a logo that is meant to be just slightly unsettling.