Example 33: Plugin-Defined DAGs

What It Is

Plugin-Defined DAGs show the unified interface for plugins and embedded flows. The Cartographer packages source-normalization child DAGs as a plugin, registers that plugin once, and embeds the exported DAG references from the ingest DAG.

There is no special plugin placement type. Plugin DAGs enter the same registry as local DAGs, and parent graphs embed them with the same EmbeddedDAGNode contract.

How It Works

defineDagonizerPlugin packages a plugin ID, nodes, DAGs, and context declarations. registerPlugin(...) scopes those registry entries by plugin identity, then exposes the plugin DAGs through the same DAG registry used by registerBundle(...). The parent DAG embeds plugin-provided DAG IRIs exactly like locally-authored child DAGs.

This is the dev-ex target from the plugin work: a reusable part ships its own nodes and DAG documents, while the host assembles the final application through canonical JSON-LD or builder output.

Diagrams, Examples, and Outputs

DAG registration and diagram

The Cartographer packages its format-normalization child DAGs as normalizeSourcesPlugin. The browser runner registers the plugin, then registers ingestSourceDAG, whose embedded placements reference the plugin-provided DAG IRIs (urn:noocodec:dag:normalize-csv, urn:noocodec:dag:normalize-json, urn:noocodec:dag:normalize-ndjson, urn:noocodec:dag:normalize-yaml).

DAG JSON-LD registered with the dispatcher

{
  "@context": {
    "@version": 1.1,
    "name": {
      "@id": "https://noocodec.dev/ontology/dag/name"
    },
    "version": {
      "@id": "https://noocodec.dev/ontology/dag/version"
    },
    "entrypoints": {
      "@id": "https://noocodec.dev/ontology/dag/entrypoints",
      "@container": "@index"
    },
    "nodes": {
      "@id": "https://noocodec.dev/ontology/dag/nodes",
      "@container": "@set"
    },
    "outputs": {
      "@id": "https://noocodec.dev/ontology/dag/outputs"
    },
    "node": {
      "@id": "https://noocodec.dev/ontology/dag/node"
    },
    "dag": {
      "@id": "https://noocodec.dev/ontology/dag/dag"
    },
    "body": {
      "@id": "https://noocodec.dev/ontology/dag/body"
    },
    "source": {
      "@id": "https://noocodec.dev/ontology/dag/source"
    },
    "sources": {
      "@id": "https://noocodec.dev/ontology/dag/sources",
      "@container": "@index"
    },
    "itemKey": {
      "@id": "https://noocodec.dev/ontology/dag/itemKey"
    },
    "execution": {
      "@id": "https://noocodec.dev/ontology/dag/execution"
    },
    "concurrency": {
      "@id": "https://noocodec.dev/ontology/dag/concurrency"
    },
    "throttle": {
      "@id": "https://noocodec.dev/ontology/dag/throttle"
    },
    "reservoir": {
      "@id": "https://noocodec.dev/ontology/dag/reservoir"
    },
    "gather": {
      "@id": "https://noocodec.dev/ontology/dag/gather"
    },
    "dagReference": {
      "@id": "https://noocodec.dev/ontology/dag/dagReference",
      "@type": "@id"
    },
    "DagReference": {
      "@id": "https://noocodec.dev/ontology/dag/DagReference"
    },
    "from": {
      "@id": "https://noocodec.dev/ontology/dag/from"
    },
    "path": {
      "@id": "https://noocodec.dev/ontology/dag/path"
    },
    "candidates": {
      "@id": "https://noocodec.dev/ontology/dag/candidates",
      "@container": "@set"
    },
    "candidateDag": {
      "@id": "https://noocodec.dev/ontology/dag/candidateDag",
      "@type": "@id"
    },
    "selectedDag": {
      "@id": "https://noocodec.dev/ontology/dag/selectedDag",
      "@type": "@id"
    },
    "resultField": {
      "@id": "https://noocodec.dev/ontology/dag/resultField"
    },
    "policy": {
      "@id": "https://noocodec.dev/ontology/dag/policy"
    },
    "reducer": {
      "@id": "https://noocodec.dev/ontology/dag/reducer"
    },
    "outcome": {
      "@id": "https://noocodec.dev/ontology/dag/outcome"
    },
    "phase": {
      "@id": "https://noocodec.dev/ontology/dag/phase"
    },
    "stateMapping": {
      "@id": "https://noocodec.dev/ontology/dag/stateMapping"
    },
    "container": {
      "@id": "https://noocodec.dev/ontology/dag/container"
    },
    "DAG": {
      "@id": "https://noocodec.dev/ontology/dag/DAG"
    },
    "Placement": {
      "@id": "https://noocodec.dev/ontology/dag/Placement"
    },
    "SingleNode": {
      "@id": "https://noocodec.dev/ontology/dag/SingleNode"
    },
    "ScatterNode": {
      "@id": "https://noocodec.dev/ontology/dag/ScatterNode"
    },
    "EmbeddedDAGNode": {
      "@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
    },
    "GatherNode": {
      "@id": "https://noocodec.dev/ontology/dag/GatherNode"
    },
    "TerminalNode": {
      "@id": "https://noocodec.dev/ontology/dag/TerminalNode"
    },
    "PhaseNode": {
      "@id": "https://noocodec.dev/ontology/dag/PhaseNode"
    }
  },
  "@id": "urn:noocodec:dag:ingest-source",
  "@type": "DAG",
  "name": "dag:ingest-source",
  "version": "1.0",
  "entrypoints": {
    "main": "urn:noocodec:dag:ingest-source/node/select-source"
  },
  "nodes": [
    {
      "@id": "urn:noocodec:dag:ingest-source/node/select-source",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/select-source",
      "node": "urn:noocodec:node:select-source",
      "outputs": {
        "compressed": "urn:noocodec:dag:ingest-source/node/decompress",
        "plain": "urn:noocodec:dag:ingest-source/node/route-format",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/decompress",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/decompress",
      "node": "urn:noocodec:node:decompress",
      "outputs": {
        "route-format": "urn:noocodec:dag:ingest-source/node/route-format",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/route-format",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/route-format",
      "node": "urn:noocodec:node:route-format",
      "outputs": {
        "csv": "urn:noocodec:dag:ingest-source/node/parse-csv",
        "json": "urn:noocodec:dag:ingest-source/node/parse-json",
        "ndjson": "urn:noocodec:dag:ingest-source/node/parse-ndjson",
        "yaml": "urn:noocodec:dag:ingest-source/node/parse-yaml",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/parse-csv",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/parse-csv",
      "node": "urn:noocodec:node:parse-csv",
      "outputs": {
        "normalized": "urn:noocodec:dag:ingest-source/node/normalize-csv",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/parse-json",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/parse-json",
      "node": "urn:noocodec:node:parse-json",
      "outputs": {
        "normalized": "urn:noocodec:dag:ingest-source/node/normalize-json",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/parse-ndjson",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/parse-ndjson",
      "node": "urn:noocodec:node:parse-ndjson",
      "outputs": {
        "normalized": "urn:noocodec:dag:ingest-source/node/normalize-ndjson",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/parse-yaml",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/parse-yaml",
      "node": "urn:noocodec:node:parse-yaml",
      "outputs": {
        "normalized": "urn:noocodec:dag:ingest-source/node/normalize-yaml",
        "invalid": "urn:noocodec:dag:ingest-source/node/rejected"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/normalize-csv",
      "@type": "EmbeddedDAGNode",
      "name": "dag:ingest-source/node/normalize-csv",
      "outputs": {
        "success": "urn:noocodec:dag:ingest-source/node/coerce-types",
        "error": "urn:noocodec:dag:ingest-source/node/rejected"
      },
      "dag": "urn:noocodec:dag:normalize-csv",
      "stateMapping": {
        "input": {
          "parsedRecords": "parsedRecords",
          "currentSource": "currentSource"
        },
        "output": {
          "mappedRecords": "mappedRecords"
        }
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/normalize-json",
      "@type": "EmbeddedDAGNode",
      "name": "dag:ingest-source/node/normalize-json",
      "outputs": {
        "success": "urn:noocodec:dag:ingest-source/node/coerce-types",
        "error": "urn:noocodec:dag:ingest-source/node/rejected"
      },
      "dag": "urn:noocodec:dag:normalize-json",
      "stateMapping": {
        "input": {
          "parsedRecords": "parsedRecords",
          "currentSource": "currentSource"
        },
        "output": {
          "mappedRecords": "mappedRecords"
        }
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/normalize-ndjson",
      "@type": "EmbeddedDAGNode",
      "name": "dag:ingest-source/node/normalize-ndjson",
      "outputs": {
        "success": "urn:noocodec:dag:ingest-source/node/coerce-types",
        "error": "urn:noocodec:dag:ingest-source/node/rejected"
      },
      "dag": "urn:noocodec:dag:normalize-ndjson",
      "stateMapping": {
        "input": {
          "parsedRecords": "parsedRecords",
          "currentSource": "currentSource"
        },
        "output": {
          "mappedRecords": "mappedRecords"
        }
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/normalize-yaml",
      "@type": "EmbeddedDAGNode",
      "name": "dag:ingest-source/node/normalize-yaml",
      "outputs": {
        "success": "urn:noocodec:dag:ingest-source/node/coerce-types",
        "error": "urn:noocodec:dag:ingest-source/node/rejected"
      },
      "dag": "urn:noocodec:dag:normalize-yaml",
      "stateMapping": {
        "input": {
          "parsedRecords": "parsedRecords",
          "currentSource": "currentSource"
        },
        "output": {
          "mappedRecords": "mappedRecords"
        }
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/coerce-types",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/coerce-types",
      "node": "urn:noocodec:node:coerce-types",
      "outputs": {
        "validate-event": "urn:noocodec:dag:ingest-source/node/validate-event"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/validate-event",
      "@type": "SingleNode",
      "name": "dag:ingest-source/node/validate-event",
      "node": "urn:noocodec:node:validate-event",
      "outputs": {
        "validated": "urn:noocodec:dag:ingest-source/node/ingested"
      }
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/ingested",
      "@type": "TerminalNode",
      "name": "dag:ingest-source/node/ingested",
      "outcome": "completed"
    },
    {
      "@id": "urn:noocodec:dag:ingest-source/node/rejected",
      "@type": "TerminalNode",
      "name": "dag:ingest-source/node/rejected",
      "outcome": "failed"
    }
  ]
}

Mermaid generated from the same DAG

Mermaid source

%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
  %% dag:ingest-source (v1.0)
  entry_main(["main"])
  entry_main --> urn_noocodec_dag_ingest-source/node/select-source
  urn_noocodec_dag_ingest-source/node/select-source["dag:ingest-source/node/select-source"]
  urn_noocodec_dag_ingest-source/node/select-source -->|compressed| urn_noocodec_dag_ingest-source/node/decompress
  urn_noocodec_dag_ingest-source/node/select-source -->|plain| urn_noocodec_dag_ingest-source/node/route-format
  urn_noocodec_dag_ingest-source/node/select-source -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/decompress["dag:ingest-source/node/decompress"]
  urn_noocodec_dag_ingest-source/node/decompress -->|route-format| urn_noocodec_dag_ingest-source/node/route-format
  urn_noocodec_dag_ingest-source/node/decompress -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/route-format["dag:ingest-source/node/route-format"]
  urn_noocodec_dag_ingest-source/node/route-format -->|csv| urn_noocodec_dag_ingest-source/node/parse-csv
  urn_noocodec_dag_ingest-source/node/route-format -->|json| urn_noocodec_dag_ingest-source/node/parse-json
  urn_noocodec_dag_ingest-source/node/route-format -->|ndjson| urn_noocodec_dag_ingest-source/node/parse-ndjson
  urn_noocodec_dag_ingest-source/node/route-format -->|yaml| urn_noocodec_dag_ingest-source/node/parse-yaml
  urn_noocodec_dag_ingest-source/node/route-format -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/parse-csv["dag:ingest-source/node/parse-csv"]
  urn_noocodec_dag_ingest-source/node/parse-csv -->|normalized| urn_noocodec_dag_ingest-source/node/normalize-csv
  urn_noocodec_dag_ingest-source/node/parse-csv -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/parse-json["dag:ingest-source/node/parse-json"]
  urn_noocodec_dag_ingest-source/node/parse-json -->|normalized| urn_noocodec_dag_ingest-source/node/normalize-json
  urn_noocodec_dag_ingest-source/node/parse-json -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/parse-ndjson["dag:ingest-source/node/parse-ndjson"]
  urn_noocodec_dag_ingest-source/node/parse-ndjson -->|normalized| urn_noocodec_dag_ingest-source/node/normalize-ndjson
  urn_noocodec_dag_ingest-source/node/parse-ndjson -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/parse-yaml["dag:ingest-source/node/parse-yaml"]
  urn_noocodec_dag_ingest-source/node/parse-yaml -->|normalized| urn_noocodec_dag_ingest-source/node/normalize-yaml
  urn_noocodec_dag_ingest-source/node/parse-yaml -->|invalid| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/normalize-csv[["dag:ingest-source/node/normalize-csv"]]
  urn_noocodec_dag_ingest-source/node/normalize-csv -->|success| urn_noocodec_dag_ingest-source/node/coerce-types
  urn_noocodec_dag_ingest-source/node/normalize-csv -->|error| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/normalize-json[["dag:ingest-source/node/normalize-json"]]
  urn_noocodec_dag_ingest-source/node/normalize-json -->|success| urn_noocodec_dag_ingest-source/node/coerce-types
  urn_noocodec_dag_ingest-source/node/normalize-json -->|error| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/normalize-ndjson[["dag:ingest-source/node/normalize-ndjson"]]
  urn_noocodec_dag_ingest-source/node/normalize-ndjson -->|success| urn_noocodec_dag_ingest-source/node/coerce-types
  urn_noocodec_dag_ingest-source/node/normalize-ndjson -->|error| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/normalize-yaml[["dag:ingest-source/node/normalize-yaml"]]
  urn_noocodec_dag_ingest-source/node/normalize-yaml -->|success| urn_noocodec_dag_ingest-source/node/coerce-types
  urn_noocodec_dag_ingest-source/node/normalize-yaml -->|error| urn_noocodec_dag_ingest-source/node/rejected
  urn_noocodec_dag_ingest-source/node/coerce-types["dag:ingest-source/node/coerce-types"]
  urn_noocodec_dag_ingest-source/node/coerce-types -->|validate-event| urn_noocodec_dag_ingest-source/node/validate-event
  urn_noocodec_dag_ingest-source/node/validate-event["dag:ingest-source/node/validate-event"]
  urn_noocodec_dag_ingest-source/node/validate-event -->|validated| urn_noocodec_dag_ingest-source/node/ingested
  urn_noocodec_dag_ingest-source/node/ingested((("dag:ingest-source/node/ingested")))
  urn_noocodec_dag_ingest-source/node/rejected>"dag:ingest-source/node/rejected"]

DAG JSON-LD registered with the dispatcher

{
  "@context": {
    "@version": 1.1,
    "name": {
      "@id": "https://noocodec.dev/ontology/dag/name"
    },
    "version": {
      "@id": "https://noocodec.dev/ontology/dag/version"
    },
    "entrypoints": {
      "@id": "https://noocodec.dev/ontology/dag/entrypoints",
      "@container": "@index"
    },
    "nodes": {
      "@id": "https://noocodec.dev/ontology/dag/nodes",
      "@container": "@set"
    },
    "outputs": {
      "@id": "https://noocodec.dev/ontology/dag/outputs"
    },
    "node": {
      "@id": "https://noocodec.dev/ontology/dag/node"
    },
    "dag": {
      "@id": "https://noocodec.dev/ontology/dag/dag"
    },
    "body": {
      "@id": "https://noocodec.dev/ontology/dag/body"
    },
    "source": {
      "@id": "https://noocodec.dev/ontology/dag/source"
    },
    "sources": {
      "@id": "https://noocodec.dev/ontology/dag/sources",
      "@container": "@index"
    },
    "itemKey": {
      "@id": "https://noocodec.dev/ontology/dag/itemKey"
    },
    "execution": {
      "@id": "https://noocodec.dev/ontology/dag/execution"
    },
    "concurrency": {
      "@id": "https://noocodec.dev/ontology/dag/concurrency"
    },
    "throttle": {
      "@id": "https://noocodec.dev/ontology/dag/throttle"
    },
    "reservoir": {
      "@id": "https://noocodec.dev/ontology/dag/reservoir"
    },
    "gather": {
      "@id": "https://noocodec.dev/ontology/dag/gather"
    },
    "dagReference": {
      "@id": "https://noocodec.dev/ontology/dag/dagReference",
      "@type": "@id"
    },
    "DagReference": {
      "@id": "https://noocodec.dev/ontology/dag/DagReference"
    },
    "from": {
      "@id": "https://noocodec.dev/ontology/dag/from"
    },
    "path": {
      "@id": "https://noocodec.dev/ontology/dag/path"
    },
    "candidates": {
      "@id": "https://noocodec.dev/ontology/dag/candidates",
      "@container": "@set"
    },
    "candidateDag": {
      "@id": "https://noocodec.dev/ontology/dag/candidateDag",
      "@type": "@id"
    },
    "selectedDag": {
      "@id": "https://noocodec.dev/ontology/dag/selectedDag",
      "@type": "@id"
    },
    "resultField": {
      "@id": "https://noocodec.dev/ontology/dag/resultField"
    },
    "policy": {
      "@id": "https://noocodec.dev/ontology/dag/policy"
    },
    "reducer": {
      "@id": "https://noocodec.dev/ontology/dag/reducer"
    },
    "outcome": {
      "@id": "https://noocodec.dev/ontology/dag/outcome"
    },
    "phase": {
      "@id": "https://noocodec.dev/ontology/dag/phase"
    },
    "stateMapping": {
      "@id": "https://noocodec.dev/ontology/dag/stateMapping"
    },
    "container": {
      "@id": "https://noocodec.dev/ontology/dag/container"
    },
    "DAG": {
      "@id": "https://noocodec.dev/ontology/dag/DAG"
    },
    "Placement": {
      "@id": "https://noocodec.dev/ontology/dag/Placement"
    },
    "SingleNode": {
      "@id": "https://noocodec.dev/ontology/dag/SingleNode"
    },
    "ScatterNode": {
      "@id": "https://noocodec.dev/ontology/dag/ScatterNode"
    },
    "EmbeddedDAGNode": {
      "@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
    },
    "GatherNode": {
      "@id": "https://noocodec.dev/ontology/dag/GatherNode"
    },
    "TerminalNode": {
      "@id": "https://noocodec.dev/ontology/dag/TerminalNode"
    },
    "PhaseNode": {
      "@id": "https://noocodec.dev/ontology/dag/PhaseNode"
    }
  },
  "@id": "urn:noocodec:dag:normalize-csv",
  "@type": "DAG",
  "name": "dag:normalize-csv",
  "version": "1.0",
  "entrypoints": {
    "main": "urn:noocodec:dag:normalize-csv/node/normalize-csv-map"
  },
  "nodes": [
    {
      "@id": "urn:noocodec:dag:normalize-csv/node/normalize-csv-map",
      "@type": "SingleNode",
      "name": "dag:normalize-csv/node/normalize-csv-map",
      "node": "urn:noocodec:node:normalize-csv-map",
      "outputs": {
        "normalized": "urn:noocodec:dag:normalize-csv/node/normalized"
      }
    },
    {
      "@id": "urn:noocodec:dag:normalize-csv/node/normalized",
      "@type": "TerminalNode",
      "name": "dag:normalize-csv/node/normalized",
      "outcome": "completed"
    }
  ]
}

Mermaid generated from the same DAG

Mermaid source

%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
  %% dag:normalize-csv (v1.0)
  entry_main(["main"])
  entry_main --> urn_noocodec_dag_normalize-csv/node/normalize-csv-map
  urn_noocodec_dag_normalize-csv/node/normalize-csv-map["dag:normalize-csv/node/normalize-csv-map"]
  urn_noocodec_dag_normalize-csv/node/normalize-csv-map -->|normalized| urn_noocodec_dag_normalize-csv/node/normalized
  urn_noocodec_dag_normalize-csv/node/normalized((("dag:normalize-csv/node/normalized")))

DAG JSON-LD registered with the dispatcher

{
  "@context": {
    "@version": 1.1,
    "name": {
      "@id": "https://noocodec.dev/ontology/dag/name"
    },
    "version": {
      "@id": "https://noocodec.dev/ontology/dag/version"
    },
    "entrypoints": {
      "@id": "https://noocodec.dev/ontology/dag/entrypoints",
      "@container": "@index"
    },
    "nodes": {
      "@id": "https://noocodec.dev/ontology/dag/nodes",
      "@container": "@set"
    },
    "outputs": {
      "@id": "https://noocodec.dev/ontology/dag/outputs"
    },
    "node": {
      "@id": "https://noocodec.dev/ontology/dag/node"
    },
    "dag": {
      "@id": "https://noocodec.dev/ontology/dag/dag"
    },
    "body": {
      "@id": "https://noocodec.dev/ontology/dag/body"
    },
    "source": {
      "@id": "https://noocodec.dev/ontology/dag/source"
    },
    "sources": {
      "@id": "https://noocodec.dev/ontology/dag/sources",
      "@container": "@index"
    },
    "itemKey": {
      "@id": "https://noocodec.dev/ontology/dag/itemKey"
    },
    "execution": {
      "@id": "https://noocodec.dev/ontology/dag/execution"
    },
    "concurrency": {
      "@id": "https://noocodec.dev/ontology/dag/concurrency"
    },
    "throttle": {
      "@id": "https://noocodec.dev/ontology/dag/throttle"
    },
    "reservoir": {
      "@id": "https://noocodec.dev/ontology/dag/reservoir"
    },
    "gather": {
      "@id": "https://noocodec.dev/ontology/dag/gather"
    },
    "dagReference": {
      "@id": "https://noocodec.dev/ontology/dag/dagReference",
      "@type": "@id"
    },
    "DagReference": {
      "@id": "https://noocodec.dev/ontology/dag/DagReference"
    },
    "from": {
      "@id": "https://noocodec.dev/ontology/dag/from"
    },
    "path": {
      "@id": "https://noocodec.dev/ontology/dag/path"
    },
    "candidates": {
      "@id": "https://noocodec.dev/ontology/dag/candidates",
      "@container": "@set"
    },
    "candidateDag": {
      "@id": "https://noocodec.dev/ontology/dag/candidateDag",
      "@type": "@id"
    },
    "selectedDag": {
      "@id": "https://noocodec.dev/ontology/dag/selectedDag",
      "@type": "@id"
    },
    "resultField": {
      "@id": "https://noocodec.dev/ontology/dag/resultField"
    },
    "policy": {
      "@id": "https://noocodec.dev/ontology/dag/policy"
    },
    "reducer": {
      "@id": "https://noocodec.dev/ontology/dag/reducer"
    },
    "outcome": {
      "@id": "https://noocodec.dev/ontology/dag/outcome"
    },
    "phase": {
      "@id": "https://noocodec.dev/ontology/dag/phase"
    },
    "stateMapping": {
      "@id": "https://noocodec.dev/ontology/dag/stateMapping"
    },
    "container": {
      "@id": "https://noocodec.dev/ontology/dag/container"
    },
    "DAG": {
      "@id": "https://noocodec.dev/ontology/dag/DAG"
    },
    "Placement": {
      "@id": "https://noocodec.dev/ontology/dag/Placement"
    },
    "SingleNode": {
      "@id": "https://noocodec.dev/ontology/dag/SingleNode"
    },
    "ScatterNode": {
      "@id": "https://noocodec.dev/ontology/dag/ScatterNode"
    },
    "EmbeddedDAGNode": {
      "@id": "https://noocodec.dev/ontology/dag/EmbeddedDAGNode"
    },
    "GatherNode": {
      "@id": "https://noocodec.dev/ontology/dag/GatherNode"
    },
    "TerminalNode": {
      "@id": "https://noocodec.dev/ontology/dag/TerminalNode"
    },
    "PhaseNode": {
      "@id": "https://noocodec.dev/ontology/dag/PhaseNode"
    }
  },
  "@id": "urn:noocodec:dag:normalize-json",
  "@type": "DAG",
  "name": "dag:normalize-json",
  "version": "1.0",
  "entrypoints": {
    "main": "urn:noocodec:dag:normalize-json/node/normalize-json-map"
  },
  "nodes": [
    {
      "@id": "urn:noocodec:dag:normalize-json/node/normalize-json-map",
      "@type": "SingleNode",
      "name": "dag:normalize-json/node/normalize-json-map",
      "node": "urn:noocodec:node:normalize-json-map",
      "outputs": {
        "normalized": "urn:noocodec:dag:normalize-json/node/normalized"
      }
    },
    {
      "@id": "urn:noocodec:dag:normalize-json/node/normalized",
      "@type": "TerminalNode",
      "name": "dag:normalize-json/node/normalized",
      "outcome": "completed"
    }
  ]
}

Mermaid generated from the same DAG

Mermaid source

%%{init: {"flowchart":{"nodeSpacing":92,"rankSpacing":104,"padding":28}}}%%
flowchart TB
  %% dag:normalize-json (v1.0)
  entry_main(["main"])
  entry_main --> urn_noocodec_dag_normalize-json/node/normalize-json-map
  urn_noocodec_dag_normalize-json/node/normalize-json-map["dag:normalize-json/node/normalize-json-map"]
  urn_noocodec_dag_normalize-json/node/normalize-json-map -->|normalized| urn_noocodec_dag_normalize-json/node/normalized
  urn_noocodec_dag_normalize-json/node/normalized((("dag:normalize-json/node/normalized")))

The important point is interface unification: plugin DAGs and ordinary embedded DAGs both enter the same registry and both execute through EmbeddedDAGNode.

Run

bash

npm run docs:dev

Open The Cartographer. The browser runner registers normalizeSourcesPlugin alongside the Cartographer DAG bundle before execution.

What It Lets You Do

Plugin-defined DAGs let applications consume reusable flows as installable parts while preserving the same embedded-DAG interface used inside the application. Use this when a team wants to package nodes and child DAGs once, register them by plugin ID, and embed the exported DAG IRIs from higher-level flows.

The result is composability without a second assembly language. Local DAGs, plugin DAGs, literal dag references, and dynamic DagReference bodies all resolve through the same registry.

Code Samples

The plugin packages real Cartographer normalization nodes and child DAGs:

import { defineDagonizerPlugin } from '@studnicky/dagonizer/plugin';

import { normalizeCsv } from '../nodes/ingest/normalizeCsv.ts';
import { normalizeJson } from '../nodes/ingest/normalizeJson.ts';
import { normalizeNdjson } from '../nodes/ingest/normalizeNdjson.ts';
import { normalizeYaml } from '../nodes/ingest/normalizeYaml.ts';
import { normalizeCsvDAG } from '../embedded-dags/NormalizeCsvDAG.ts';
import { normalizeJsonDAG } from '../embedded-dags/NormalizeJsonDAG.ts';
import { normalizeNdjsonDAG } from '../embedded-dags/NormalizeNdjsonDAG.ts';
import { normalizeYamlDAG } from '../embedded-dags/NormalizeYamlDAG.ts';

export const normalizeSourcesPlugin = defineDagonizerPlugin({
  'id': '@studnicky/dagonizer-cartographer-normalize-sources',
  'context': {
    'plugin': 'urn:noocodec:dagonizer:cartographer:normalize-sources',
  },
  'nodes': [
    normalizeCsv,
    normalizeJson,
    normalizeNdjson,
    normalizeYaml,
  ],
  'dags': [
    normalizeCsvDAG,
    normalizeJsonDAG,
    normalizeNdjsonDAG,
    normalizeYamlDAG,
  ],
  'exports': {
    'csv': 'normalize-csv',
    'json': 'normalize-json',
    'ndjson': 'normalize-ndjson',
    'yaml': 'normalize-yaml',
  },
});

The ingest DAG embeds the plugin-provided child DAG IRIs through the normal builder surface:

/**
 * IngestSourceDAG: orthogonal-compression, per-format-normalize ingestion DAG.
 *
 * Compression is a pre-step independent of format; each format has its own
 * parse node and normalization sub-DAG; all converge on the shared coerce/validate tail.
 *
 *   select-source ─compressed──► decompress ──► route-format
 *                 ─plain─────────────────────► route-format
 *                 ─invalid──────────────────────────────────► rejected
 *
 *   route-format ─csv────► parse-csv    ► [normalize-csv]    ─┐
 *                ─json───► parse-json   ► [normalize-json]   ─┤
 *                ─ndjson─► parse-ndjson ► [normalize-ndjson] ─┤─► coerce-types ─► validate-event ─► ingested
 *                ─yaml───► parse-yaml   ► [normalize-yaml]   ─┘
 *                ─invalid──────────────────────────────────────► rejected
 *
 * Terminals: ingested (completed), rejected (failed — unselectable / bad payload).
 */

// #region ingest-source-dag
import { selectSource }   from '../nodes/ingest/selectSource.ts';
import { decompress }     from '../nodes/ingest/decompress.ts';
import { routeFormat }    from '../nodes/ingest/routeFormat.ts';
import { parseCsv }       from '../nodes/ingest/parseCsv.ts';
import { parseJson }      from '../nodes/ingest/parseJson.ts';
import { parseNdjson }    from '../nodes/ingest/parseNdjson.ts';
import { parseYaml }      from '../nodes/ingest/parseYaml.ts';
import { normalizeCsv }   from '../nodes/ingest/normalizeCsv.ts';
import { normalizeJson }  from '../nodes/ingest/normalizeJson.ts';
import { normalizeNdjson } from '../nodes/ingest/normalizeNdjson.ts';
import { normalizeYaml }  from '../nodes/ingest/normalizeYaml.ts';
import { coerceTypes }    from '../nodes/ingest/coercion.ts';
import { validateEvent }  from '../nodes/ingest/validateEvent.ts';

import { normalizeCsvDAG }   from './NormalizeCsvDAG.ts';
import { normalizeJsonDAG }  from './NormalizeJsonDAG.ts';
import { normalizeNdjsonDAG } from './NormalizeNdjsonDAG.ts';
import { normalizeYamlDAG }  from './NormalizeYamlDAG.ts';

import type { CartographerState }    from '../CartographerState.ts';

import type { DAGType, DispatcherBundleType } from '@studnicky/dagonizer';
import { DAGBuilder, DAGIdentity } from '@studnicky/dagonizer';

const ingestSourceDagIri = 'urn:noocodec:dag:ingest-source' as const;
const normalizeCsvDagIri = 'urn:noocodec:dag:normalize-csv' as const;
const normalizeJsonDagIri = 'urn:noocodec:dag:normalize-json' as const;
const normalizeNdjsonDagIri = 'urn:noocodec:dag:normalize-ndjson' as const;
const normalizeYamlDagIri = 'urn:noocodec:dag:normalize-yaml' as const;
const placement = (placementIdentifier: string): string =>
  DAGIdentity.placementId(ingestSourceDagIri, placementIdentifier);

export const ingestSourceDAG: DAGType = new DAGBuilder(ingestSourceDagIri, '1.0')

  // 1. select-source: read source from scatter metadata; route by compression.
  .node(placement('select-source'), selectSource, {
    'compressed': placement('decompress'),
    'plain':      placement('route-format'),
    'invalid':    placement('rejected'),
  })

  // 2. decompress: base64-decode + gunzip → plain text in state.decodedText.
  //    Format-agnostic: any gzipped format passes through here to route-format.
  .node(placement('decompress'), decompress, {
    'route-format': placement('route-format'),
    'invalid':      placement('rejected'),
  })

  // 3. route-format: dispatch to the format-specific parser.
  .node(placement('route-format'), routeFormat, {
    'csv':     placement('parse-csv'),
    'json':    placement('parse-json'),
    'ndjson':  placement('parse-ndjson'),
    'yaml':    placement('parse-yaml'),
    'invalid': placement('rejected'),
  })

  // 4a. parse-csv: CSV text → state.parsedRecords.
  .node(placement('parse-csv'), parseCsv, {
    'normalized': placement('normalize-csv'),
    'invalid':    placement('rejected'),
  })

  // 4b. parse-json: JSON array → state.parsedRecords.
  .node(placement('parse-json'), parseJson, {
    'normalized': placement('normalize-json'),
    'invalid':    placement('rejected'),
  })

  // 4c. parse-ndjson: NDJSON text → state.parsedRecords.
  .node(placement('parse-ndjson'), parseNdjson, {
    'normalized': placement('normalize-ndjson'),
    'invalid':    placement('rejected'),
  })

  // 4d. parse-yaml: YAML sequence → state.parsedRecords.
  .node(placement('parse-yaml'), parseYaml, {
    'normalized': placement('normalize-yaml'),
    'invalid':    placement('rejected'),
  })

  // 5a. normalize-csv: embedded sub-DAG — apply FieldMap by header name.
  //     The FieldMap is name-keyed, so shuffled CSV column order aligns correctly.
  .embed<CartographerState, CartographerState>(placement('normalize-csv'), normalizeCsvDagIri, {
    'success': placement('coerce-types'),
    'error':   placement('rejected'),
  }, {
    // Embedded DAGs run in an isolated state clone: thread the parsed records
    // and the current source (for its mappingKey) IN, and the aligned records
    // back OUT, or the normalize node sees empty input and nothing merges back.
    'inputs':  { 'parsedRecords': 'parsedRecords', 'currentSource': 'currentSource' },
    'outputs': { 'mappedRecords': 'mappedRecords' },
  })

  // 5b. normalize-json: embedded sub-DAG — apply FieldMap by key name.
  .embed<CartographerState, CartographerState>(placement('normalize-json'), normalizeJsonDagIri, {
    'success': placement('coerce-types'),
    'error':   placement('rejected'),
  }, {
    // Embedded DAGs run in an isolated state clone: thread the parsed records
    // and the current source (for its mappingKey) IN, and the aligned records
    // back OUT, or the normalize node sees empty input and nothing merges back.
    'inputs':  { 'parsedRecords': 'parsedRecords', 'currentSource': 'currentSource' },
    'outputs': { 'mappedRecords': 'mappedRecords' },
  })

  // 5c. normalize-ndjson: embedded sub-DAG — apply FieldMap by key name.
  .embed<CartographerState, CartographerState>(placement('normalize-ndjson'), normalizeNdjsonDagIri, {
    'success': placement('coerce-types'),
    'error':   placement('rejected'),
  }, {
    // Embedded DAGs run in an isolated state clone: thread the parsed records
    // and the current source (for its mappingKey) IN, and the aligned records
    // back OUT, or the normalize node sees empty input and nothing merges back.
    'inputs':  { 'parsedRecords': 'parsedRecords', 'currentSource': 'currentSource' },
    'outputs': { 'mappedRecords': 'mappedRecords' },
  })

  // 5d. normalize-yaml: embedded sub-DAG — apply FieldMap by key name.
  .embed<CartographerState, CartographerState>(placement('normalize-yaml'), normalizeYamlDagIri, {
    'success': placement('coerce-types'),
    'error':   placement('rejected'),
  }, {
    // Embedded DAGs run in an isolated state clone: thread the parsed records
    // and the current source (for its mappingKey) IN, and the aligned records
    // back OUT, or the normalize node sees empty input and nothing merges back.
    'inputs':  { 'parsedRecords': 'parsedRecords', 'currentSource': 'currentSource' },
    'outputs': { 'mappedRecords': 'mappedRecords' },
  })

  // 6. coerce-types: string cells → number / bool / epoch. Shared tail.
  .node(placement('coerce-types'), coerceTypes, {
    'validate-event': placement('validate-event'),
  })

  // 7. validate-event: build CanonicalEvents → state.ingestedEvents. Shared tail.
  .node(placement('validate-event'), validateEvent, {
    'validated': placement('ingested'),
  })

  // Terminals
  .terminal(placement('ingested'), { outcome: 'completed' })
  .terminal(placement('rejected'), { outcome: 'failed' })

  .build();

export const ingestSourceBundle: DispatcherBundleType<CartographerState> = {
  // Normalize DAGs registered FIRST so the embedded-DAG placements above resolve.
  'nodes': [
    selectSource, decompress, routeFormat,
    parseCsv, parseJson, parseNdjson, parseYaml,
    normalizeCsv, normalizeJson, normalizeNdjson, normalizeYaml,
    coerceTypes, validateEvent,
  ],
  'dags': [
    normalizeCsvDAG, normalizeJsonDAG, normalizeNdjsonDAG, normalizeYamlDAG,
    ingestSourceDAG,
  ],
};
// #endregion ingest-source-dag

The browser demo registers the plugin before registering the ingest and top-level bundles:

vue

dispatcher.registerPlugin(normalizeSourcesPlugin);

Details for Nerds

One registry seam. registerPlugin(normalizeSourcesPlugin) adds nodes and DAGs through the same registry used by registerBundle().
No plugin-specific placement. IngestSourceDAG embeds urn:noocodec:dag:normalize-csv and urn:noocodec:dag:normalize-json exactly like any other child DAG.
Scoped package exports. Applications depend on plugin exports and DAG IRIs, not on plugin internals.
Runnable browser assembly. The Cartographer HMR page registers the plugin during each run, so docs and runtime use the same assembly pattern.

Guide: Plugins - defineDagonizerPlugin, PluginLoader, and registry-wide plugin loading
Guide: DAGBuilder - the embed() builder and embedded DAG state mapping
Example 05: EmbeddedDAGNode composition - same child/parent shape without plugin packaging

Example 33: Plugin-Defined DAGs

What It Is

How It Works

Diagrams, Examples, and Outputs

DAG registration and diagram

Cartographer ingest DAG embedding plugin DAGs

plugin-provided normalize-csv DAG

plugin-provided normalize-json DAG

Run

What It Lets You Do

Code Samples

Details for Nerds

Example 33: Plugin-Defined DAGs ​

What It Is ​

How It Works ​

Diagrams, Examples, and Outputs ​

DAG registration and diagram ​

Run ​

What It Lets You Do ​

Code Samples ​

Details for Nerds ​

Related Concepts ​

Example 33: Plugin-Defined DAGs

What It Is

How It Works

Diagrams, Examples, and Outputs

DAG registration and diagram

Run

What It Lets You Do

Code Samples

Details for Nerds

Related Concepts