Architecture

dcupl follows a modular architecture where you can use as much or as little as you need. At its simplest, you use the SDK alone. For more complex needs, you add the Console platform.

High-level overview

flowchart TB
  subgraph free["Free on npm"]
    SDK["@dcupl/core<br/>Query Engine"]
    LOADER["@dcupl/loader<br/>Data Loading"]
    CONNECT["@dcupl/connect<br/>Debugging"]
    CLI["@dcupl/cli<br/>Scaffold, Sync, Local Apps"]
  end

  subgraph saas["Console Platform (Optional)"]
    CONSOLE["Console UI<br/>Model & Workflow Editor"]
    RUNNER["Runner (workflow-runner)<br/>Data Transformation"]
    RAPI["Runner (app-runner)<br/>REST API (RAPI) instances"]
  end

  subgraph storage["Storage"]
    CDN["CDN<br/>JSON Files"]
    SOURCES["Data Sources<br/>CSV, APIs, etc."]
  end

  SOURCES --> RUNNER
  RUNNER --> CDN
  CDN --> LOADER
  LOADER --> SDK
  CONSOLE --> RUNNER
  CONSOLE --> RAPI
  SDK --> APP[Your Application]
  CONNECT -.-> CONSOLE
  CLI -.->|"files push / pull"| CDN

SDK packages

The SDK is a set of free npm packages. Every app installs @dcupl/core; add the others when you need them, e.g. npm install @dcupl/core @dcupl/loader. @dcupl/loader and @dcupl/connect build on @dcupl/core, and the packages require each other in exactly their own version, so install every @dcupl/* package in the same version.

@dcupl/common

The types and helpers you import in your own code. Add it next to @dcupl/core when you import from it (npm install @dcupl/common):

• Types - ModelDefinition, DcuplQuery, DcuplQueryGroup, DcuplListOptions and the other types the SDK takes and returns
• DcuplQueryBuilder - Builds and edits query groups
• logger - The shared logger that @dcupl/loader logs through (see Logging Configuration)

@dcupl/core

The main query engine and entry point:

• Dcupl - The instance you create: you register models and data, call init(), create lists, and run one-off queries and functions on a model (dcupl.query, dcupl.fn)
• DcuplList - A filtered view of one model (dcupl.lists.create())
• Catalog - The query interface on a list (list.catalog): filtering, sorting, facets and aggregations

@dcupl/loader

Data ingestion and configuration loading:

• DcuplAppLoader - Loads a loader configuration (dcupl.lc.json) and its resources from the dcupl CDN or any URL
• DcuplCsvParser - CSV parsing with type coercion
• Variable interpolation for environment-specific URLs

@dcupl/connect

Development debugging tool:

• Real-time SDK inspection in Console UI
• Forwards dcupl, list and loader events (such as query changes) to the Console window that opened your app
• State visualization during development

Execution contexts

The same SDK code runs in multiple environments:

flowchart LR
  subgraph browser["Browser"]
    B_SDK["dcupl SDK"]
    B_DATA["Data in Memory"]
    B_SDK --> B_DATA
  end

  subgraph node["Node.js"]
    N_SDK["dcupl SDK"]
    N_DATA["Data in Memory"]
    N_SDK --> N_DATA
  end

  subgraph rapi["REST API instance on a runner"]
    R_SDK["dcupl SDK"]
    R_DATA["Data in Memory"]
    R_SDK --> R_DATA
  end

  USER[User] --> browser
  SSR[SSR Framework] --> node
  API[API Request] --> rapi

Browser (client-side)

The most common use case. Data loads into the browser, and all queries execute locally.

Benefits:

• Zero server cost for queries
• Instant user interactions
• Works offline after initial load

Best for: Interactive filtering interfaces on data that fits in the browser's memory (see Data size and limits)

Node.js (server-side)

The same SDK runs on the server for SSR or batch processing.

Benefits:

• SEO-friendly initial renders
• Can handle larger datasets with more memory
• Server-to-server data processing

Best for: Initial page renders, large dataset processing, API endpoints

Hybrid pattern

Combine both for optimal results:

sequenceDiagram
  participant Browser
  participant Server
  participant Data

  Server->>Data: Load data
  Server->>Server: Query for initial render
  Server->>Browser: HTML with data
  Browser->>Data: Load the same data
  Browser->>Browser: dcupl.init()
  Browser->>Browser: All subsequent queries (local)

The server renders the initial page for SEO. The browser then loads the same data into its own dcupl instance (the SDK does not transfer its state from the server) and handles all further user interactions locally.

The Console platform

The Console (console.dcupl.com) is an optional SaaS platform on top of the SDK. In it you edit models and workflows, keep versioned project files, and deploy workflows and REST API instances to runners. Your project files, and the output a workflow writes into them, are served from the dcupl CDN; workflows can also write to S3, Azure or Git. The High-Level Overview shows how these parts connect, and the sections below describe each one.

Console UI

The web application at console.dcupl.com, for:

• Visual model editor
• Workflow designer
• Data exploration (the Data Explorer)
• Project management

Console API

The HTTP API of the Console, at https://api.dcupl.com. The CLI calls it for dcupl files push and pull and for the dcupl workflow commands, authenticated with a CLI API key (see API Keys). For a self-hosted Console, point the CLI at yours with dcupl config set --console-api-url <url>. Scripts and CI jobs can also call it directly with an api-key header, for example the Reporting API endpoints.

Workflow runner

Executes data transformation workflows:

• Fetches data from external sources
• Applies transformations
• Can validate the data against your models (in a dcupl instance node)
• Stores processed JSON

Runner (REST API instances)

A runner of type app-runner hosts deployed REST API instances:

• Exposes SDK queries as REST APIs
• Keeps the data in server memory, for datasets too large for the browser
• Checks the API key set in the template (auth) on every request; an instance without one is open
• Can also serve an MCP endpoint per instance, so AI agents can query it. You switch it on per REST API template; MCP is part of a paid plan that dcupl turns on for your project, so ask your dcupl contact (see MCP endpoint).

Files and CDN

The Console stores versioned project files (models, data and the loader configuration), and the dcupl CDN (cdn.dcupl.com) serves them to the SDK's loader. The CLI syncs your local files with the Console (dcupl files push / pull).

CLI

The @dcupl/cli package works on the same files from your terminal: it scaffolds projects, validates models (dcupl validate), syncs files with the Console, deploys workflows, and runs local data apps (dcupl app) with the SDK in memory, which you can query from the terminal. See How the CLI works.

Data storage

dcupl uses a JSON-first architecture for portability:

flowchart LR
  subgraph config["Configuration"]
    LC["dcupl.lc.json<br/>LoaderConfiguration"]
    MODELS["*.dcupl.json<br/>Model Definitions"]
  end

  subgraph data["Data"]
    JSON["*.json<br/>Data Files"]
  end

  LC --> MODELS
  LC --> JSON
  MODELS --> SDK
  JSON --> SDK

LoaderConfiguration

The central configuration file (dcupl.lc.json) defines:

• Resources - Model, data, script, operator and transformer file URLs with tags
• Applications - Which resources to load for each app
• Environments - Environment-specific variables
• Variables - URL interpolation values (per environment or application)

Model files

JSON files defining data structure:

{
  "key": "Product",
  "properties": [
    { "key": "name", "type": "string" },
    { "key": "price", "type": "int" }
  ]
}

Data files

JSON arrays of data matching model structure:

[
  { "key": "p1", "name": "Laptop", "price": 999 },
  { "key": "p2", "name": "Mouse", "price": 29 }
]

Integration patterns

Pattern 1: SDK only

Simplest setup for static or developer-managed data.

import { Dcupl } from '@dcupl/core';

const dcupl = new Dcupl();
dcupl.models.set(productModel);
dcupl.data.set(productData, { model: 'Product' });
await dcupl.init();

Pattern 2: SDK + loader

Load configuration and data from URLs (the dcupl CDN or your own server).

import { Dcupl } from '@dcupl/core';
import { DcuplAppLoader } from '@dcupl/loader';

const dcupl = new Dcupl({
  config: { projectId: 'your-project', apiKey: 'your-api-key' },
});
const loader = new DcuplAppLoader();
dcupl.loaders.add(loader);

await loader.config.fetch();
await loader.process();
await dcupl.init();

For files you host yourself, pass their location instead: await loader.config.fetch({ baseUrl: 'https://example.com/dcupl' }) loads https://example.com/dcupl/dcupl.lc.json and fills the ${baseUrl} variable in the resource URLs. projectId and apiKey are only needed for the dcupl CDN.

Pattern 3: SDK + Console + REST API

Full platform with server-side queries for large datasets.

flowchart LR
  CLIENT[Browser SDK] --> |"Small datasets"| LOCAL[Local Query]
  CLIENT --> |"Large datasets"| RAPI[REST API instance]
  RAPI --> SERVER[Server SDK]

Key takeaways

1. Modular design - Use only what you need
2. Universal SDK - Same code runs everywhere
3. JSON-first - All configuration is portable
4. Layered architecture - SDK alone, with Loader, or full Console