# dcupl Documentation — Full Corpus > Concatenated documentation for dcupl, generated from https://docs.dcupl.com. Each section below corresponds to a single documentation page. These docs describe @dcupl/core 2.1 and @dcupl/cli 2.0. --- # Introduction to dcupl Source: https://docs.dcupl.com/introduction Description: What dcupl gives coding agents, what it is technically (SDK, CLI, Console, REST API), where it fits and where it does not # Introduction to dcupl Your agent can write the app. It can't guess your data. dcupl gives coding agents the real model of your product data, JSON answers they can parse and a build step you approve. The same model runs in production, served via TypeScript SDK and REST API (RAPI), and via MCP where it is turned on. ## Three things your agent gets that it can't make up Agents don't crash on bad data. They succeed on the wrong data. That is the failure dcupl removes: - **The real model, not a guess.** The agent reads actual properties and types before it writes a filter (`dcupl app models get --model Product`). For dcupl's own formats, `dcupl schemas get ModelDefinition --example` does the same. - **An answer it can parse.** With `--json`, the `app`, `validate`, `workflow`, `files` and `schemas` commands answer in structured JSON, errors included. The agent knows what happened instead of scraping console text. - **Changes you approve.** Data changes are staged first. One `dcupl app build` applies them, when you say so. Works with Claude Code, Cursor, Codex and Copilot. The CLI plus `--json` is all an agent needs; the [dcupl skill](/agentic/the-skill) for Claude Code adds command patterns and known pitfalls. See [Build with agents](/agentic). ## What dcupl is Technically, dcupl is a TypeScript SDK, a CLI and the Console, all working on the same models and data: ```mermaid flowchart TB subgraph sdk["SDK (free)"] MODELS[Models] --> ENGINE[Query Engine] DATA[Data] --> ENGINE ENGINE --> LISTS[Lists & Catalogs] end CLI["CLI (free)"] subgraph console["Console (optional)"] EDITOR[Model Editor] --> WORKFLOWS[Data Workflows] WORKFLOWS --> STORAGE[CDN Storage] STORAGE -.->|"fetch data"| DATA end CLI -->|"dcupl app"| ENGINE CLI -->|"files push / pull"| STORAGE ``` ### TypeScript SDK (free) The core `@dcupl/core` package runs in any JavaScript environment, in the browser or in Node.js. It provides: - **Models** - Define your data structure (like a database schema) - **Query engine** - Filter, sort, and search with a flexible query language - **Lists and catalogs** - Create queryable views of your data with facets and aggregations The SDK is free, on npm, and works without any backend. ### CLI (free) The free `@dcupl/cli` is the agent's way in, and yours: `dcupl init` scaffolds a project, `dcupl serve` serves its files locally, `dcupl validate` checks the models, `dcupl app` queries the data from your terminal, and `dcupl files push` / `dcupl files pull` sync the files with the Console. With `--json`, the `app`, `validate`, `workflow`, `files` and `schemas` commands answer in JSON. See [How the CLI works](/cli/how-it-works). ### Console (optional) The [dcupl Console](/console) is a web platform that adds: - **Visual model editor** - Design data models with a UI - **Data workflows** - Load, join and clean exports (PIM dumps, ERP prices, file drops on Azure Blob or S3), run them on a dev runner first, then go to production - **REST API deployment** - Run dcupl queries server-side when needed; a deployed instance can be an MCP server where it is turned on Files and versions, the data explorer and team roles are on the [Console overview](/console). You can use the SDK alone for simpler projects, or add the Console for data pipelines and APIs. ## Where to start Pick the path that matches how you work: - **You write the app in TypeScript.** Start with the [SDK quick start](/sdk/quick-start): models, data and a filtered list in a few minutes. - **You manage data in the Console.** Start with your [first Console project](/console/getting-started): import a file, check the model, run the application. - **Your agent does the work.** Start with the [agent setup](/agentic/setup): install the CLI, give the agent the skill and let it drive `dcupl` with `--json`. Whichever path you take, the [CLI](/cli) scaffolds, serves, validates, queries and syncs the same project files. ## Queries without a backend This is the frontend-first part of dcupl. A product catalog, data table or search interface usually needs a backend round-trip for every filter change. dcupl loads the data once and runs every query locally: ```mermaid flowchart LR subgraph traditional["Traditional approach"] direction TB FE[Frontend] -->|"filter request"| API[Backend API] API -->|"SQL query"| DB[(Database)] DB -->|"results"| API API -->|"JSON"| FE end subgraph dcupl["dcupl approach"] direction TB DATA[Data Source] -->|"load once"| SDK["dcupl SDK\n(in browser)"] SDK -->|"instant queries"| UI[Your UI] end ``` That removes three costs of the round-trip: - **Network latency** - Users wait for a network round-trip on every interaction - **Server costs** - Every user action requires server compute - **Backend dependencies** - Frontend teams wait for API endpoints Because you load the actual data from the start (spreadsheets, CSV or JSON files, or files managed in the Console), a prototype built with dcupl reflects reality instead of breaking when the real data arrives. ## When should you use dcupl? dcupl works well for applications where users explore and filter data: | Use Case | Example | Why dcupl Helps | | ----------------------- | ---------------------------- | -------------------------------------------- | | **Product catalogs** | E-commerce, marketplaces | Fast faceted search with instant filtering | | **Data dashboards** | Analytics, admin panels | Client-side aggregations without server load | | **Search interfaces** | Document search, directories | Instant results as users type | | **Configuration tools** | Product configurators | Complex dependent filters | ### When to consider alternatives dcupl is not the right choice for every project: - **Real-time collaboration** - If multiple users edit the same data simultaneously, use a real-time database like Firebase or Supabase - **Constantly changing data** - dcupl is optimized for read-heavy workloads - **Sensitive data** - Any data you ship to the browser is inspectable by the user. Keep secrets, personal data, and records users shouldn't see on the server and only send what the current user is allowed to query - **Very large datasets in the browser** - Data that is too large to download to every client doesn't belong there. Run it on a server-hosted dcupl instance exposed as a [REST API](/console/rapi), or use a dedicated search service ## What dcupl is not | Not this | Explanation | | --------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Not a database** | dcupl doesn't replace PostgreSQL or MongoDB. It loads data into memory for fast querying. You can store files on the dcupl CDN, but it is not a transactional database. | | **Not a CMS** | dcupl doesn't manage content creation or editorial workflows. Use it alongside a CMS if needed. | | **Not a backend framework** | dcupl reduces backend dependencies; it doesn't replace your entire backend. | | **Not a search service** | Search services like Algolia run server-side and charge per query, with typo tolerance and relevance ranking. dcupl queries locally and has neither. | For a detailed comparison, see [dcupl vs alternatives](/learn/concepts/dcupl-vs-alternatives). ## Data size and limits Measured with 1 million items: indexed queries took 25–65 ms, facets about 2.7 s. Query time grows with the number of items, and what fits depends on your models, your data and the device, because the SDK keeps all data in memory: | Environment | Constraint | What to do | | ----------- | ----------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------- | | **Browser** | Browser memory (varies by device) | Load only the data the app needs, or query a REST API instance | | **Node.js** | Node.js heap memory (default depends on system memory, often ~2–4 GB; raise it with `--max-old-space-size`) | Increase the heap size, or run the queries on a REST API instance | | **Any** | A model holds at most about 16.7 million items (V8's `Map` size limit) | Split the data across models or reduce it | To work with large datasets: - Structure data efficiently (normalize with references, avoid duplication) - Load only what's needed: select resources with tags, or drop rows with a parsed-file transformer in the loader - Run heavy queries on a [REST API instance](/console/rapi) and send the results to the client, or combine both: heavy queries on the server, instant filtering of the result on the client ## Where dcupl runs | Environment | Use Case | | --------------------------------- | ------------------------------------------------------------- | | **Browser** | Interactive UIs, instant filtering, works offline once loaded | | **Node.js** | SSR, build-time data processing, scripts | | **REST API instance on a runner** | Server-side API for non-browser clients, heavy queries | Same SDK, same query API, with any JavaScript framework (React, Vue, Angular, Svelte or none). With SSR, the first page load carries real data; the browser then loads the same data into its own dcupl instance and takes over the interactions. ## Who uses dcupl? **Developers who build with an agent** point the agent at an export and get a typed model and a working app on real data. The agent does the keystrokes; you see what it did, what is pending and what passed, and you approve the build. See the [CLI quickstart](/cli) and [Build with agents](/agentic). **Agencies that deliver data-heavy apps for clients** reuse models and workflows across projects and keep agent changes reviewable. Models, workflows and quality rules are JSON files in the repository; workflows run on a dev runner before they reach the client, and the Console canvas edits the same files the agent writes. See [Workflows](/console/workflows). **Companies that run the app and own the data** get clean, typed product data for their apps and their own AI agents. The runtime can run on their own infrastructure, and a deployed REST API instance can be an MCP server where it is turned on. See [REST APIs](/console/rapi) and the [MCP endpoint](/console/rapi/mcp). ## Key concepts Here are the terms you'll see throughout the docs: **Model** - A definition of your data structure. Similar to a database table schema or TypeScript interface. **Property** - A field in a model (name, price, category). Properties have types like `string`, `int`, `float`, `boolean`, or `date`. **List** - A queryable view of model data. You create lists, add filter conditions, and retrieve results. **Catalog** - The query interface on a list. Use it to filter, sort, get facet counts, and run aggregations. **Facets** - Counts of items per value, used for filter UIs (e.g., "Electronics (24), Furniture (12)"). **Reference** - A property that points to items of another model, single- or multi-valued (a product's `category`). References let you query across models, e.g. on `category.name`. **init / update** - Data and model changes are staged until you apply them: `await dcupl.init()` once, then `await dcupl.update()` after later changes. Both resolve to `false` when they fail instead of throwing, so check the result. ## A simple example Here is a complete example that loads products and filters them: ```typescript filename="example.ts" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); // 1. Define the data structure dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'int' }, { key: 'category', type: 'string' }, ], }); // 2. Load data (can come from an API, file, or inline) dcupl.data.set( [ { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics' }, { key: 'p2', name: 'Mouse', price: 29, category: 'Electronics' }, { key: 'p3', name: 'Desk', price: 299, category: 'Furniture' }, { key: 'p4', name: 'Chair', price: 199, category: 'Furniture' }, ], { model: 'Product' }, ); // 3. Initialize (builds indexes for fast queries) await dcupl.init(); // 4. Create a list and filter const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // 5. Get results console.log(list.catalog.query.items()); // [{ key: 'p1', name: 'Laptop', ... }, { key: 'p2', name: 'Mouse', ... }] // 6. Get facet counts for a filter UI console.log(list.catalog.fn.facets({ attribute: 'category' })); // [{ value: 'Electronics', size: 2, selected: true, ... }, // { value: 'Furniture', size: 2, selected: false, ... }] ``` To build and run this step by step, continue with the [Quick start](/sdk/quick-start). ## Next steps - [Quick start](/sdk/quick-start) — Build your first dcupl application in 5 minutes - [CLI](/cli) — Scaffold a project, run it locally, sync it with the Console - [Build with agents](/agentic) — Let an AI agent drive the CLI - [Console](/console) — Manage models, workflows and REST APIs without code - [Core concepts](/sdk/core-concepts) — Understand models, lists, and queries - [Architecture](/learn/concepts/architecture) — How SDK, CLI and Console work together - [React Integration](/learn/tutorials/react-integration) — Use dcupl in a React app - [Vue Integration](/learn/tutorials/vue-integration) — Use dcupl in a Vue app --- # TypeScript SDK Source: https://docs.dcupl.com/sdk Description: The dcupl SDK is the in-memory query engine under dcupl: define typed models, load data from any source and query them in the browser or in Node.js, with… # TypeScript SDK The dcupl SDK is the in-memory query engine under dcupl: define typed models, load data from any source and query them in the browser or in Node.js, with relationships, indexing, facets and change events built in. **npm** ```bash npm install @dcupl/core ``` **yarn** ```bash yarn add @dcupl/core ``` **pnpm** ```bash pnpm add @dcupl/core ``` These docs describe `@dcupl/core` 2.1 and `@dcupl/cli` 2.0. The npm packages are published under the ISC license. [Quick start](/sdk/quick-start) [Playground](/playground) ## How it works ### Define models Create typed schemas with properties, relationships, and validation rules. Models define the structure of your data and how each property is parsed, validated and indexed. ```typescript dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'int' }, { key: 'category', type: 'string' }, ] }); ``` ### Load data Load data from inline arrays, fetch from APIs, or use the Loader with Console. Data is indexed automatically for fast queries. ```typescript // Inline data dcupl.data.set(products, { model: 'Product' }); // Initialize and build indexes await dcupl.init(); ``` ### Query instantly Filter, sort, aggregate, and analyze your data. All queries run in memory, equality filters against inverse indexes — no server round-trips needed. ```typescript const list = dcupl.lists.create({ modelKey: 'Product' }); // Add filters list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); // Get results const items = list.catalog.query.items(); ``` ## Key capabilities ### Data models Typed schemas with validation rules and computed properties ### Relationships Connect models via references — products to categories, orders to users ### Indexed queries Equality filters (eq, in) use inverse indexes — no full scan, no server round-trip ### Analysis functions Facets, aggregations, groupBy, pivot tables — built-in ### Real-time updates Upsert, delete, refresh data — lists refresh and notify you on update() ### TypeScript first TypeScript types included, IntelliSense, works with any framework ## Explore the SDK The SDK docs are organized into three tracks. Start with Learn if you're new, jump to Build when you're applying dcupl to a real project, and use Reference to look up exact APIs. ### Learn - [Quick start](/sdk/quick-start) — Build a working example in 5 minutes - [Setup options](/sdk/setup-options) — Choose between SDK-only and Console + Loader setups - [Core concepts](/sdk/core-concepts) — Models, data, lists, and the query engine - [Querying](/sdk/queries/query-filter) — Filters, logic, ranges, search, and deep queries ### Build - [Analysis functions](/sdk/fn/facets) — Facets, aggregations, groupBy, pivot, and suggestions - [Advanced patterns](/sdk/advanced/faceted-search) — Faceted search, master–detail, infinite scroll, realtime - [Guides](/sdk/guides/typescript) — TypeScript, testing, debugging, deployment, scripts, and common mistakes - [Performance & caching](/sdk/advanced/performance) — Tune indexing, memoization, and data loading for large datasets ### Reference - [API reference](/sdk/api) — Every class, method, and type exported by @dcupl/core - [Quick reference](/sdk/reference) — One-page cheatsheet for the most common SDK calls --- # Quick start Source: https://docs.dcupl.com/sdk/quick-start Description: Build a filterable product list in 5 minutes # Quick start Build a filterable product list in 5 minutes. **What you'll build:** A query that filters products by category and returns matching items instantly — no backend required. ## Prerequisites - A supported Node.js version, see [Compatibility](/sdk/compatibility#nodejs) (`node -v` to check) - A TypeScript/JavaScript project (or use `npx tsx` to run standalone) ## Install ```bash group="install" tab="npm" npm install @dcupl/core ``` ```bash group="install" tab="yarn" yarn add @dcupl/core ``` ```bash group="install" tab="pnpm" pnpm add @dcupl/core ``` TypeScript types are included; type names such as `ModelDefinition` come from `@dcupl/common`, which you install (`npm install @dcupl/common`) only when you import them in your own code. ## Create your first query Create a new file and paste this complete example: ```typescript filename="demo.ts" import { Dcupl } from '@dcupl/core'; async function main() { // 1. Create a dcupl instance const dcupl = new Dcupl(); // 2. Define a model with filterable properties dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'int' }, { key: 'category', type: 'string' }, ], }); // 3. Load data (inline or from external API) dcupl.data.set( [ { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics' }, { key: 'p2', name: 'Mouse', price: 29, category: 'Electronics' }, { key: 'p3', name: 'Desk', price: 299, category: 'Furniture' }, ], { model: 'Product' }, ); // 4. Initialize (builds indexes). Resolves false, and logs the error, // if processing fails; it doesn't throw. await dcupl.init(); // 5. Create a list to query the model const list = dcupl.lists.create({ modelKey: 'Product' }); // 6. Add filter conditions list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // 7. Get results const results = list.catalog.query.items(); console.log('Filtered results:', results); // → [ // { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics' }, // { key: 'p2', name: 'Mouse', price: 29, category: 'Electronics' } // ] } main(); ``` > Run it with `npx tsx demo.ts` or your preferred TypeScript runner. **Want to skip setup?** [→ Try it in the Playground](/playground) — experiment with filters, sorting, and facets live. ## Common operations ```typescript title="Common query operations" // Conditions on different attributes combine with AND (with step 6: Electronics and price >= 50) list.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 50 }); list.catalog.query.items(); // → Laptop // Conditions on the same attribute combine with OR list.catalog.query.clear(); // removes all conditions and sort/pagination options list.catalog.query.addCondition([ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Furniture' }, ]); // → Laptop, Mouse, Desk // OR across attributes: a group with an explicit groupType list.catalog.query.clear(); list.catalog.query.addGroup({ groupKey: 'cheap-or-furniture', groupType: 'or', queries: [ { attribute: 'category', operator: 'eq', value: 'Furniture' }, { attribute: 'price', operator: 'lt', value: 50 }, ], }); // → Desk, Mouse // Sort ('ASC' or 'DESC', uppercase) and paginate list.catalog.query.clear(); list.catalog.query.applyOptions({ sort: { attributes: ['price'], order: ['ASC'] }, start: 0, count: 2 }); list.catalog.query.items(); // → Mouse, Desk ``` Every other common call (facets, metadata, aggregations) is on one page: [Quick reference](/sdk/reference). ## Update data Data changes after `init()` are staged. Apply them with `dcupl.update()`; lists then return the new results. ```typescript title="Update data after init" dcupl.data.upsert([{ key: 'p4', name: 'Monitor', price: 199, category: 'Electronics' }], { model: 'Product' }); await dcupl.update(); // resolves false, and logs the error, if the update fails list.catalog.query.items(); // → Mouse, Monitor (still sorted by price, 2 per page) ``` ## Next steps - [Setup options](/sdk/setup-options) — Load from the Console, your own files or a script tag - [Core concepts](/sdk/core-concepts) — Understand models, lists, and query - [Tutorial: Your First App](/learn/tutorials/first-app) — Build a small app around these calls - [Models](/sdk/core-concepts/models) — Property types, references, validation rules - [Querying](/sdk/queries/query-filter) — All filter operators - [Facets](/sdk/fn/facets) — Build filter UIs with counts --- # Setup options Source: https://docs.dcupl.com/sdk/setup-options Description: Choose between Console + Loader (recommended), the Loader with your own files, or an SDK-only setup # Setup options dcupl offers three setups: **Console + Loader** (recommended), the **Loader with your own files**, and **SDK-only**. All of them use the same query API—your application code stays the same regardless of which you choose. ## Console + Loader (recommended) The [Console](/console) manages your models and data sources visually. The **Loader** fetches everything at runtime from Console's CDN. **Why it's recommended:** - **Visual model editor** – Define schemas without writing JSON - **External data sources** – Load data from any URL, or pull it in from APIs, S3, Azure or git with workflows - **Environments** – Per-environment variables and headers (e.g. dev, staging, prod) in the loader configuration - **Team collaboration** – Non-developers can edit models and data - **Version control** – Draft → Production file versions for safe deployments Where the project ID and the Browser key come from, and how to pin a version or pick an environment: [Load from the console](/sdk/guides/load-from-console). ## Loader with your own files The Loader doesn't need the Console. Point it at any host that serves a `dcupl.lc.json` and the files it lists: your own server or CDN, or `dcupl serve` from the [CLI](/cli), which serves your project's `dcupl/` folder on `http://localhost:8083` by default. Models are JSON files you host next to your data, and versioning is up to you (e.g. git). **Use cases:** - Local development against the files on your disk - Projects that keep models and data in git and deploy them with the app - Hosting the files yourself instead of on the dcupl CDN ## SDK-only Define models and load data directly in code. No external dependencies. **Use cases:** - Quick prototypes and learning - Small, static datasets bundled with your app - Full programmatic control required ## Comparison | | Console + Loader (recommended) | Loader with your own files | SDK-only | | ---------------- | ------------------------------- | ------------------------------- | ---------------------- | | **Packages** | `@dcupl/core` + `@dcupl/loader` | `@dcupl/core` + `@dcupl/loader` | `@dcupl/core` | | **Models** | Visual editor in Console | JSON files you host | Defined in code | | **Data** | Auto-fetched from URLs | Auto-fetched from URLs | Inline or manual fetch | | **Environments** | Loader environments you define | Loader environments you define | Manual config | | **Versioning** | Draft → Production versions | Up to you (e.g. git) | Up to you (e.g. git) | ## Setup code ```typescript group="setup" tab="Console + Loader" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; const dcupl = new Dcupl({ config: { projectId: 'your-project-id', // From Console apiKey: 'your-api-key', // From Console version: 'draft', // CDN file version: 'draft' (default) or 'production' }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // Fetch configuration and data from Console CDN await loader.config.fetch(); await loader.process(); // loads every resource in the configuration await dcupl.init(); // Query your data const list = dcupl.lists.create({ modelKey: 'Product' }); const results = list.catalog.query.items(); ``` ```typescript group="setup" tab="Your own files" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // Fetches `${baseUrl}/dcupl.lc.json`; `${baseUrl}` in resource URLs // resolves to the same host await loader.config.fetch({ baseUrl: 'http://localhost:8083' }); // dcupl serve, or your own host await loader.process(); await dcupl.init(); // Query your data const list = dcupl.lists.create({ modelKey: 'Product' }); const results = list.catalog.query.items(); ``` ```typescript group="setup" tab="SDK-Only" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); // Define model in code dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ], }); // Load data inline dcupl.data.set( [ { key: 'p1', name: 'Widget', price: 29.99 }, { key: 'p2', name: 'Gadget', price: 49.99 }, ], { model: 'Product' }, ); await dcupl.init(); // Query your data const list = dcupl.lists.create({ modelKey: 'Product' }); const results = list.catalog.query.items(); ``` For every `new Dcupl()` option, see [DcuplInitOptions](/sdk/api/dcupl#dcuplinitoptions). ## Script tag (no build step) For a page without a bundler, load the `@dcupl/bundle` browser build. It contains the SDK and the Loader and defines one global function, `initDcupl()`: ```html filename="index.html" ``` - `initDcupl({ options, appLoaderConfig, processOptions })` runs the Console + Loader setup above: `new Dcupl(options)`, `loader.config.fetch(appLoaderConfig)`, `loader.process(processOptions)` and `dcupl.init()`. It resolves `{ dcupl, loader }` and also sets `window.dcupl`. It rejects when the loader configuration can't be fetched (`Invalid Loader Configuration`). - For your own files, pass `appLoaderConfig: { baseUrl: 'http://localhost:8083' }` instead of the project config. - It resolves even when `init()` resolves `false`; the cause is logged to the console (see [When init() Resolves false](/sdk/guides/error-handling#when-init-resolves-false)). - The URL above loads the latest release. In production, pin a version and add its `integrity` hash together with `crossorigin="anonymous"`. Without `crossorigin`, the browser blocks a cross-origin script that has an `integrity` hash: ```html title="Pinned version" ``` ## Applications A loader configuration can define **applications**: named presets in the `applications[]` array of `dcupl.lc.json` that pick resources by tag. Pass a key to `process({ applicationKey })` to load only that part of the project; [Load one application](/sdk/guides/load-from-console#load-one-application) shows the call and what an unknown key does, [Applications in the Loader API](/sdk/api/loader#applications) the full behaviour, and [Applications](/console/applications) how the Console uses them. ## Migrating from SDK-only to Console 1. Create a project in [Console](/console) 2. Define models in the visual editor (or import existing JSON) 3. Add data sources (URLs, file uploads) 4. Get your `projectId` and `apiKey` 5. Replace `dcupl.models.set()` and `dcupl.data.set()` with the Loader setup 6. **Query code stays the same**—no changes needed ## Next steps - [Your first console project](/console/getting-started) — Set up your first project - [Load from the console](/sdk/guides/load-from-console) — Project ID, Browser key, version and environments - [SDK Quick Start](/sdk/quick-start) — Learn the query API - [Core concepts](/sdk/core-concepts) — Deeper dive into models and lists --- # Quick reference Source: https://docs.dcupl.com/sdk/reference Description: One-page cheat sheet for the most common dcupl SDK calls # Quick reference The SDK calls you use most, on one page. Every option, return value and type is in the [API reference](/sdk/api). ## Install ```bash group="install" tab="npm" npm install @dcupl/core @dcupl/loader ``` ```bash group="install" tab="yarn" yarn add @dcupl/core @dcupl/loader ``` ```bash group="install" tab="pnpm" pnpm add @dcupl/core @dcupl/loader ``` `@dcupl/loader` is only needed to load a `dcupl.lc.json` configuration (see [Setup options](/sdk/setup-options)). ## Lifecycle ```typescript filename="lifecycle.ts" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); // options: /sdk/api/dcupl#dcuplinitoptions dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'brand', type: 'string' }, { key: 'price', type: 'int' }, ], }); dcupl.data.set( [ { key: 'p1', name: 'Laptop', brand: 'Acme', price: 999 }, { key: 'p2', name: 'Mouse', brand: 'Acme', price: 29 }, { key: 'p3', name: 'Desk', brand: 'Oak & Co', price: 299 }, ], { model: 'Product' }, ); // Builds the indexes. Resolves false, and logs the error, if processing fails. await dcupl.init(); ``` ## Load with the loader Instead of `models.set()` and `data.set()`, load a `dcupl.lc.json` and the files it lists: ```typescript filename="setup.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // The request url is `${baseUrl}/${loaderFileName}`. // baseUrl must be non-empty; an empty string falls back to the dcupl CDN default. await loader.config.fetch({ baseUrl: window.location.origin, // browser: your app's origin; in Node, pass a full URL loaderFileName: 'dcupl.lc.json', }); await loader.process(); // loads every resource the configuration lists await dcupl.init(); ``` ## Lists and queries ```typescript filename="query.ts" const list = dcupl.lists.create({ modelKey: 'Product' }); // Conditions on different attributes combine with AND, on the same attribute with OR list.catalog.query.addCondition({ attribute: 'brand', operator: 'eq', value: 'Acme' }); list.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 50 }); list.catalog.query.items(); // → Laptop // A group with an explicit groupType ('or' or 'and') list.catalog.query.clear(); list.catalog.query.addGroup({ groupKey: 'cheap-or-oak', groupType: 'or', queries: [ { attribute: 'brand', operator: 'eq', value: 'Oak & Co' }, { attribute: 'price', operator: 'lt', value: 50 }, ], }); list.catalog.query.items(); // → Desk, Mouse list.catalog.query.removeGroup('cheap-or-oak'); // Sort and paginate (order is 'ASC' or 'DESC') list.catalog.query.applyOptions({ sort: { attributes: ['price'], order: ['DESC'] }, start: 0, count: 2 }); list.catalog.query.items(); // → Laptop, Desk list.catalog.query.first(); // → Laptop list.catalog.query.count(); // items on the current page → 2 // Remove all conditions, sort and pagination list.catalog.query.clear(); ``` Operators: [Query Operators](/sdk/api/query-operators). Every query method: [Catalog API](/sdk/api/catalog). ## Analysis functions ```typescript filename="fn.ts" // Counts per value, for filter UIs list.catalog.fn.facets({ attribute: 'brand' }); // → [{ value: 'Acme', size: 2, ... }, { value: 'Oak & Co', size: 1, ... }] // sum, avg, count, min, max, distinct list.catalog.fn.aggregate({ attribute: 'price', types: ['min', 'max', 'avg'] }); // → { attribute: 'price', min: 29, max: 999, avg: 442.3333333333333, ... } // Items grouped by a value list.catalog.fn.groupBy({ attribute: 'brand' }); // → { items: [{ value: 'Acme', _meta: { size: 2 }, ... }, { value: 'Oak & Co', _meta: { size: 1 }, ... }], ... } // Autocomplete ('/…/' is a regex) list.catalog.fn.suggest({ attribute: 'name', value: '/^la/', regexFlags: 'i' }); // → [{ value: 'Laptop', keys: ['p1'] }] // All items matching the current query (count() is capped by the page size) list.catalog.fn.metadata().currentSize; // → 3 ``` Options and return types: [Functions API](/sdk/api/functions). ## Global queries Read a model without creating a list: ```typescript filename="global-query.ts" dcupl.query.one({ modelKey: 'Product', itemKey: 'p1' }); // → { key: 'p1', name: 'Laptop', brand: 'Acme', price: 999 } dcupl.query.execute({ modelKey: 'Product', queries: [ { attribute: 'brand', operator: 'eq', value: 'Acme' }, { attribute: 'price', operator: 'gte', value: 50 }, ], sort: { attributes: ['price'], order: ['ASC'] }, count: 20, }); // → [{ key: 'p1', name: 'Laptop', ... }] ``` ## Update data Data changes are staged until you call `dcupl.update()`; lists then return the new results. ```typescript filename="update.ts" // Change existing items; properties you leave out keep their values dcupl.data.update([{ key: 'p1', price: 899 }], { model: 'Product' }); // Add new items, or update them if they exist dcupl.data.upsert([{ key: 'p4', name: 'Lamp', brand: 'Oak & Co', price: 40 }], { model: 'Product' }); // Remove items by key dcupl.data.remove([{ key: 'p2' }], { model: 'Product' }); // Apply the staged changes. Resolves false, and logs the error, if the update fails. await dcupl.update(); list.catalog.query.items(); // → Laptop (899), Desk, Lamp ``` ## Events ```typescript filename="events.ts" // Every event of the instance: dcupl_initialized, dcupl_updated_manually, list_created, ... const off = dcupl.on((message) => console.log(message.type)); // Re-render when a list's result changes, e.g. after dcupl.update() const offList = list.onResultChange(() => render(list.catalog.query.items())); // Unsubscribe off(); offList(); ``` ## Clean up ```typescript filename="cleanup.ts" list.destroy(); // a list you no longer need dcupl.destroy(); // the whole instance, e.g. when your app unmounts ``` ## See also - [Quick start](/sdk/quick-start) - Get started in 5 minutes - [Core concepts](/sdk/core-concepts) - Understand the fundamentals - [Setup options](/sdk/setup-options) - Console + Loader, your own files, or SDK-only - [API reference](/sdk/api) - Every class, method and type --- # Compatibility Source: https://docs.dcupl.com/sdk/compatibility Description: The versions these docs describe, the Node.js and TypeScript versions dcupl needs, the browser targets and the package version rule # Compatibility These docs describe `@dcupl/core` 2.1 and `@dcupl/cli` 2.0. This page collects the requirements that change with a release; every other page links here instead of repeating them. ## Requirements | Requirement | Supported | | ----------- | ----------------------------------------------------------------------------------------------- | | Node.js | 20.19+ or 22.12+, or any later major (check with `node -v`) | | TypeScript | 5.4 or later | | Browsers | Current evergreen browsers (Chrome, Edge, Firefox, Safari): the packages ship ES2022 JavaScript | ### Node.js The SDK, the CLI and the tutorials' Vite setups are documented for Node.js **20.19+ on Node 20, or 22.12+ on Node 22**; later majors work too. On an older version the CLI installs but every command fails on startup, see [Node version too old](/cli/installation#node-version-too-old). The loader uses the global `fetch` that Node.js provides. ### TypeScript The public types use `NoInfer`, which TypeScript added in 5.4. An older compiler can't resolve `NoInfer` in the `@dcupl/*` type declarations. The packages are tested with `"module": "NodeNext"` and `"strict": true`; see [TypeScript](/sdk/guides/typescript) for the setup. ### Browsers and servers The packages are compiled to ES2022 and published as ES modules and CommonJS, so they work with `import` and `require()`, with any bundler, and in current evergreen browsers. Browsers without ES2022 support need your bundler to transpile `@dcupl/*` to an older target. For a page without a build step, the `@dcupl/bundle` script build is described under [Script tag](/sdk/setup-options#script-tag-no-build-step). The same packages run on a server (Node.js, SSR); see [Node.js server and SSR](/sdk/guides/deployment#nodejs-server-and-ssr). ## Use the same version for every @dcupl/\* package Use the same version for every `@dcupl/*` package. The packages require each other in exactly their own version: `@dcupl/core` depends on exactly its own version of `@dcupl/common`, and `@dcupl/loader` and `@dcupl/connect` on exactly their own versions of `@dcupl/core` and `@dcupl/common`. Install and update them together: ```bash npm install @dcupl/core@ @dcupl/loader@ ``` If you list `@dcupl/common` yourself, pin it to the same version. [Common mistakes](/sdk/guides/common-mistakes#version-mismatch) shows the symptom of a mismatch. Changes between major versions are in the [migration guide](/sdk/guides/migration); for the CLI see [Upgrading from 1.3](/cli/installation#upgrading-from-13). ## Next steps - [Quick start](/sdk/quick-start) — Install the SDK and run a first query - [Setup options](/sdk/setup-options) — Console + Loader, your own files, or the SDK alone - [TypeScript](/sdk/guides/typescript) — Types for models, lists and queries - [CLI installation](/cli/installation) — Install the CLI and check the Node.js version --- # Core concepts Source: https://docs.dcupl.com/sdk/core-concepts Description: Understanding the fundamentals of the dcupl SDK # Core concepts Before diving into code, let's understand the key concepts that make dcupl work. This 5-minute overview will give you the mental model you need to be productive. ## What is the dcupl core SDK? The SDK is an **in-memory query engine** that runs in your browser or Node.js environment, a mini-database for your application: data loads once, and filtering, sorting and facets run locally without a network round trip. Why that matters and where it does not fit is on the [Introduction](/introduction). ## The three core concepts | Concept | What it is | | --------- | --------------------------------------------------------------- | | **Model** | A schema defining your data structure (properties + references) | | **List** | A queryable view of items from a model | | **Query** | The interface for filtering, sorting, and analyzing data | ```mermaid flowchart LR M[Model] --> L[List] L --> Q[Query] Q --> R[Results] ``` ### 1. Models **Models** define the structure of your data - like database tables or TypeScript interfaces. They include **properties** (fields) and **references** (relationships). ```typescript import type { ModelDefinition } from '@dcupl/common'; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string' }, ], references: [{ key: 'brand', type: 'singleValued', model: 'Brand' }], }; ``` **Think of it as:** The blueprint for your data **Learn more:** [Models](/sdk/core-concepts/models) ### 2. Lists **Lists** are filtered views of your data. You create a list to query a specific model. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); ``` **Think of it as:** A live view of your data that updates when filters change **Learn more:** [Lists](/sdk/core-concepts/lists) ### 3. Query **Query** is the interface attached to each list for filtering, sorting, and retrieving data. ```typescript productList.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 100 }); // Filter productList.catalog.query.items(); // Get results productList.catalog.fn.facets({ attribute: 'category' }); // Get facets ``` **Think of it as:** The SQL-like query language for your data **Learn more:** [Query](/sdk/core-concepts/query-overview) ## Architecture overview The dcupl SDK consists of modular packages that work together: ```mermaid flowchart TB subgraph Packages["SDK Packages"] direction TB CORE["@dcupl/core
Query engine, models, lists"] LOADER["@dcupl/loader
Config & data loading"] COMMON["@dcupl/common
Shared types & utilities"] end subgraph DataSources["Data Sources"] direction TB CONSOLE["dcupl Console
Managed backend"] LOCAL["Local Files
JSON, CSV"] API["Custom APIs
JSON or CSV over HTTP"] end subgraph Runtime["Runtime"] direction TB MODELS["Models & Data"] LISTS["Lists"] QUERY["Query API"] RESULTS["Query results"] end DataSources --> LOADER LOADER --> CORE COMMON -.-> CORE COMMON -.-> LOADER CORE --> MODELS MODELS --> LISTS LISTS --> QUERY QUERY --> RESULTS ``` ### Package responsibilities | Package | Purpose | | --------------- | ------------------------------------------------------ | | `@dcupl/core` | The query engine - models, lists, query API | | `@dcupl/loader` | Configuration fetching, resource loading, transformers | | `@dcupl/common` | Shared TypeScript types and utilities | ## The data flow Here's how data moves through the system: ```mermaid flowchart LR subgraph Setup["1. Setup"] direction TB DEF["Define Models"] LOAD["Load Data"] end subgraph Init["2. Initialize"] INIT["dcupl.init()"] end subgraph Query["3. Query"] direction TB LIST["Create List"] FILTER["Add Filters"] EXEC["Execute Query"] end subgraph Output["4. Results"] direction TB ITEMS["Items"] FACETS["Facets"] AGGS["Aggregations"] end Setup --> Init --> Query --> Output ``` **Step by step:** 1. **Define Models** - Create schemas for your data 2. **Load Data** - Add data inline or via DcuplAppLoader 3. **Initialize** - Call `dcupl.init()` to process models and data 4. **Create Lists** - Create filtered views of your models 5. **Query** - Use query API to filter, sort, analyze 6. **Get Results** - Items, facets, aggregations, pivot tables ## Simple example Let's see all three concepts in action: ```typescript import { Dcupl } from '@dcupl/core'; // Create instance const dcupl = new Dcupl(); // 1. Define MODEL with properties dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ], data: [ { key: 'p1', name: 'Laptop', price: 999 }, { key: 'p2', name: 'Mouse', price: 29 }, ], }); // Initialize await dcupl.init(); // 2. Create LIST const list = dcupl.lists.create({ modelKey: 'Product' }); // 3. QUERY via catalog list.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 100, }); const results = list.catalog.query.items(); // Results: [{ key: 'p1', name: 'Laptop', price: 999 }] ``` ## Key takeaways 1. **Models** define structure (properties + references) 2. **Lists** are filtered views of data 3. **Query** is how you filter and analyze data 4. You can have multiple lists for the same model 5. Everything starts with `dcupl.init()` and ends with `list.catalog.query` ## Next steps - [Models](/sdk/core-concepts/models) — Learn about model configuration, properties, and references - [Lists](/sdk/core-concepts/lists) — Master list management and events - [Query](/sdk/core-concepts/query-overview) — Learn the query structure: conditions, groups, and options ## Explore patterns - [Faceted search](/sdk/advanced/faceted-search) — Build filter UIs with facets - [Master-detail](/sdk/advanced/master-detail) — Navigate related data - [Real-Time Updates](/sdk/advanced/realtime-updates) — Live data synchronization --- # Models Source: https://docs.dcupl.com/sdk/core-concepts/models Description: Define the structure and schema of your data with properties, types, and references. # Models Define the structure and schema of your data with properties, types, and references. ## What is a model? A `ModelDefinition` is a configuration object that describes the structure and schema of your data. - **Properties**: the fields in your data structure - **Data types**: the type of each property (string, int, boolean…) - **References**: relationships to other models - **Data**: optional inline data Think of models as **blueprints** or **table schemas** for your data. ```typescript filename="product.model.ts" import type { ModelDefinition } from '@dcupl/common'; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'sku', type: 'string', index: true }, { key: 'price', type: 'float' }, { key: 'tags', type: 'Array' }, { key: 'inStock', type: 'boolean' }, ], references: [ { key: 'category', type: 'singleValued', model: 'Category' }, ], }; ``` *Product — 5 properties · 1 reference · what this model holds once data is loaded — Demo data* | sku , string | name , string | price , float | tags , Array | inStock , boolean | category , → Category | | --- | --- | --- | --- | --- | --- | | SKU-1001 | NYM-J cable 3x1.5, 50 m | 42.9 | indoor, 3-core | true | cable | | SKU-2001 | Rocker switch, white | 6.9 | flush | true | switch | | SKU-2002 | LED dimmer switch, white | 48 | led, flush | false | switch | | SKU-3001 | LED panel 620×620, 36 W | 89.5 | led, ceiling | true | lighting | Point at a property in the code to find its column. category holds the key of a Category item. ## Property types dcupl supports these data types: | Type | Description | Example | | --- | --- | --- | | `string` | Text values | `"Laptop"` | | `int` | Whole numbers | `42` | | `float` | Decimal numbers | `99.99` | | `boolean` | True/false | `true` | | `date` | Date/time values | `"2025-01-15"` | | `json` | Complex nested objects | `{ color: 'red' }` | | `any` | Any value, stored as is | `42`, `'a'` | | `Array`, `Array`, `Array`, `Array` | Typed arrays | `['red', 'blue']` | A `boolean` property also reads the strings `'true'`/`'false'`/`'1'`/`'0'` (any case) and the numbers `1`/`0`. Any other string or number, and any object or array, is a `WrongDataType` quality error, and the property stays `undefined`. With `quality.forceStrictDataType`, only `true`/`false` pass. There is no plain `array` type. A property with any other `type` is rejected with an `InvalidModelDefinition` quality error ("Property type is invalid"), and its values are not loaded. ```typescript filename="property-types.ts" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'rating', type: 'float' }, { key: 'inStock', type: 'boolean' }, { key: 'createdAt', type: 'date' }, { key: 'metadata', type: 'json' }, { key: 'tags', type: 'Array' }, ], }; ``` ## Property options Properties support additional options for indexing, computed values, and validation. #### index Queries and facets work without it: dcupl builds a property's lookup index on the first query that needs it. `index: true` builds it at load time instead, so use it for attributes you filter on often. #### derive Create computed properties from references. Pull values from referenced models automatically. #### expression Template-based computed properties using `${property}` placeholders. The result is always a string. #### quality Data validation rules: required fields and built-in validators (`email`, `pattern`, `min`/`max`, `enum`, `unique`, …). More options (`filter`, `aggregate`, `origin`, `separator`, `inputFormat` and `UTC` for dates, `fractionDigits`) are in the [Models API reference](/sdk/api/models#property). ```typescript filename="property-options.ts" const orderModel: ModelDefinition = { key: 'Order', properties: [ // Index built at load time instead of on first query { key: 'orderId', type: 'string', index: true }, // Derived from reference { key: 'customerName', type: 'string', derive: { localReference: 'customer', remoteProperty: 'name', }, }, // Template expression { key: 'displayText', type: 'string', expression: 'Order #${orderId} - ${customerName}', }, // Validation rules { key: 'email', type: 'string', quality: { required: true, validators: { email: {} }, }, }, ], references: [ { key: 'customer', type: 'singleValued', model: 'Customer' }, ], }; ``` ## References References link one model to another, enabling: - Navigate between related data - Query across relationships (deep queries) - Build complex data structures - Maintain data normalization #### Single-valued (many-to-one) Each item references **one** other item. Use for: Order → Customer, Product → Category. #### Multi-valued (to many) Each item holds a **list** of keys; the same target can be referenced by many items. Use for: Order → Products, User → Roles. References can also be computed (`resolve`, `derive`, `groupBy`, `query`) or check their keys (`validity`): see [Reference](/sdk/api/models#reference) in the Models API. ```typescript filename="references.ts" const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'total', type: 'int' }, ], references: [ // Single-valued: one customer per order { key: 'customer', type: 'singleValued', model: 'Customer', }, // Multi-valued: many products per order { key: 'products', type: 'multiValued', model: 'Product', }, ], }; // Data format: // { key: 'o1', total: 150, customer: 'c1', products: ['p1', 'p2'] } ``` ## Resolved references Create the **inverse** of another reference automatically using `resolve`. Instead of manually maintaining both directions of a relationship, dcupl can compute the inverse for you. Example: If Order → Customer exists, you can automatically get Customer → Orders. The resolved reference is computed at initialization time and stays in sync with your data. ```typescript filename="resolved-refs.ts" // Customer model with inverse reference const customerModel: ModelDefinition = { key: 'Customer', properties: [ { key: 'name', type: 'string' }, { key: 'email', type: 'string' }, ], references: [ { key: 'orders', type: 'multiValued', model: 'Order', // Automatically resolve from Order.customer resolve: { reference: 'customer', model: 'Order' }, }, ], }; // Now customer.orders contains all orders // for that customer automatically ``` ## Model options Models support additional configuration options: | Option | Description | | --- | --- | | `keyProperty` | Use a custom property as item identifier (default: `'key'`). An array builds a composite key: `['ID', 'Language']` gives items keys like `'A1::en'` | | `keyPropertySeparator` | Separator joining the parts of a composite key (default: `'::'`) | | `autoGenerateKey` | Generate unique keys for items without one | | `autoGenerateProperties` | Infer properties from the data: `true` reads the first row and checks number columns against every row, `{ deep: true }` all rows, `{ sampleSize: n }` up to n rows. String values like `"42"` or `"true"` are typed as numbers, booleans, or dates; zero-padded values like `"007"` stay strings. | | `supportsAutoCreation` | Auto-create an item of this model when another model references a key it doesn't have | | `valueMappings` | Transform values during data loading | | `meta` | Attach metadata to the model | | `quality` | Configure validation rules | ```typescript filename="model-options.ts" const productModel: ModelDefinition = { key: 'Product', // Use 'sku' as identifier instead of 'key' keyProperty: 'sku', // Require explicit keys autoGenerateKey: false, // Don't auto-create a Product when another model // references a Product key that doesn't exist supportsAutoCreation: false, // Attach metadata meta: { name: 'Products', icon: 'package', }, properties: [ { key: 'sku', type: 'string' }, { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ], }; ``` ## Quality Define **quality criteria** to validate your data at both model and attribute level. #### Model-level quality Set defaults for the quality flags of all attributes of a model in `quality.attributes`. Validators can only be set per attribute. `quality.enabled: false` turns the checks off for the model. #### Attribute-level quality Override or extend quality rules for specific properties: - `required` - Property must have a value. Defaults to `true`, so a missing value is reported unless you set `required: false` - `nullable` - Allow `null` (default `false`) - `forceStrictDataType` - Report and drop wrong-typed values instead of converting them (default `false`) - `validators` - Validation rules: `email`, `pattern`, `startsWith`, `endsWith`, `includes`, `enum`, `min`/`max`, `minLength`/`maxLength`, `unique`. They run on `string`, `int` and `float` properties - `validatorHandling` - `'loose'` (default) reports a failed validator and keeps the value, `'strict'` also drops it Quality issues are reported during data loading. Read the totals with `dcupl.quality.errors.summary()` and the individual errors by querying the `$DcuplErrorTrackingErrors` model. ```typescript filename="quality.ts" const userModel: ModelDefinition = { key: 'User', // Model-level quality: default rules for all attributes quality: { attributes: { required: false }, }, properties: [ // Override: make this property required { key: 'email', type: 'string', quality: { required: true, validators: { email: {} }, }, }, // Regex validation { key: 'phone', type: 'string', quality: { validators: { pattern: { value: '^\\+?[0-9]{10,14}$' }, }, }, }, // Prefix validation { key: 'website', type: 'string', quality: { validators: { startsWith: { value: 'https://' } }, }, }, // No quality override - uses model defaults { key: 'nickname', type: 'string' }, ], }; ``` ## Meta Attach **custom metadata** to models and attributes for documentation, tooling, or frontend logic. #### Model meta Describe the model itself: - Display names and descriptions - Data source information - Icons or UI hints #### Attribute meta Describe individual properties: - Labels for forms or tables - Tooltips and help text - Formatting hints (currency, percentage) - Source system field mappings Meta is a free-form object with a required `name` and an optional `description` - structure the rest however fits your needs. ```typescript filename="meta.ts" const productModel: ModelDefinition = { key: 'Product', // Model-level metadata meta: { name: 'Products', description: 'Product catalog from ERP system', source: { system: 'SAP', table: 'MARA', refreshInterval: '15min', }, ui: { icon: 'package', color: '#3b82f6', }, }, properties: [ { key: 'name', type: 'string', // Attribute-level metadata meta: { name: 'Product Name', description: 'Official product name from catalog', searchable: true, }, }, { key: 'price', type: 'float', meta: { name: 'Price', format: 'currency', currency: 'EUR', source: { field: 'NETPR' }, }, }, { key: 'rating', type: 'float', meta: { name: 'Customer Rating', format: 'stars', min: 0, max: 5, }, }, ], }; ``` ## Best practices - **Use clear, descriptive keys** - `Product`, `CustomerOrder` instead of `Item`, `Data` - **Keep models focused** - Each model should represent a single entity - **Use specific types** - `float` for prices, `int` for counts, `boolean` for flags, not `string` - **Use typed arrays** - `Array`, `Array`, … (there is no untyped `array` type) - **Index hot filter attributes** - `index: true` moves the index build from the first query to load time; every index costs memory - **Choose the right reference type** - Single-valued for one key per item, multi-valued for a list of keys - **Limit reference depth** - Deep queries (3+ levels) add overhead ```typescript filename="best-practices.ts" // Good: Clear types, an index for a hot filter attribute const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'sku', type: 'string', index: true }, { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'rating', type: 'float' }, { key: 'inStock', type: 'boolean' }, { key: 'tags', type: 'Array' }, ], references: [ { key: 'category', type: 'singleValued', model: 'Category' }, ], }; // Avoid: String types for everything // { key: 'price', type: 'string' } // Bad // { key: 'count', type: 'string' } // Bad ``` ## Next steps - [Models API](/sdk/api/models) — Full reference: every model, property and reference option - [Lists](/sdk/core-concepts/lists) — Create filtered views of your models - [Query overview](/sdk/core-concepts/query-overview) — Learn the query interface - [Derived properties](/sdk/advanced/derived-properties) — Compute values from references --- # Lists Source: https://docs.dcupl.com/sdk/core-concepts/lists Description: Filtered views of your data - create, query, and manage independent data views. # Lists Filtered views of your data - create, query, and manage independent data views. ## What is a list? A `DcuplList` is a **filtered view** of your model data. Think of it as a live, queryable dataset: filter changes apply immediately, and data changes show up after `dcupl.update()`. Each list: - Represents a specific model's data - Provides a query interface (via catalog) - Can have filters applied to it - Recomputes its result when filters change or `dcupl.update()` applies data changes - Manages its own state independently ```typescript filename="create-list.ts" const productList = dcupl.lists.create({ modelKey: 'Product' }); // Access items through the catalog const items = productList.catalog.query.items(); ``` ## Why lists? #### Multiple views of same data Create multiple lists for the same model, each with different filters. Great for dashboards, comparison views, or different user contexts. #### Independent state Each list maintains its own query filters, sort order, and cache. Filtering one list doesn't affect others. #### Scoped lists A list created with a `query` only contains the items that match it. Its own filters start empty and work on top of that subset. ```typescript filename="multiple-lists.ts" // Multiple views of the same Product model const allProducts = dcupl.lists.create({ modelKey: 'Product' }); // Scoped list: only contains Electronics products const electronics = dcupl.lists.create({ modelKey: 'Product', query: { modelKey: 'Product', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' } ], }, }); const inStock = dcupl.lists.create({ modelKey: 'Product', query: { modelKey: 'Product', queries: [ { attribute: 'inStock', operator: 'eq', value: true } ], }, }); ``` ## Creating lists Lists can be created with various options: | Option | Description | | --- | --- | | `modelKey` | **Required.** The model to create a view of | | `listKey` | Named identifier for later retrieval | | `query` | Seed query. Makes a **scoped** list: it contains only the items that match the query when the list is created. The list's own filters start empty, and `clear()` does not remove the scope. | | `listItemKeys` | Scope the list to these item keys instead of a query. It always serves the current items for exactly these keys, with or without `autoUpdate` | | `autoUpdate` | Lists with a seed `query` only: re-run it on every `dcupl.update()` that affects the list, so new matches join and items that stop matching leave. A `listItemKeys` list serves the same items either way (with it, the list also emits `list_updated` on every update that affects it) | A list created with `listItemKeys` or `query` is scoped: it keeps a set of keys (the `listItemKeys` you pass, or the keys the seed query matches when the list is created) and always serves the model's current items for those keys. After every `dcupl.update()`, also after a full rebuild (`data.set()`, `data.reset()` or a large update), changed values show, removed items drop out, and an item added under one of those keys is served. For a `listItemKeys` list that is all there is, so `autoUpdate` changes only the events, not the items. Items that start or stop matching a seed query join or leave only when you call `list.update()` or enable auto-update. Lists without a scope see every data change after `dcupl.update()`: affected lists drop their cached results and recompute on the next read. Set auto-update for all lists with `new Dcupl({ performance: { autoUpdateLists: { enabled: true } } })`, or per call with `dcupl.update({ autoUpdateLists: true })`. The full behaviour is in [create()](/sdk/api/lists#create). `autoUpdate` on `lists.create()` works at runtime, but the TypeScript type for `create()` does not list it. ```typescript filename="creating-lists.ts" // Basic creation const allProducts = dcupl.lists.create({ modelKey: 'Product' }); // With named key for later retrieval const featured = dcupl.lists.create({ modelKey: 'Product', listKey: 'featured-products', }); const sameList = dcupl.lists.get('featured-products'); // Scoped list: only items matching the seed query const electronics = dcupl.lists.create({ modelKey: 'Product', query: { modelKey: 'Product', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' } ], }, }); // Scoped list from item keys const picked = dcupl.lists.create({ modelKey: 'Product', listItemKeys: ['p1', 'p2'], }); // A new Electronics product... dcupl.data.upsert( [{ key: 'p9', name: 'Phone', category: 'Electronics' }], { model: 'Product' } ); await dcupl.update(); // ...shows up in allProducts right away. To add it to the // scoped electronics list, re-run its seed query: electronics.update(); // (or apply the update with dcupl.update({ autoUpdateLists: true })) ``` ## Subscribe to changes Lists emit events when their state changes. Subscribe to react to these changes. #### When to update the UI Use `list.onResultChange()`. It fires when the list's result can have changed: - **Filter changes** - `addCondition`, `addGroup`, `clear`, ... - **Data changes** - once per `dcupl.update()` that affects the list Reads such as `items()` or `fn.facets()` never trigger it. It returns a function that removes the subscription. #### All events `list.on()` receives every message of the list, including reads. Each message has a `type`, for example: | Type | When | | --- | --- | | `result_updated` | The result can have changed (what `onResultChange` listens to) | | `query_apply`, `query_remove`, `query_reset` | The list's query changed | | `query_execute`, `fn_facets`, ... | A read ran | | `list_updated` | `list.update()` was called | ```typescript filename="subscribe-changes.ts" const list = dcupl.lists.create({ modelKey: 'Product' }); // renderProducts: your own render function const unsubscribe = list.onResultChange(() => { renderProducts(list.catalog.query.items()); }); // Later: cleanup unsubscribe(); // Every event of the list const off = list.on((event) => { console.log(event.type, event.listKey); }); // React example (setProducts from useState) useEffect(() => { const list = dcupl.lists.create({ modelKey: 'Product' }); const unsubscribe = list.onResultChange(() => { setProducts(list.catalog.query.items()); }); // Set initial data setProducts(list.catalog.query.items()); return () => { unsubscribe(); list.destroy(); }; }, []); ``` ## List lifecycle A list's life is: **create** (`lists.create()`) → **query** (filters and reads, any number of times) → **data updates** (each `dcupl.update()` refreshes the affected lists) → **destroy** (`list.destroy()`). A list has no state property to check; it is usable from `create()` until `destroy()`. Always destroy lists when done to prevent memory leaks. ```typescript filename="lifecycle.ts" // Create const list = dcupl.lists.create({ modelKey: 'Product' }); // Use - apply filters list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); // Get results const items = list.catalog.query.items(); // Apply data changes - the list's next read sees them dcupl.data.upsert([{ key: 'p9', name: 'Tablet', category: 'Electronics' }], { model: 'Product' }); await dcupl.update(); // Destroy when done list.destroy(); ``` ## Managing lists dcupl provides methods to manage all your lists: - `get(key)` - Retrieve a list by its key - `getAll()` - Get all lists - `getAll({ modelKey })` - Get lists for a specific model - `destroy({ listKey })` - Destroy a specific list - `destroy({ modelKey })` - Destroy all lists of a model; `destroy()` destroys every list ```typescript filename="managing-lists.ts" // Get list by key const list = dcupl.lists.get('my-list'); // Get all lists const allLists = dcupl.lists.getAll(); // Get lists for a specific model const productLists = dcupl.lists.getAll({ modelKey: 'Product' }); // Destroy specific list dcupl.lists.destroy({ listKey: 'my-list' }); // Or destroy via list instance list.destroy(); ``` ## Best practices #### Use named lists for retrieval Give lists a `listKey` so you can retrieve them anywhere in your app without passing references. #### Destroy lists in component cleanup Always destroy lists when components unmount to prevent memory leaks. #### Reuse lists instead of creating new ones Change filters on an existing list rather than creating new lists for each filter change. ```typescript filename="best-practices.ts" // Named lists for easy retrieval const mainList = dcupl.lists.create({ modelKey: 'Product', listKey: 'main-product-list', }); // Retrieve anywhere const sameList = dcupl.lists.get('main-product-list'); // Cleanup in React useEffect(() => { const list = dcupl.lists.create({ modelKey: 'Product' }); return () => list.destroy(); }, []); // Reuse lists - change filters, don't recreate const filterList = dcupl.lists.create({ modelKey: 'Product' }); function filterByCategory(category: string) { filterList.catalog.query.clear(); filterList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category }); return filterList.catalog.query.items(); } ``` ## Next steps - [Query](/sdk/core-concepts/query-overview) — Master the query interface - [Querying](/sdk/queries/query-filter) — Learn to filter list data - [Data management](/sdk/core-concepts/data-management) — Update list data - [Faceted search](/sdk/advanced/faceted-search) — Real-world list examples --- # Query overview Source: https://docs.dcupl.com/sdk/core-concepts/query-overview Description: Understanding the dcupl query structure, groups, and conditions for building complex queries. # Query overview Understanding the dcupl query structure, groups, and conditions for building complex queries. ## QueryExecutionContext A query is a tree of conditions and groups of conditions. You run one directly with `dcupl.query.execute(context)`, pass one as the seed `query` of `dcupl.lists.create()`, or build one step by step on a list with `list.catalog.query` (`list.catalog.query.current()` returns it). In every case the query is a `QueryExecutionContext`. `QueryExecutionContext` is the root container for executing queries. It holds: - **modelKey** - Target model to query - **queries** - Array of conditions and groups (the query tree) - **groupType** - How to combine queries: `'and'` or `'or'` - **Item options** - Pagination, projection, sorting Think of it as the **complete specification** for a query operation. ```typescript filename="query-context.ts" import type { QueryExecutionContext } from '@dcupl/common'; // The item type used by the examples on this page type Product = { key: string; name: string; status: string; category: string; price: number }; // Basic structure const context: QueryExecutionContext = { modelKey: 'Product', groupType: 'and', // Combine with AND (default) queries: [ // QueryCondition or QueryConditionGroup items ], // Optional: pagination, projection, sort start: 0, count: 50, }; ``` ## QueryCondition A `QueryCondition` is a single filter criterion: - **attribute** - Property to filter on - **operator** - How to compare (`eq`, `find`, `gte`, etc.) - **value** - Value to compare against - **options** - Optional: `transform`, `invert`, `arrayValueHandling`, `regexFlags` (for `find`) See [Operators Reference](/sdk/api/query-operators) for all available operators. ```typescript filename="query-condition.ts" import type { QueryCondition } from '@dcupl/common'; // Exact match const exact: QueryCondition = { attribute: 'status', operator: 'eq', value: 'active', }; // Text search: '/term/' matches anywhere in the value, // a bare string must match the whole value const search: QueryCondition = { attribute: 'name', operator: 'find', value: '/laptop/', options: { regexFlags: 'i' }, // case-insensitive }; // Numeric range const minPrice: QueryCondition = { attribute: 'price', operator: 'gte', value: 100, }; // With options const caseInsensitive: QueryCondition = { attribute: 'category', operator: 'eq', value: 'electronics', options: { transform: ['lowercase'] }, // compare lowercased values }; ``` ## QueryConditionGroup A `QueryConditionGroup` combines multiple conditions with boolean logic: - **groupType** - `'and'` or `'or'` - **queries** - Array of conditions (or nested groups) #### AND group All conditions must match. Use for narrowing results. #### OR group Any condition can match. Use for broadening results. ```typescript filename="query-groups.ts" import type { QueryConditionGroup } from '@dcupl/common'; // AND group: active AND in stock const andGroup: QueryConditionGroup = { groupType: 'and', queries: [ { attribute: 'status', operator: 'eq', value: 'active' }, { attribute: 'inStock', operator: 'eq', value: true }, ], }; // OR group: category is Electronics OR Computers const orGroup: QueryConditionGroup = { groupType: 'or', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Computers' }, ], }; ``` ## Nesting groups Groups can contain other groups, enabling complex boolean expressions: `(A AND B) OR (C AND D)` The recursive structure allows arbitrary nesting depth for any boolean logic you need. Structure: ```text QueryExecutionContext └── queries[] ├── QueryCondition └── QueryConditionGroup └── queries[] ├── QueryCondition └── QueryConditionGroup (recursive) ``` ```typescript filename="nested-groups.ts" // Complex: (premium AND active) OR (featured AND sale) // (Product: see the first example) const context: QueryExecutionContext = { modelKey: 'Product', groupType: 'or', // Top level OR queries: [ // First group: premium AND active { groupType: 'and', queries: [ { attribute: 'tier', operator: 'eq', value: 'premium' }, { attribute: 'status', operator: 'eq', value: 'active' }, ], }, // Second group: featured AND sale { groupType: 'and', queries: [ { attribute: 'featured', operator: 'eq', value: true }, { attribute: 'onSale', operator: 'eq', value: true }, ], }, ], }; ``` ## Item options `QueryExecutionContext` includes the item options (`ItemDataOptions` in `@dcupl/common`): | Option | Description | | --- | --- | | `start` | Skip N items (pagination offset) | | `count` | Limit results to N items | | `projection` | Select specific properties to return, e.g. `{ $: false, name: true }` | | `sort` | Order results: `{ attributes, order }` with `'ASC'` or `'DESC'` per attribute | ```typescript filename="item-options.ts" // (Product: see the first example) const context: QueryExecutionContext = { modelKey: 'Product', queries: [ { attribute: 'status', operator: 'eq', value: 'active' }, ], // Pagination: page 2, 20 items per page start: 20, count: 20, // Only return these fields (key is always included) projection: { $: false, name: true, price: true }, // Sort by price descending sort: { attributes: ['price'], order: ['DESC'] }, }; ``` ## Building queries Real-world example: E-commerce product filter Requirements: - Active products only - Category: Electronics OR Computers - Price between $100 and $1000 - Sort by price, paginated The context object fully describes this complex filter in a declarative, serializable format. Conditions placed directly in an `'and'` group must all match, so the two price conditions form a range. ```typescript filename="building-queries.ts" // dcupl: an initialized instance with a Product model (Product: see the first example) const productFilter: QueryExecutionContext = { modelKey: 'Product', groupType: 'and', queries: [ // Must be active { attribute: 'status', operator: 'eq', value: 'active' }, // Category: Electronics OR Computers { groupType: 'or', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Computers' }, ], }, // Price range: $100 - $1000 { attribute: 'price', operator: 'gte', value: 100 }, { attribute: 'price', operator: 'lte', value: 1000 }, ], // Sort and paginate sort: { attributes: ['price'], order: ['ASC'] }, start: 0, count: 24, }; // Execute against the model const results = dcupl.query.execute(productFilter); ``` ## In action On a list you rarely write the context yourself: `list.catalog.query` keeps the tree, and `addCondition`, `addGroup`, `setCondition` and `setGroup` change it. [Methods & Operators](/sdk/queries/methods-operators) explains each method; the [Catalog API](/sdk/api/catalog#query-operations) lists them all. ```typescript filename="in-action.ts" const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active' }); // The list keeps its query as a QueryExecutionContext const context = list.catalog.query.current(); // { modelKey: 'Product', groupKey: 'root', groupType: 'and', // queries: [{ groupKey: 'status', groupType: 'or', queries: [{ attribute: 'status', ... }] }] } // Run the same context without a list const sameItems = dcupl.query.execute(context); ``` ## Next steps - [Methods & Operators](/sdk/queries/methods-operators) — Build and change a list query step by step - [Operators](/sdk/api/query-operators) — Complete reference of all filter operators - [Deep queries](/sdk/queries/query-deep) — Query across model relationships - [Lists](/sdk/core-concepts/lists) — Create filtered views of your models --- # Data management Source: https://docs.dcupl.com/sdk/core-concepts/data-management Description: Loading, updating, and managing your data with dcupl's data API. # Data management Loading, updating, and managing your data with dcupl's data API. ## Data API overview All data operations are accessed through `dcupl.data`: - `set()` - Replace all data for a model - `update()` - Incremental updates to existing items - `upsert()` - Insert or update items - `remove()` - Delete specific items - `reset()` - Remove all items of a model - `apply()` - Stage a whole `DataContainer` (`{ model, type, data }`, `type` one of `'set'`, `'update'`, `'upsert'`, `'remove'`), for example one you built from a sync message Data operations are staged. They take effect when you call `dcupl.init()` to initialize dcupl, or `dcupl.update()` to process changes after initialization. ```typescript filename="data-api.ts" // Data operations overview: each call is staged dcupl.data.set([{ key: 'p1', name: 'Laptop', price: 999 }], { model: 'Product' }); // Replace all data dcupl.data.update([{ key: 'p1', price: 899 }], { model: 'Product' }); // Update existing items dcupl.data.upsert([{ key: 'p2', name: 'Mouse', price: 29 }], { model: 'Product' }); // Insert or update dcupl.data.remove([{ key: 'p2' }], { model: 'Product' }); // Delete items dcupl.data.reset({ model: 'Product' }); // Remove all items of a model dcupl.data.apply({ model: 'Product', type: 'upsert', data: [{ key: 'p3', name: 'Desk' }] }); // A whole container // Process the staged changes (dcupl.init() the first time) await dcupl.update(); ``` ## set() - replace all data **Use when:** You want to completely replace all data for a model. This method clears existing data and replaces it with the provided items. Useful for initial data loading or full refreshes. ```typescript filename="set-data.ts" dcupl.data.set( [ { key: 'p1', name: 'Laptop', price: 999 }, { key: 'p2', name: 'Mouse', price: 29 }, ], { model: 'Product' } ); await dcupl.update(); ``` ## update() - incremental update **Use when:** You want to modify existing items. Items whose key does not exist yet are ignored, also in a model that has no data yet: `update()` never inserts rows. Use `upsert()` to add them (see [update()](/sdk/api/data#update)). Only include changed fields for better performance. Each field in the payload replaces the stored value as a whole: arrays and `json` objects are replaced, not merged. Fields you leave out (or set to `undefined`) keep their value; send `null` to clear a field. ```typescript filename="update-data.ts" dcupl.data.update( [ { key: 'p1', price: 899 }, // Only changed fields { key: 'p2', tags: ['sale'] }, // Replaces the whole tags array ], { model: 'Product' } ); await dcupl.update(); ``` ## upsert() - insert or update **Use when:** You don't know if items exist. If an item with the key exists, it's updated. If not, it's created. Combines the behavior of insert and update. ```typescript filename="upsert-data.ts" dcupl.data.upsert( [ { key: 'p1', price: 899 }, // Updates if exists { key: 'p99', name: 'Chair' }, // Creates if doesn't exist ], { model: 'Product' } ); await dcupl.update(); ``` ## remove() - delete items **Use when:** You want to delete specific items. Provide items with at least the key property to identify which items to remove. ```typescript filename="remove-data.ts" dcupl.data.remove( [{ key: 'p1' }, { key: 'p2' }], { model: 'Product' } ); await dcupl.update(); ``` ## Data options `set()`, `update()`, `upsert()` and `remove()` take a [`DataOptions`](/sdk/api/data#dataoptions) object; `reset()` takes only `{ model }`. An option you leave out falls back to the model's setting: | Option | Description | | --- | --- | | `model` | **Required.** Specify which model the data belongs to | | `keyProperty` | Override the primary key field (default: the model's, else `'key'`). An array builds a composite key. A row with no value for it (for a composite key: no value for any of its parts) is dropped: `set()` and `upsert()` record it as a `MissingKey` quality error (unless `autoGenerateKey` gives it a key, which happens only with the default `'key'`), `update()` and `remove()` skip it. An `upsert()` after `init()` that `dcupl.update()` applies on the [partial path](/sdk/api/data#partial-updates) currently drops the row without the error; the next full rebuild (`init()`, or an `update()` that rebuilds) records it | | `keyPropertySeparator` | Separator joining the parts of a composite key (default: the model's, else `'::'`) | | `autoGenerateKey` | Generate a key for rows without one. Works only with the default `'key'`. When `keyProperty`, in these options or on the model, names another field or is an array, it is ignored and rows without a key are still dropped as `MissingKey` | ```typescript filename="data-options.ts" // Custom key property dcupl.data.set( [{ id: '1', name: 'Laptop' }], { model: 'Product', keyProperty: 'id', } ); // Auto-generate keys dcupl.data.set( [ { name: 'Laptop', price: 999 }, // No key provided ], { model: 'Product', autoGenerateKey: true, } ); ``` ## Performance tips #### Batch updates Always batch multiple items into a single update call. Avoid calling update inside loops. #### Use incremental updates Prefer `update()` over `set()` when possible. Full replacement is slower than incremental updates. #### Only include changed fields When updating, only include the fields that actually changed. This reduces processing overhead. ```typescript filename="performance.ts" // Good: Single update with multiple items dcupl.data.update( [ { key: 'p1', price: 899 }, { key: 'p2', price: 25 }, ], { model: 'Product' } ); await dcupl.update(); // Avoid: Multiple individual updates const changed = [{ key: 'p1', price: 899 }, { key: 'p2', price: 25 }]; for (const product of changed) { dcupl.data.update([product], { model: 'Product' }); await dcupl.update(); // Triggers processing each time! } // Good: only the changed field dcupl.data.update([{ key: 'p1', price: 899 }], { model: 'Product' }); // Wasteful: all fields when only price changed dcupl.data.update([{ key: 'p1', name: 'Laptop', price: 899, category: 'Electronics' }], { model: 'Product' }); ``` ## Best practices #### Always await dcupl.update() Data operations are queued and processed when you call `dcupl.update()`. Always await it before querying to ensure you get the latest data. Missing the await can lead to stale data bugs that are hard to track down. ```typescript filename="best-practices.ts" const list = dcupl.lists.create({ modelKey: 'Product' }); // Good: Await update before querying dcupl.data.update([{ key: 'p1', price: 899 }], { model: 'Product' }); await dcupl.update(); const results = list.catalog.query.items(); // Bad: Missing await - data may be stale! dcupl.data.update([{ key: 'p1', price: 799 }], { model: 'Product' }); dcupl.update(); // Not awaited! const staleResults = list.catalog.query.items(); // Stale data! ``` ## Next steps - [Lists](/sdk/core-concepts/lists) — Query your updated data - [Real-Time Updates](/sdk/advanced/realtime-updates) — Live data updates pattern - [Hybrid Data](/sdk/guides/hybrid-data) — Combine static and dynamic data --- # Query methods & operators Source: https://docs.dcupl.com/sdk/queries/methods-operators Description: Guide to building and changing a list query. The full method reference is the Catalog API. # Query methods & operators Guide to building and changing a list query. The full method reference is the Catalog API. ## Query object Access the query API through `list.catalog.query`: - Every `DcuplList` has a `catalog` with a `query` object - The query object manages conditions, groups, and options - Changes to the query automatically update filtered results The query API provides methods to **add**, **set**, **remove**, and **execute** queries. ```typescript filename="query-object.ts" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [{ key: 'status', type: 'string' }], data: [{ key: 'p1', status: 'active' }], }); await dcupl.init(); // Create a list const list = dcupl.lists.create({ modelKey: 'Product', }); // Access the query object const query = list.catalog.query; // Add conditions, get results query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); const items = query.items(); ``` ## Adding conditions `addCondition()` adds filter conditions to the query: - Pass a single `QueryCondition` object - Or pass an array of conditions - Each condition goes into a group named after its attribute - Conditions on **different** attributes combine with **AND** - Conditions on the **same** attribute combine with **OR** Use `addCondition` when building queries incrementally, like adding filters one at a time based on user selections. For a range on one attribute (AND) or an OR across attributes, use `addGroup()` with an explicit `groupType`. ```typescript filename="add-condition.ts" // Single condition query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // Different attributes combine with AND query.addCondition([ { attribute: 'inStock', operator: 'eq', value: true }, { attribute: 'rating', operator: 'gte', value: 4 }, ]); // Result: category=Electronics AND inStock=true AND rating>=4 // The same attribute combines with OR query.addCondition({ attribute: 'category', operator: 'eq', value: 'Computers', }); // Result: (category=Electronics OR category=Computers) // AND inStock=true AND rating>=4 // A range needs an 'and' group - two addCondition calls // on 'price' would mean price>=100 OR price<=500 query.addGroup({ groupKey: 'price', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: 100 }, { attribute: 'price', operator: 'lte', value: 500 }, ], }); ``` ## Setting conditions `setCondition()` replaces the conditions on one attribute: - Replaces the conditions in the group of the condition's attribute - Keeps the conditions on all other attributes - Useful for single-choice filters, like a category dropdown To start over with a completely different filter, call `clear()` first. ```typescript filename="set-condition.ts" // Add some conditions first query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); query.addCondition({ attribute: 'price', operator: 'gte', value: 100, }); // setCondition replaces the 'category' conditions only query.setCondition({ attribute: 'category', operator: 'eq', value: 'Furniture', }); // Now: category=Furniture AND price>=100 // Start over with a different filter query.clear(); query.setCondition([ { attribute: 'status', operator: 'eq', value: 'active' }, { attribute: 'onSale', operator: 'eq', value: true }, ]); // Now: status=active AND onSale=true ``` ## Query groups Use `addGroup()` and `setGroup()` for complex boolean logic: - `groupType: 'and'` - All conditions must match - `groupType: 'or'` - Any condition can match - `groupKey` - Identifier to extend, replace, or remove the group later Groups are combined with AND. `addGroup()` with an existing `groupKey` extends that group; `setGroup()` replaces it. `addGroup()` always adds a group without a `groupKey` as a new group, while `setGroup()` without a key replaces the first keyless group. Give groups a key to keep them apart. Always set `groupType`: a group created or replaced (`setGroup()`) without one is stored as `'or'`. Groups can be nested for arbitrary boolean expressions like `(A AND B) OR (C AND D)`. ```typescript filename="query-groups.ts" // Add OR group: Electronics OR Computers query.addGroup({ groupType: 'or', groupKey: 'category-filter', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Computers' }, ], }); // setGroup replaces the group with the same groupKey query.setGroup({ groupType: 'or', groupKey: 'category-filter', queries: [ { attribute: 'category', operator: 'eq', value: 'Furniture' }, ], }); // Nested group: inStock=true AND (brand=Apple OR brand=Samsung) query.addGroup({ groupType: 'and', groupKey: 'stock-and-brand', queries: [ { attribute: 'inStock', operator: 'eq', value: true }, { groupType: 'or', queries: [ { attribute: 'brand', operator: 'eq', value: 'Apple' }, { attribute: 'brand', operator: 'eq', value: 'Samsung' }, ], }, ], }); // Result: category=Furniture // AND inStock=true AND (brand=Apple OR brand=Samsung) ``` ## Removing conditions Remove specific conditions or clear the entire query: | Method | Description | | --- | --- | | `removeCondition(condition)` | Remove one condition (pass the full condition) | | `removeGroup(groupKey)` | Remove a group by its key. Attribute groups from `addCondition` are keyed by the attribute name. | | `clear()` | Remove all conditions and groups, and the sort and pagination options | ```typescript filename="removing-conditions.ts" // Setup: add conditions query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); query.addCondition({ attribute: 'price', operator: 'gte', value: 100 }); query.addGroup({ groupKey: 'brand-filter', groupType: 'or', queries: [ { attribute: 'brand', operator: 'eq', value: 'Apple' }, { attribute: 'brand', operator: 'eq', value: 'Samsung' }, ], }); // Remove one condition query.removeCondition({ attribute: 'price', operator: 'gte', value: 100 }); // Remove all conditions on an attribute query.removeGroup('category'); // Remove group by key query.removeGroup('brand-filter'); // Clear everything query.clear(); // Query is now empty ``` ## Query options `applyOptions()` configures pagination, sorting, and projection: | Option | Description | | --- | --- | | `start` | Skip N items (offset for pagination) | | `count` | Limit results to N items | | `sort` | `{ attributes, order }` - one `'ASC'` or `'DESC'` per attribute (uppercase) | | `projection` | Select specific properties to return, e.g. `{ $: false, name: true }` | ```typescript filename="apply-options.ts" // Pagination: page 2, 20 items per page query.applyOptions({ start: 20, count: 20, }); // Sorting: by price descending, then by name query.applyOptions({ sort: { attributes: ['price', 'name'], order: ['DESC', 'ASC'] }, }); // Projection: only return specific fields ($: false drops the rest; // key is always included) query.applyOptions({ projection: { $: false, name: true, price: true, category: true }, }); // Combine all options query.applyOptions({ start: 0, count: 50, sort: { attributes: ['createdAt'], order: ['DESC'] }, projection: { $: false, name: true, price: true }, }); ``` ## Getting results Retrieve query results with these methods: | Method | Returns | Use Case | | --- | --- | --- | | `items()` | Array of matching items | Display lists, tables | | `first()` | First item of the current page (an applied `start` counts) or undefined | Single item lookup | | `any()` | Boolean: any item on the current page (`false` when `start` is past the last match) | Check existence | | `count()` | Number of items on the current page | Page size checks | | `fn.metadata().currentSize` | Number of all matching items | Display total count | ```typescript filename="results.ts" // Get all matching items (respects start/count options) const products = query.items(); // => [{ key: 'p1', name: 'Laptop', ... }, ...] // Get just the first match const firstProduct = query.first(); // => { key: 'p1', name: 'Laptop', ... } or undefined // Check if any items match const hasResults = query.any(); // => true or false // Count items on the current page (respects start/count) const pageCount = query.count(); // => 20 // Total number of matching items (ignores start/count) const totalCount = list.catalog.fn.metadata().currentSize; // => 42 // Common pattern: pagination info const items = query.items(); const total = list.catalog.fn.metadata().currentSize; console.log(`Showing ${items.length} of ${total} results`); ``` ## Query state Inspect the current state of a query: | Method | Returns | Use Case | | --- | --- | --- | | `current()` | QueryExecutionContext | Serialize, debug, or restore query | | `isEmpty()` | Boolean | Check if query has conditions | | `has({ groupKey })` | Boolean | Check for a specific group | ```typescript filename="query-state.ts" // Get complete query context const context = query.current(); // => { modelKey: 'Product', groupKey: 'root', groupType: 'and', // queries: [...], start: 0, count: 50 } // Useful for debugging or saving to URL/storage console.log(JSON.stringify(context, null, 2)); // Check if query is empty if (query.isEmpty()) { console.log('No filters applied'); } // Check if an attribute is filtered // (addCondition groups are keyed by the attribute name) const hasCategory = query.has({ groupKey: 'category' }); // Check if specific group exists const hasBrandGroup = query.has({ groupKey: 'brand-filter' }); // => true if a group with that key exists ``` ## Operators The `operator` property determines how values are compared: | Category | Operators | | --- | --- | | Equality | `eq`, `in` | | Comparison | `gt`, `gte`, `lt`, `lte` | | Text and pattern | `find` | | Type and value checks | `typeof`, `isTruthy`, `size` | There are no separate "not" operators: negate any condition with `options: { invert: true }`. Text search uses `find` with a regular expression in slashes; a bare string must match the whole value. Ranges are an `'and'` group of `gte` and `lte`. Any other `operator` name runs the [script](/sdk/guides/scripts) registered under that key as a custom operator. Without one, the query throws an `UnknownOperator` error naming the operator and the valid ones, so a typo like `'neq'` fails instead of silently returning no items. See [Operators Reference](/sdk/api/query-operators) for full documentation. ```typescript filename="operators.ts" // Equality { attribute: 'status', operator: 'eq', value: 'active' } // Not equal: invert any condition { attribute: 'status', operator: 'eq', value: 'deleted', options: { invert: true } } // Comparison { attribute: 'price', operator: 'gte', value: 100 } { attribute: 'price', operator: 'lt', value: 1000 } // Contains: wrap the term in slashes (a bare string matches the whole value) { attribute: 'name', operator: 'find', value: '/laptop/', options: { regexFlags: 'i' } } // Starts with / Ends with { attribute: 'sku', operator: 'find', value: '/^SKU-/' } { attribute: 'email', operator: 'find', value: '/@company\\.com$/' } // In list { attribute: 'category', operator: 'in', value: ['Electronics', 'Computers'] } // Not in list { attribute: 'status', operator: 'in', value: ['archived', 'deleted'], options: { invert: true } } // Has a value (null counts as a value here: typeof null is 'object') { attribute: 'description', operator: 'typeof', value: 'undefined', options: { invert: true } } // Has a usable value (not undefined, null or '') { attribute: 'description', operator: 'isTruthy', value: true } // Array or string length { attribute: 'tags', operator: 'size', value: 0 } ``` ## Condition options Fine-tune condition behavior with the `options` property: | Option | Description | | --- | --- | | `transform` | Array of transforms applied to both sides before comparing: `'lowercase'`, `'trim'`, `'removeWhitespace'` | | `invert` | Negate the condition result | | `arrayValueHandling` | How to match array values (`'some'`, the default, or `'every'`) | | `regexFlags` | `find` only: flags for a `'/regex/'` value, e.g. `'i'` for case-insensitive | ```typescript filename="condition-options.ts" // Case-insensitive exact match with transform query.addCondition({ attribute: 'name', operator: 'eq', value: 'laptop', options: { transform: ['lowercase'] }, }); // Case-insensitive substring search with regexFlags query.addCondition({ attribute: 'name', operator: 'find', value: '/lap/', options: { regexFlags: 'i' }, }); // Invert: find items NOT matching query.addCondition({ attribute: 'status', operator: 'eq', value: 'archived', options: { invert: true }, }); // Equivalent to: status != 'archived' // Array handling: at least one tag is 'sale' or 'featured' query.addCondition({ attribute: 'tags', operator: 'in', value: ['sale', 'featured'], options: { arrayValueHandling: 'some' }, }); // Array handling: every tag is 'premium' or 'verified' // (an empty tags array never matches 'every') query.addCondition({ attribute: 'tags', operator: 'in', value: ['premium', 'verified'], options: { arrayValueHandling: 'every' }, }); ``` ## Next steps - [Catalog API](/sdk/api/catalog) — Full reference of every query method (apply, remove, execute, one, many, ...) - [Basic filtering](/sdk/queries/query-filter) — Interactive filtering with facets - [Query logic](/sdk/queries/query-logic) — Complex boolean expressions - [Results & Pagination](/sdk/queries/query-results) — Pagination, sorting, and projections - [Operators Reference](/sdk/api/query-operators) — Complete list of all operators --- # Basic filtering Source: https://docs.dcupl.com/sdk/queries/query-filter Description: Filter data using conditions with addCondition(). The foundation of dcupl queries - add conditions to narrow down results based on field values, types, and… # Basic filtering Filter data using conditions with addCondition(). The foundation of dcupl queries - add conditions to narrow down results based on field values, types, and boolean logic. ## Syntax ```typescript list.catalog.query.addCondition({ attribute: string, operator: Operator, value: any, options?: ConditionOptions }) ``` ## Parameters - `attribute` (`string`, required) — The property to filter on (supports dot notation for deep queries) - `operator` (`Operator`, required) — Comparison operator: eq, in, typeof, isTruthy, size (see other pages for find, gt/gte/lt/lte) - `value` (`any`, required) — Value to compare against. Type depends on operator (in needs an array). - `options.transform` (`('lowercase' | 'trim' | 'removeWhitespace')[]`, optional) — Transform both attribute and search value before comparison - `options.invert` (`boolean`, optional) — Negate the operator result (NOT logic). An item without a value (undefined/null) matches neither way, except with eq, in, typeof and isTruthy - `options.arrayValueHandling` (`'some' | 'every'`, optional) — For array attributes: 'some' matches any element (default), 'every' requires all elements to match (and at least one element) ## Return type Returns the updated query. Call items() to get matching results. ```typescript // All query operators type Operator = 'eq' | 'in' | 'find' | 'gt' | 'gte' | 'lt' | 'lte' | 'typeof' | 'isTruthy' | 'size'; // Options interface interface ConditionOptions { transform?: ('lowercase' | 'trim' | 'removeWhitespace')[]; invert?: boolean; arrayValueHandling?: 'some' | 'every'; regexFlags?: string; // find only, e.g. 'i' (see Text Search) } ``` ## Usage notes - Call addCondition() multiple times to add more conditions - Conditions on different attributes use AND logic; conditions on the same attribute use OR (see Query Logic below) - Use in to match any of several values (invert it for "not in") - Use eq for exact matches, isTruthy for boolean-like checks - typeof 'undefined' finds absent values, isTruthy false also finds null and ''; for wrongly typed source data, use quality checks (see Quality & Validation) - Use options.invert to negate any operator (eq + invert = 'not equal') - Transform option applies to BOTH the data value and your search value - For text search, see Text Search. For ranges, see Range Queries. ## Related guides/docs - [Text search](/sdk/queries/query-search) - [Range queries](/sdk/queries/query-range) - [Query logic](/sdk/queries/query-logic) - [Results & Pagination](/sdk/queries/query-results) - [Query operators reference](/sdk/api/query-operators) - [Quality & validation](/sdk/advanced/quality-validation) ## Examples **Filter by category** Find all products in a specific category using exact match. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const electronics = productList.catalog.query.items(); console.log('Electronics products:', electronics.length); ``` **Filter active products** Find products that are currently active using boolean filter. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); const activeProducts = productList.catalog.query.items(); ``` **Match any of several values** Use the in operator to match any value from a list. The value must be an array: anything else throws "invalid query - in operator needs an array value" and the condition is not added. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // in matches if the value equals any listed value productList.catalog.query.addCondition({ attribute: 'category', operator: 'in', value: ['Electronics', 'Computers'], }); // Add options: { invert: true } for "not in" const results = productList.catalog.query.items(); ``` **Check in-stock items** Use isTruthy to find items with truthy stock values (> 0, true, etc.). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // isTruthy matches: true, 1, 5, 100, "yes", non-empty values // isTruthy excludes: false, 0, null, undefined, "" productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); const inStockProducts = productList.catalog.query.items(); ``` **Find items with a price** dcupl converts typed properties when it loads them: '49.99' becomes 49.99 and 'abc' becomes undefined. On a float property, typeof 'number' therefore separates items with a valid price from items without one. To find wrongly typed source data, use quality checks. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // typeof checks JavaScript types: 'string', 'number', 'boolean', 'object', 'undefined' productList.catalog.query.addCondition({ attribute: 'price', operator: 'typeof', value: 'number', }); const pricedProducts = productList.catalog.query.items(); console.log('Products with a price:', pricedProducts.length); ``` **Filter by array length** size compares the length of an array (or a string) with the value. Items without a value never match, not even size 0. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Products with exactly two tags productList.catalog.query.addCondition({ attribute: 'tags', operator: 'size', value: 2, }); const twoTagProducts = productList.catalog.query.items(); ``` **Find missing data** Find products without a description (data quality check). typeof "undefined" matches an absent value only: an explicit null or an empty string is a value. isTruthy false finds all three. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Absent only: null is 'object' and '' is 'string' productList.catalog.query.addCondition({ attribute: 'description', operator: 'typeof', value: 'undefined', }); // Absent, null or '': { attribute: 'description', operator: 'isTruthy', value: false } const missingDescriptions = productList.catalog.query.items(); console.log('Products needing descriptions:', missingDescriptions.length); ``` **Exclude values with invert** Find all products except discontinued ones using invert option. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // invert: true negates the operator (eq + invert = not equal) productList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'discontinued', options: { invert: true }, }); const availableProducts = productList.catalog.query.items(); ``` **Case-insensitive match** Match brand name regardless of case using transform option. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // transform applies to both attribute value and search value productList.catalog.query.addCondition({ attribute: 'brand', operator: 'eq', value: 'nike', options: { transform: ['lowercase'] }, }); // Matches: "Nike", "NIKE", "nike", "NiKe" const nikeProducts = productList.catalog.query.items(); ``` **Filter array values** Find products tagged with "featured" using array value handling. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // arrayValueHandling: 'some' (default) - match if ANY element matches // arrayValueHandling: 'every' - match only if ALL elements match // (an empty array never matches) productList.catalog.query.addCondition({ attribute: 'tags', operator: 'eq', value: 'featured', options: { arrayValueHandling: 'some' }, }); const featuredProducts = productList.catalog.query.items(); ``` **Multiple conditions (AND)** Combine filters on different attributes - all must match (AND logic). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Conditions on different attributes are combined with AND // (conditions on the same attribute are combined with OR) productList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); // Result: active AND Electronics AND in stock const results = productList.catalog.query.items(); ``` **Array of conditions** Pass multiple conditions as an array for cleaner code. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition([ { attribute: 'status', operator: 'eq', value: 'active' }, { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'inStock', operator: 'isTruthy', value: true }, ]); const results = productList.catalog.query.items(); ``` **SDK** ```typescript // Basic filtering example const list = dcupl.lists.create({ modelKey: 'Article' }); // Filter by category list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Running', }); // Only in-stock items list.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); // Get filtered results const results = list.catalog.query.items(); console.log('Found:', results.length, 'items'); ``` --- # Text search Source: https://docs.dcupl.com/sdk/queries/query-search Description: Search text fields with the find operator. Wrap the search term in slashes, like /shoe/, to match it anywhere in the value (the text between the slashes is a… # Text search Search text fields with the find operator. Wrap the search term in slashes, like /shoe/, to match it anywhere in the value (the text between the slashes is a regular expression), and set regexFlags to i for case-insensitive search. Without slashes, find compares the whole value exactly. Essential for search bars, autocomplete, and text filtering. ## Syntax ```typescript list.catalog.query.addCondition({ attribute: string, operator: 'find', value: string | object, // '/term/' = contains, 'term' = exact, { brand: 'Apple' } = partial object match options?: SearchOptions }) ``` ## Parameters - `attribute` (`string`, required) — The text field to search in - `operator` (`'find'`, required) — The find operator for text and pattern matching - `value` (`string | object`, required) — /term/ matches the term anywhere in the value (regex); a bare term without slashes must equal the whole value. An object matches part of a json value ({ brand: 'Apple' }), see the Query Operators Reference - `options.regexFlags` (`string`, optional) — Regex flags for a '/term/' value, e.g. 'i' for case-insensitive. Put flags here, not after the closing slash ('/term/i' is an exact match) - `options.transform` (`('lowercase' | 'trim' | 'removeWhitespace')[]`, optional) — Transform both data and search value before comparison - `options.invert` (`boolean`, optional) — Exclude matches (find items NOT containing the term). Items without a value are left out too - `options.arrayValueHandling` (`'some' | 'every'`, optional) — For array fields: 'some' (any element) or 'every' (all elements) ## Return type addCondition() returns the updated query (DcuplGlobalQueryOptions); call items() for the matching items: ```typescript // list.catalog.query addCondition(condition: DcuplQuery | DcuplQuery[]): DcuplGlobalQueryOptions; // the list's query items(options?: DcuplListQueryOptions): T[]; ``` ## Usage notes - '/phone/' matches "Smartphone" and "headphones", not "Phone Case"; with regexFlags: 'i' it matches all three. '/^run/' with 'i' matches "Running Shoes" and "runner", not "outrun". 'Phone Case' (no slashes) matches only that whole value - A /regex/ is tested against non-string values as strings: /^1/ on a number attribute matches 150 and 100 - Transforms: 'lowercase', 'trim' (leading/trailing whitespace) and 'removeWhitespace' (all whitespace) - Wrap the term in slashes (/term/) for substring search; a bare string must match the whole value - Set regexFlags to i for case-insensitive, user-facing search - Escape regex characters (. \* + ? ^ $ { } ( ) | \[ \] \\) in user input before wrapping it in slashes - Transforms apply to BOTH the data value AND your search term - Use multiple find conditions on same field for OR logic - Combine with other operators (eq, gte) for filtered search - For suggestions/autocomplete, see catalog.fn.suggest() ## Related methods - `catalog.fn.suggest()` — Get value suggestions for autocomplete (case-insensitive prefix match, or a /regex/) ## Related guides/docs - [Basic filtering](/sdk/queries/query-filter) - [Results & Pagination](/sdk/queries/query-results) - [Query Operators Reference: find](/sdk/api/query-operators#find) ## Examples **Substring search** Wrap the term in slashes to find products whose name contains "Bike" (case-sensitive by default). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/Bike/', }); // Matches: "Mountain Bike", "Bike Helmet" // No match: "bike lock" (case-sensitive), "Bicycle" const results = productList.catalog.query.items(); ``` **Case-insensitive search** Add regexFlags: 'i' to ignore case (recommended for user-facing search). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/running/', options: { regexFlags: 'i' }, }); // Matches: "Running Shoes", "Trail Running Shoes", "running socks" const results = productList.catalog.query.items(); ``` **Search with user input** Escape regex characters in user input before wrapping it in slashes. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const escapeRegExp = (text: string) => text.replace(/[.*+?^${}()|[\]\\]/g, (char) => '\\' + char); const userInput = ' shoe '; productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: `/${escapeRegExp(userInput.trim())}/`, options: { regexFlags: 'i' }, }); // Matches: "Running Shoes", "Trail Running Shoes", "Snowshoes" const results = productList.catalog.query.items(); ``` **Pattern search** The text between the slashes is a JavaScript regular expression, so anchors and alternation work too. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Names that start with "trail" or "mountain", any case productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/^(trail|mountain)/', options: { regexFlags: 'i' }, }); // Matches: "Trail Running Shoes", "Mountain Bike" // No match: "Bike Trailer" (does not start with it) const results = productList.catalog.query.items(); ``` **Exact match (no slashes)** A bare string compares the whole value. Transforms normalize both sides before comparing. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // trim: remove leading/trailing whitespace // removeWhitespace: remove ALL whitespace productList.catalog.query.addCondition({ attribute: 'sku', operator: 'find', value: 'ABC123', options: { transform: ['trim', 'removeWhitespace'] }, }); // Matches: " ABC 123 ", "ABC123", "A B C 1 2 3" // No match: "ABC1234" (a bare string never matches part of a value) const results = productList.catalog.query.items(); ``` **Search multiple fields** Search name OR description by putting both conditions in one or-group. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Conditions on different attributes are combined with AND, // so group them with groupType 'or' productList.catalog.query.addGroup({ groupKey: 'search', groupType: 'or', queries: [ { attribute: 'name', operator: 'find', value: '/waterproof/', options: { regexFlags: 'i' } }, { attribute: 'description', operator: 'find', value: '/waterproof/', options: { regexFlags: 'i' } }, ], }); // Matches: "Waterproof Jacket", and "Hiking Boots" (described as waterproof) const results = productList.catalog.query.items(); ``` **Exclude search terms** Find products NOT containing "refurbished" using invert. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/refurbished/', options: { regexFlags: 'i', invert: true, // Exclude matches }, }); // Returns products without "refurbished" in the name const newProducts = productList.catalog.query.items(); ``` **Search in array fields** Find products where ANY tag contains the search term. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'tags', operator: 'find', value: '/sale/', options: { regexFlags: 'i', arrayValueHandling: 'some', // Match if ANY element contains "sale" }, }); // Matches: tags = ["summer-sale", "clearance"] // Matches: tags = ["new", "SALE-2024"] const saleProducts = productList.catalog.query.items(); ``` **Search with category filter** Combine text search with other filters. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Text search productList.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/pro/', options: { regexFlags: 'i' }, }); // Combined with category filter (AND logic) productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // Result: Electronics products with "pro" in the name const results = productList.catalog.query.items(); ``` **SDK** ```typescript // Case-insensitive substring search const list = dcupl.lists.create({ modelKey: 'Article' }); // Find products whose name contains "shoe" (any case) list.catalog.query.addCondition({ attribute: 'productName', operator: 'find', value: '/shoe/', options: { regexFlags: 'i' }, }); // Get search results const results = list.catalog.query.items(); console.log('Found:', results.length, 'matching products'); ``` --- # Range queries Source: https://docs.dcupl.com/sdk/queries/query-range Description: Filter by numeric ranges and date comparisons using gt, gte, lt, lte operators. Use setGroup with "and" type to combine range conditions. Perfect for price… # Range queries Filter by numeric ranges and date comparisons using gt, gte, lt, lte operators. Use setGroup with "and" type to combine range conditions. Perfect for price filters, date ranges, stock thresholds, and rating cutoffs. ## Syntax ```typescript list.catalog.query.setGroup({ groupKey: string, groupType: 'and', queries: [ { attribute: string, operator: 'gte', value: number | Date }, { attribute: string, operator: 'lte', value: number | Date } ] }) ``` ## Parameters - `groupKey` (`string`, optional) — Set it so you can replace or remove the range later. Without it, setGroup() replaces the first group without a key - `groupType` (`'and' | 'or'`, optional) — Set it to "and" so all conditions must match (range behavior), on every setGroup(): a group without groupType is stored as "or", also when it replaces an existing range - `queries` (`Condition[]`, required) — Array of conditions with attribute, operator (gt/gte/lt/lte), and value ## Return type setGroup() returns the updated query (DcuplGlobalQueryOptions); call items() for the matching items: ```typescript // list.catalog.query setGroup(group: DcuplQueryGroup): DcuplGlobalQueryOptions; // the list's query items(options?: DcuplListQueryOptions): T[]; ``` ## Usage notes - gt: value > bound, gte: value >= bound, lt: value < bound, lte: value <= bound - gte + lte is an inclusive range \[50, 200\], gt + lt an exclusive one (50, 200), gte + lt a half-open one \[50, 200) - Use setGroup with groupType "and" to combine gte + lte for ranges - Two addCondition() calls on the same attribute are combined with OR, so they do not form a range - groupKey allows you to update or remove the range filter later - Works with numbers, dates, and strings (strings compare by character code, so case-sensitively) - For date properties, pass a Date object (or a timestamp); an ISO string value does not match - Use gt (exclusive) when you want to exclude the boundary - Use gte (inclusive) when you want to include the boundary ## Related methods - `catalog.query.setGroup()` — Create grouped conditions with AND/OR logic - `catalog.fn.aggregate()` — Get min/max values for range bounds ## Related guides/docs - [Basic filtering](/sdk/queries/query-filter) - [Query logic](/sdk/queries/query-logic) ## Examples **Minimum price filter** Find products priced above a threshold using greater-than. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // gt = exclusive (>), does NOT include the threshold value productList.catalog.query.addCondition({ attribute: 'price', operator: 'gt', value: 100, }); // Matches: 100.01, 150, 500 // No match: 100 (exclusive) const results = productList.catalog.query.items(); ``` **Maximum price filter** Find products under a price limit using less-than-or-equal. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // lte = inclusive (<=), includes the threshold value productList.catalog.query.addCondition({ attribute: 'price', operator: 'lte', value: 50, }); // Matches: 50, 49.99, 25, 0 const affordableProducts = productList.catalog.query.items(); ``` **Price range (inclusive)** Find products between $50 and $200 (inclusive on both ends). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Use setGroup with "and" type for inclusive range productList.catalog.query.setGroup({ groupKey: 'price-range', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: 50 }, { attribute: 'price', operator: 'lte', value: 200 }, ], }); // Matches: 50, 75, 100, 150, 200 const results = productList.catalog.query.items(); ``` **Price range (exclusive)** Find products between $50 and $200 (exclusive on both ends). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Use setGroup with "and" type for exclusive range productList.catalog.query.setGroup({ groupKey: 'price-range', groupType: 'and', queries: [ { attribute: 'price', operator: 'gt', value: 50 }, { attribute: 'price', operator: 'lt', value: 200 }, ], }); // Matches: 50.01, 75, 100, 199.99 // No match: 50, 200 const results = productList.catalog.query.items(); ``` **Low stock alert** Find products with critically low inventory. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'stock', operator: 'lt', value: 10, }); // Also filter to active products only productList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); const lowStockProducts = productList.catalog.query.items(); console.log('Need to reorder:', lowStockProducts.length, 'products'); ``` **Date after (recent items)** Find orders placed on or after a specific date. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); orderList.catalog.query.addCondition({ attribute: 'orderDate', operator: 'gte', value: new Date('2025-01-01'), }); const recentOrders = orderList.catalog.query.items(); ``` **Date range (last 30 days)** Find items created within the last 30 days. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const thirtyDaysAgo = new Date(); thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30); // Use setGroup with "and" type for date range productList.catalog.query.setGroup({ groupKey: 'date-range', groupType: 'and', queries: [ { attribute: 'createdAt', operator: 'gte', value: thirtyDaysAgo }, { attribute: 'createdAt', operator: 'lte', value: new Date() }, ], }); const newProducts = productList.catalog.query.items(); ``` **Rating threshold** Find products with 4+ stars rating. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'rating', operator: 'gte', value: 4.0, }); const topRatedProducts = productList.catalog.query.items(); ``` **Combine range with other filters** Find electronics between $100-$500 that are in stock. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Use setGroup for price range productList.catalog.query.setGroup({ groupKey: 'price-range', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: 100 }, { attribute: 'price', operator: 'lte', value: 500 }, ], }); // Add other filters separately productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); const results = productList.catalog.query.items(); ``` **SDK** ```typescript // Price range filter: $50 - $150 const list = dcupl.lists.create({ modelKey: 'Article' }); // Use setGroup with "and" type for range queries list.catalog.query.setGroup({ groupKey: 'price-range', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: 50 }, { attribute: 'price', operator: 'lte', value: 150 }, ], }); const results = list.catalog.query.items(); console.log('Products in range:', results.length); ``` --- # Query logic Source: https://docs.dcupl.com/sdk/queries/query-logic Description: Combine conditions with AND/OR logic. Conditions on different attributes are combined with AND, conditions on the same attribute with OR. For any other… # Query logic Combine conditions with AND/OR logic. Conditions on different attributes are combined with AND, conditions on the same attribute with OR. For any other combination, use addGroup with an explicit groupType. Build complex boolean expressions for advanced filtering with faceted search patterns. ## Syntax ```typescript list.catalog.query.addGroup({ groupKey?: string, groupType?: 'and' | 'or', queries: Array }) ``` ## Parameters - `groupKey` (`string`, optional) — Names the group. Adding a group with an existing groupKey extends it; setGroup() replaces it; removeGroup(groupKey) removes it. The keyed group is found at any nesting depth (a top-level match first) and changed where it is. Removing a group that leaves its wrapper group empty removes the wrapper too (and every wrapper above it that becomes empty). addGroup() always adds a group without a key as a new group; setGroup() without a key replaces the first keyless group. - `groupType` (`'and' | 'or'`, optional) — Logic within the group. Set it explicitly: a group without groupType is stored as 'or', whether it is new or replaces an existing group with setGroup() (addGroup() without groupType extending a keyed group keeps that group's groupType), while a nested group inside queries without groupType is evaluated as 'and'. - `queries` (`Array`, required) — Conditions or nested groups in this group ## Return type addGroup(), setGroup() and addCondition() return the updated query (DcuplGlobalQueryOptions). Use current() to inspect it, clear() to reset: ```typescript // Shape of the returned query, e.g. after addCondition() on category and brand type DcuplGlobalQueryOptions = { modelKey: string; groupKey: 'root'; groupType: 'and'; // top-level groups are ANDed queries: Array; // here: [{ groupKey: 'category', groupType: 'or', queries: [...] }, // { groupKey: 'brand', groupType: 'or', queries: [...] }] }; ``` ## Usage notes - Default: addCondition() puts each condition into a group named after its attribute, so different attributes use AND and the same attribute uses OR - Use addGroup with groupType: 'or' for OR across different attributes - Use addGroup with groupType: 'and' for ranges on one attribute - Different groups are ANDed together at the top level - Use setCondition() for single-select dropdowns (replaces the conditions on that attribute) - Use addCondition() for multi-select checkboxes (accumulates, OR within the attribute) - removeCondition() removes one condition (pass the full condition); removeGroup(groupKey) removes a whole group - clear() removes all conditions ## Related methods - `catalog.query.addGroup()` — Add or extend a group of conditions - `catalog.query.setGroup()` — Replace a group by its groupKey - `catalog.query.current()` — Get current query structure - `catalog.query.isEmpty()` — Check if any conditions exist - `catalog.query.clear()` — Remove all conditions - `catalog.query.removeCondition()` — Remove specific condition ## Examples **Default AND logic** Multiple conditions on different attributes use AND by default. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // These conditions are ANDed automatically productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); // Result: Electronics AND in stock const results = productList.catalog.query.items(); ``` **OR logic with same attribute** Multiple values for same attribute create OR logic by default. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Same attribute = OR logic productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Computers', }); // Result: Electronics OR Computers const results = productList.catalog.query.items(); ``` **Explicit OR group** Use addGroup with groupType: "or" for OR logic across different attributes. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Conditions on different attributes are combined with AND, // so put them in one group with groupType 'or' productList.catalog.query.addGroup({ groupKey: 'special', groupType: 'or', queries: [ { attribute: 'featured', operator: 'eq', value: true }, { attribute: 'onSale', operator: 'eq', value: true }, ], }); // Result: featured OR on sale const specialProducts = productList.catalog.query.items(); ``` **Multiple OR groups (AND between groups)** Combine multiple OR groups with AND logic between them. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Category group (OR): Electronics OR Computers productList.catalog.query.addGroup({ groupKey: 'category-group', groupType: 'or', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Computers' }, ], }); // Brand group (OR): Apple OR Samsung productList.catalog.query.addGroup({ groupKey: 'brand-group', groupType: 'or', queries: [ { attribute: 'brand', operator: 'eq', value: 'Apple' }, { attribute: 'brand', operator: 'eq', value: 'Samsung' }, ], }); // Result: (Electronics OR Computers) AND (Apple OR Samsung) const results = productList.catalog.query.items(); ``` **Faceted search pattern** Build a typical e-commerce filter with multiple facets. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); function applyFacetedFilters(filters: { categories?: string[]; brands?: string[]; priceMin?: number; priceMax?: number; }) { const query = productList.catalog.query; query.clear(); // Categories (OR within group). Key the group by its attribute: // fn.facets({ attribute: 'category' }) leaves the group with that key out when counting if (filters.categories?.length) { query.addGroup({ groupKey: 'category', groupType: 'or', queries: filters.categories.map((cat) => ({ attribute: 'category', operator: 'eq' as const, value: cat, })), }); } // Brands: the 'in' operator matches any of the listed values if (filters.brands?.length) { query.addCondition({ attribute: 'brand', operator: 'in', value: filters.brands }); } // Price range (AND within group) const price = []; if (filters.priceMin !== undefined) { price.push({ attribute: 'price', operator: 'gte' as const, value: filters.priceMin }); } if (filters.priceMax !== undefined) { price.push({ attribute: 'price', operator: 'lte' as const, value: filters.priceMax }); } if (price.length) { query.addGroup({ groupKey: 'price', groupType: 'and', queries: price }); } // Result: (cat1 OR cat2) AND (brand1 OR brand2) AND (price range) return query.items(); } ``` **Replace vs. add** Control whether a new condition replaces or adds to the existing ones on the same attribute. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // addCondition adds: Electronics productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // setCondition replaces the conditions on 'category' productList.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: 'Books', }); // Now only Books products match // (addCondition here would give Electronics OR Books) const results = productList.catalog.query.items(); ``` **Remove specific condition** Remove a specific condition while keeping others. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Add multiple category filters productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Books', }); // Remove just the Electronics filter (pass the full condition) productList.catalog.query.removeCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // Now only Books products match const results = productList.catalog.query.items(); // Remove all conditions on 'category' productList.catalog.query.removeGroup('category'); ``` **Query state inspection** Check current query state and reset. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // Check current query state const currentQuery = productList.catalog.query.current(); console.log('Current query:', currentQuery); // Check if query has any conditions const isEmpty = productList.catalog.query.isEmpty(); console.log('Is empty:', isEmpty); // false // Check for a specific group (addCondition groups are keyed by attribute) const hasCategory = productList.catalog.query.has({ groupKey: 'category' }); // Clear all conditions productList.catalog.query.clear(); // Now returns all products const allProducts = productList.catalog.query.items(); ``` **Conditional UI based on query state** Use isEmpty() and has() to drive UI decisions. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Check if query has any conditions if (productList.catalog.query.isEmpty()) { // Show all products view showAllProductsMessage(); } // Check if a specific filter is active if (productList.catalog.query.has({ groupKey: 'category' })) { // Show reset button for category filter showCategoryResetButton(); } // Combine with faceted search function updateFilterUI() { const query = productList.catalog.query; // Show "clear all" only when filters are active clearAllButton.hidden = query.isEmpty(); // Show individual reset buttons per filter categoryReset.hidden = !query.has({ groupKey: 'category' }); brandReset.hidden = !query.has({ groupKey: 'brand' }); priceReset.hidden = !query.has({ groupKey: 'price' }); } ``` **SDK** ```typescript // Faceted search: (category1 OR category2) AND (brand1) const list = dcupl.lists.create({ modelKey: 'Article' }); // Conditions on the same attribute are combined with OR list.catalog.query.addCondition([ { attribute: 'category', operator: 'eq', value: 'Running' }, { attribute: 'category', operator: 'eq', value: 'Training' }, ]); // Conditions on different attributes are combined with AND list.catalog.query.addCondition({ attribute: 'brand', operator: 'eq', value: 'Nike', }); const results = list.catalog.query.items(); ``` --- # Deep queries Source: https://docs.dcupl.com/sdk/queries/query-deep Description: Query nested objects, referenced entities, and arrays using dot notation and wildcards. Access deeply nested data structures for filtering across model… # Deep queries Query nested objects, referenced entities, and arrays using dot notation and wildcards. Access deeply nested data structures for filtering across model relationships. ## Syntax ```typescript // Single-valued reference { attribute: 'customer.country', operator: 'eq', value: 'USA' } // Several hops { attribute: 'customer.address.city', operator: 'eq', value: 'Vienna' } // Any element of a multi-valued reference ([*] covers one hop; 'products.category' is the same) { attribute: 'products[*].category', operator: 'eq', value: 'Electronics' } // Several multi-valued hops: a plain dotted path ([*] can't be chained, and // 'orders[*].items.sku' matches nothing, without an error) { attribute: 'orders.items.sku', operator: 'eq', value: 'SKU-1' } // Specific index (0-based) { attribute: 'items[0].name', operator: 'eq', value: 'First Item' } // A field of a json property { attribute: 'metadata.brand', operator: 'eq', value: 'Apple' } ``` ## Parameters - `attribute` (`string`, required) — A property or a path (see Syntax): dots traverse references and json fields, \[\*\] marks one multi-valued hop, \[n\] picks an element by index - `options.arrayValueHandling` (`'some' | 'every'`, optional) — On a path through a multi-valued reference: 'some' (default) matches if any element matches, 'every' only if all do ## Return type Deep paths only change which items match. The query methods return the list query as usual, and items() returns the matching items: ```typescript // list.catalog.query addCondition(condition: DcuplQuery | DcuplQuery[]): DcuplGlobalQueryOptions; // the list's query items(options?: DcuplListQueryOptions): T[]; ``` ## Usage notes - Use dot notation to traverse single-valued references - Use \[\*\] wildcard (or a plain dotted path) to match ANY element of a multi-valued reference - Use \[n\] to query a specific array index (0-based) - Chain multiple levels: customer.address.city - For several multi-valued hops use a dotted path: items.variants.color. \[\*\] can't be chained, and a multi-valued reference after a \[\*\] (items\[\*\].variants.color) matches nothing, without an error - Add options: { arrayValueHandling: 'every' } to require ALL elements to match - Inverted conditions on a reference path can also match items without the reference: on a single-valued reference eq, in, typeof and isTruthy evaluate the missing value, so inverted they match those items (unless the condition itself matches a missing value, like typeof 'undefined' or isTruthy false); find, gt/gte/lt/lte and size never match there, inverted or not; on a path through a multi-valued reference or \[\*\] every inverted operator does. Add an isTruthy true condition on the reference to exclude them; typeof 'undefined' + invert keeps an explicit null reference, since typeof null is 'object' - List a model's references with dcupl.models.get('Order')?.references - For performance, copy a referenced value onto the item with a derived property (see Derived Properties below) ## Related guides/docs - [Basic filtering](/sdk/queries/query-filter) - [Text search](/sdk/queries/query-search) - [Range queries](/sdk/queries/query-range) - [Derived properties](/sdk/advanced/derived-properties) ## Examples **Single-level reference** Query by a property of a referenced entity using dot notation. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Filter orders by customer's country orderList.catalog.query.addCondition({ attribute: 'customer.country', operator: 'eq', value: 'USA', }); // customer is a reference to Customer model // country is a property on Customer const usaOrders = orderList.catalog.query.items(); ``` **Multi-level reference** Chain through multiple references with dot notation. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Order -> Customer -> Address -> City orderList.catalog.query.addCondition({ attribute: 'customer.address.city', operator: 'eq', value: 'New York', }); const nyOrders = orderList.catalog.query.items(); ``` **Array wildcard query** Query any element in an array using \[\*\] wildcard. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Find orders containing any Electronics product // products is an array of Product references orderList.catalog.query.addCondition({ attribute: 'products[*].category', operator: 'eq', value: 'Electronics', }); // Matches if ANY product in the array is Electronics // (the plain dotted path 'products.category' works the same way) const results = orderList.catalog.query.items(); ``` **Specific array index** Query a specific position in an array. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Query the first product in each order: // its name contains "laptop" (any case) orderList.catalog.query.addCondition({ attribute: 'products[0].name', operator: 'find', value: '/laptop/', options: { regexFlags: 'i' }, }); // Only matches if the FIRST product matches const results = orderList.catalog.query.items(); ``` **Nested array query** Use a plain dotted path to go through several multi-valued references. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Order -> products (multi) -> variants (multi) -> color // Dotted paths fan out through every multi-valued reference. // ([*] supports one multi-valued hop: 'products[*].category') orderList.catalog.query.addCondition({ attribute: 'products.variants.color', operator: 'eq', value: 'Red', }); // Matches orders where ANY product has ANY variant in red const results = orderList.catalog.query.items(); ``` **JSON property query** Query nested properties within JSON fields. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // metadata is a JSON property containing arbitrary nested data productList.catalog.query.addCondition({ attribute: 'metadata.brand', operator: 'eq', value: 'Apple', }); // Deeply nested JSON productList.catalog.query.addCondition({ attribute: 'metadata.features.touchscreen', operator: 'eq', value: true, }); const results = productList.catalog.query.items(); ``` **Filter by vendor location** Practical example: filter products by vendor country. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Products from vendors in Germany productList.catalog.query.addCondition({ attribute: 'vendor.address.country', operator: 'eq', value: 'Germany', }); // With additional filters productList.catalog.query.addCondition({ attribute: 'vendor.rating', operator: 'gte', value: 4.0, }); productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'isTruthy', value: true, }); const germanProducts = productList.catalog.query.items(); ``` **Handle missing references** An eq condition on a path through a missing reference does not match, but an inverted one does: "orders NOT from the USA" also returns orders without a customer (missing or null). ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Orders NOT from the USA - this also matches orders without a customer orderList.catalog.query.addCondition({ attribute: 'customer.country', operator: 'eq', value: 'USA', options: { invert: true }, }); // Exclude orders without a customer (missing or null) orderList.catalog.query.addCondition({ attribute: 'customer', operator: 'isTruthy', value: true, }); const nonUsOrders = orderList.catalog.query.items(); ``` **Text search in nested field** Combine deep queries with text search. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Customer name contains "smith" (any case) orderList.catalog.query.addCondition({ attribute: 'customer.name', operator: 'find', value: '/smith/', options: { regexFlags: 'i' }, }); const results = orderList.catalog.query.items(); ``` **Range query on deep property** Apply range filters to referenced entity properties. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Orders where products have high prices orderList.catalog.query.addCondition({ attribute: 'products[*].price', operator: 'gte', value: 500, }); const premiumOrders = orderList.catalog.query.items(); ``` **SDK** ```typescript // Query across references const list = dcupl.lists.create({ modelKey: 'Order' }); // Filter by vendor location (find without slashes compares // the whole value; the transform makes it case-insensitive) list.catalog.query.addCondition({ attribute: 'articleIds[*].vendorId.location', operator: 'find', value: 'New York', options: { transform: ['lowercase'] } }); const orders = list.catalog.query.items(); console.log('Orders with an article from a New York vendor:', orders.length); ``` --- # Query results Source: https://docs.dcupl.com/sdk/queries/query-results Description: Retrieve filtered data with items(). Control result ordering with sort, limit data with pagination (start/count), and reduce payload size with projection. # Query results Retrieve filtered data with items(). Control result ordering with sort, limit data with pagination (start/count), and reduce payload size with projection. ## Syntax ```typescript list.catalog.query.items({ sort?: { attributes: string[], order: ('ASC' | 'DESC')[] }, start?: number, count?: number, projection?: { [field: string]: boolean } }) ``` ## Parameters - `sort.attributes` (`string[]`, optional) — Fields to sort by (in priority order) - `sort.order` (`('ASC' | 'DESC')[]`, optional) — Sort direction for each attribute (uppercase; other values are ignored with a warning and the result stays unsorted) - `start` (`number`, optional) — Zero-based offset for pagination - `count` (`number`, optional) — Maximum items to return - `projection` (`object`, optional) — { field: true } to return only these fields, { $: true, field: false } to exclude fields. A nested object resolves a reference: { $: true, customer: { name: true } } ## Return type Returns an array of items matching the query with applied options. ```typescript // list.catalog.query items(options?: DcuplListQueryOptions): T[]; // matching items with the requested fields ``` ## Usage notes - 'ASC' sorts ascending (A-Z, 0-9, oldest first), 'DESC' descending; items without a value sort last in both directions - Projection (key is always included): { field: true } only this field, { $: true } all fields, { $: true, field: false } all but this one, { $: true, ref: { $: true } } all fields with this reference resolved - Without count, returns ALL matching items - Total number of matches (ignoring start/count): list.catalog.fn.metadata().currentSize - Always sort before paginating for consistent results - Sort attributes and order arrays must have matching lengths; order values are 'ASC' or 'DESC' - String sorting is case-insensitive - Projection with true includes only specified fields (plus key) - Use { $: true, field: false } to exclude specific fields - Sort attributes are properties of the item: a dotted attribute such as 'customer.name' is ignored without an error. To sort by a referenced value, add a derived property - sort.script runs a custom sort script instead (see Scripts) - Conditions passed in items({ queries }) are applied on top of the list query (AND), they do not replace it. Such a call ignores the start, count, sort and projection applied with applyOptions(): pass them in the same items() call - Other ways to read results: first(), any(), count(), one({ itemKey }) and many({ itemKeys }) (Catalog API). one() and many() only return items that pass the list query - Page formula: start = (pageNumber - 1) \* pageSize - query.count() counts what items() returns, so a count applied with applyOptions() caps it; use fn.metadata().currentSize for the total ## Related methods - `catalog.fn.facets()` — Get value counts for filter UIs - `catalog.fn.aggregate()` — Get totals without fetching items ## Related guides/docs - [Basic filtering](/sdk/queries/query-filter) - [Catalog API: first(), any(), count(), one(), many()](/sdk/api/catalog#query-operations) - [Scripts (sort scripts)](/sdk/guides/scripts) - [Common mistakes](/sdk/guides/common-mistakes) ## Examples **Basic result retrieval** Get all items matching current query filters. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); // Get all matching items const results = productList.catalog.query.items(); console.log('Total items:', results.length); ``` **Pagination with count and start** Retrieve a specific page of results. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Page 1: items 0-19 const page1 = productList.catalog.query.items({ start: 0, count: 20, }); // Page 2: items 20-39 const page2 = productList.catalog.query.items({ start: 20, count: 20, }); // Page calculation helper function getPage(pageNumber: number, pageSize: number) { return productList.catalog.query.items({ start: (pageNumber - 1) * pageSize, count: pageSize, }); } ``` **Single property sort** Sort results by one field. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Sort by price ascending (lowest first) const cheapest = productList.catalog.query.items({ sort: { attributes: ['price'], order: ['ASC'], }, }); // Sort by price descending (highest first) const expensive = productList.catalog.query.items({ sort: { attributes: ['price'], order: ['DESC'], }, }); ``` **Multi-field sort** Sort by multiple fields with different directions. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Sort by category (A-Z), then price (high to low) within category const results = productList.catalog.query.items({ sort: { attributes: ['category', 'price'], order: ['ASC', 'DESC'], }, }); // Featured first, then by rating, then alphabetically const featured = productList.catalog.query.items({ sort: { attributes: ['featured', 'rating', 'name'], order: ['DESC', 'DESC', 'ASC'], }, }); ``` **Sort with pagination** Always sort before paginating for consistent results. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // IMPORTANT: Sort must be consistent across pages const pageSize = 20; const currentPage = 1; const results = productList.catalog.query.items({ sort: { attributes: ['createdAt'], order: ['DESC'], }, start: (currentPage - 1) * pageSize, count: pageSize, }); // Get total for pagination UI (all matching items, not just this page) const totalItems = productList.catalog.fn.metadata().currentSize; const totalPages = Math.ceil(totalItems / pageSize); ``` **Field projection** Select only specific fields to return. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Only return key, name, price, and thumbnail (reduces payload) const results = productList.catalog.query.items({ projection: { name: true, price: true, thumbnail: true, // All other fields excluded; key is always included }, }); // Exclude specific fields const withoutDescription = productList.catalog.query.items({ projection: { $: true, // Include all fields... description: false, // ...except this large text field }, }); ``` **Complete e-commerce listing** Full example with filters, sort, pagination, and projection. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); function getProductListing(options: { category?: string; minPrice?: number; maxPrice?: number; sortBy: 'price' | 'rating' | 'name'; sortDir: 'ASC' | 'DESC'; page: number; pageSize: number; }) { const query = productList.catalog.query; query.clear(); // Apply filters if (options.category) { query.addCondition({ attribute: 'category', operator: 'eq', value: options.category, }); } // Price range: an 'and' group (two addCondition calls on 'price' would be ORed) query.addGroup({ groupKey: 'price', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: options.minPrice ?? 0 }, { attribute: 'price', operator: 'lte', value: options.maxPrice ?? Number.MAX_SAFE_INTEGER }, ], }); // Active products only query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', }); // Sort, paginate, and project query.applyOptions({ sort: { attributes: [options.sortBy], order: [options.sortDir], }, start: (options.page - 1) * options.pageSize, count: options.pageSize, projection: { name: true, price: true, thumbnail: true, rating: true, category: true, }, }); const results = query.items(); // Total count for pagination const totalItems = productList.catalog.fn.metadata().currentSize; return { items: results, page: options.page, pageSize: options.pageSize, totalItems, totalPages: Math.ceil(totalItems / options.pageSize), }; } ``` **Sort by a referenced property** Sort attributes are read from the item itself, so pull the referenced value in with a derived property first. ```typescript // A dotted path like 'customer.name' is not a valid sort attribute. // Derive the customer's name onto the order instead: dcupl.models.set({ key: 'Order', properties: [ { key: 'total', type: 'int' }, { key: 'customerName', type: 'string', derive: { localReference: 'customer', remoteProperty: 'name' }, }, ], references: [{ key: 'customer', type: 'singleValued', model: 'Customer' }], }); // ...after dcupl.init(): const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Sort orders by customer name, then by total const results = orderList.catalog.query.items({ sort: { attributes: ['customerName', 'total'], order: ['ASC', 'DESC'], }, }); ``` **SDK** ```typescript // Get sorted, paginated results const list = dcupl.lists.create({ modelKey: 'Article' }); // Apply filter list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Running', }); // Get page 1 with sort const results = list.catalog.query.items({ sort: { attributes: ['price'], order: ['ASC'] }, start: 0, count: 10 }); // Get total count for pagination const total = list.catalog.fn.metadata().currentSize; console.log('Showing', results.length, 'of', total); ``` --- # catalog.fn.facets() Source: https://docs.dcupl.com/sdk/fn/facets Description: Get unique values with counts for building filter UIs. Returns the distinct values of an attribute with how many items match each value under the list's… # catalog.fn.facets() Get unique values with counts for building filter UIs. Returns the distinct values of an attribute with how many items match each value under the list's current query. ## Syntax ```typescript list.catalog.fn.facets(options) ``` ## Parameters - `attribute` (`string`, required) — The attribute to get facets for - `count` (`number`, optional) — Limit the number of facet values returned (applied after sort, if set) - `sort` (`'size-desc' | 'size-asc' | 'value-asc' | 'value-desc'`, optional) — Sort the values; without it they come back in index order - `excludeGroup` (`string`, optional) — Key of the query group left out when counting (default: the group keyed by attribute), found at any nesting depth. Use it when the selection for this attribute lives in a group with a custom key - `excludeZeros` (`boolean`, optional) — Exclude facets with 0 count (default: true) - `excludeUndefineds` (`boolean`, optional) — Exclude undefined values (default: true) - `calculateResults` (`boolean`, optional) — Count every matching item and return all their keys (default: true). With false each value stops at its first match: size is 0 or 1 and resultKeys has at most one key. Ignored when sort is set (sorting needs the real sizes) - `excludeUnresolved` (`boolean`, optional) — Exclude reference keys that do not resolve to an item of the referenced model (default: true) ## Return type Returns an array of DcuplFacet objects: ```typescript // Can be imported from @dcupl/common type DcuplFacet = { key: string; // The value as a string value: boolean | string | null | undefined; // The raw value (a number for numeric attributes) size: number; // Count of items with this value enabled: boolean; // size > 0 selected: boolean; // The excluded group has a condition on this value entry: ListItem; // The referenced item for a direct reference attribute; { key } for // properties and dotted paths; undefined for an unresolved or null reference key // (listed only with excludeUnresolved: false) or a reference's no-value entry // (only with excludeUndefineds: false) resultKeys: string[]; // Matching item keys children?: DcuplFacet[]; // Not set by fn.facets() } ``` ## Usage notes - Counts are disjunctive: each value is counted under every other group of the current query. The group keyed by the faceted attribute (or the one named by excludeGroup) is left out, so selecting a value does not hide its siblings, while conditions on other attributes reduce the counts - Ideal for building filter sidebars, search refinements, and data exploration - Performance: Set calculateResults: false when you only need to know which values are available (enabled), not their counts or item keys - Performance: Use count parameter to limit results for attributes with many unique values - Values with 0 matches are left out by default (excludeZeros: true); pass excludeZeros: false to show them disabled - Troubleshooting: If a facet shows only the selected value and selected is false, the selection lives in a group with a custom key. Pass that key as excludeGroup. ## Related methods - `catalog.query.addCondition()` — Apply filters based on facet selection - `catalog.query.many()` — Get the actual json values behind the resultKeys ## Related guides/docs - [How to Build Faceted Search](/sdk/advanced/faceted-search) - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Building a filter sidebar** Create a complete filter UI with facets showing available options, counts, and selected state. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); function renderFilterSidebar() { const categoryFacets = productList.catalog.fn.facets({ attribute: 'category', excludeZeros: false, // keep values that would return 0 items (rendered disabled) }); const filterHTML = categoryFacets.map((facet) => { const checkedAttr = facet.selected ? 'checked' : ''; const disabledAttr = !facet.enabled ? 'disabled' : ''; const countClass = facet.enabled ? '' : 'text-text-muted'; return ` `; }).join(''); document.getElementById('category-filters')!.innerHTML = filterHTML; } ``` **Deep property facets** Get facets from referenced models using dot notation for related data. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Facets on a property two references away (Order -> Product -> Brand). // A path that ends at a reference ('product.brand') returns the brand keys. const brandFacets = orderList.catalog.fn.facets({ attribute: 'product.brand.name', }); console.log('Brands in orders:', brandFacets.map(f => f.value)); // Facets from a multi-level reference const countryFacets = orderList.catalog.fn.facets({ attribute: 'customer.address.country', }); console.log('Customer countries:'); countryFacets.forEach((f) => { console.log(' ' + f.value + ': ' + f.size + ' orders'); }); ``` **Show unavailable options** Display all filter options including those with zero matches (grayed out). ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Show all categories, even those with 0 products matching current filters const facets = productList.catalog.fn.facets({ attribute: 'category', excludeZeros: false, // Include facets with size=0 }); facets.forEach((facet) => { const status = facet.enabled ? '✓' : '✗'; const availability = facet.size > 0 ? 'available' : 'unavailable'; console.log(status + ' ' + facet.value + ' (' + facet.size + ') - ' + availability); }); // Use case: Show "Electronics (0)" grayed out when no electronics match filters ``` **Performance optimization** Skip full counting when you only need to know which values are available, not how many items each has. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Fast: stop at the first matching item per value const fastFacets = productList.catalog.fn.facets({ attribute: 'category', calculateResults: false, }); // size is only 0 or 1 here and resultKeys holds at most one key, // so use enabled (available or not), not size fastFacets.forEach((f) => { console.log(f.value + ': ' + (f.enabled ? 'available' : 'unavailable')); }); // Full (default): real counts and every matching item key const fullFacets = productList.catalog.fn.facets({ attribute: 'category', calculateResults: true, }); fullFacets.forEach((f) => { console.log(f.value + ': ' + f.size + ' items, e.g. ' + f.resultKeys.slice(0, 3).join(', ')); }); ``` **Limiting facet results** Limit the number of facet values returned for "Top N" style filters. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Get top 5 brands by count const topBrands = productList.catalog.fn.facets({ attribute: 'brand', sort: 'size-desc', // without sort, count keeps the first 5 values in index order count: 5, }); console.log('Top 5 Brands:'); topBrands.forEach((f, i) => { console.log((i + 1) + '. ' + f.value + ' (' + f.size + ' products)'); }); // Combine with excludeZeros for populated results only const popularCategories = productList.catalog.fn.facets({ attribute: 'category', count: 10, excludeZeros: true, excludeUndefineds: true, }); ``` **Facets with active filters** Conditions on other attributes reduce the counts; values that would return 0 items stay listed with excludeZeros: false. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Initial facets (no filters) console.log('All Categories:'); let categoryFacets = productList.catalog.fn.facets({ attribute: 'category' }); categoryFacets.forEach(f => console.log(' ' + f.value + ': ' + f.size)); // Apply a filter productList.catalog.query.addCondition({ operator: 'eq', attribute: 'brand', value: 'Nike', }); // Counts now reflect the brand filter console.log('\nCategories for Nike products:'); categoryFacets = productList.catalog.fn.facets({ attribute: 'category', excludeZeros: false }); categoryFacets.forEach(f => console.log(' ' + f.value + ': ' + f.size)); // enabled=false means selecting would result in 0 items categoryFacets.forEach(f => { if (!f.enabled) { console.log(' ⚠️ ' + f.value + ' would return 0 results'); } }); ``` **Multi-select with a custom group (excludeGroup)** Counts are disjunctive: the group keyed by the faceted attribute is left out. If the selection lives in a group with another key, name it with excludeGroup. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Selection stored in a group with a custom key productList.catalog.query.addGroup({ groupKey: 'category-filter', groupType: 'or', queries: [{ operator: 'eq', attribute: 'category', value: 'Electronics' }], }); // Without excludeGroup the selection counts against itself: // only Electronics is listed, and selected is false const collapsed = productList.catalog.fn.facets({ attribute: 'category' }); // Leave the selection group out: every category keeps its count, // Electronics is marked selected const facets = productList.catalog.fn.facets({ attribute: 'category', excludeGroup: 'category-filter', }); facets.forEach((f) => { console.log((f.selected ? '[x] ' : '[ ] ') + f.value + ' (' + f.size + ')'); }); ``` **SDK** ```typescript // Get facets for the 'brand' attribute const facets = list.catalog.fn.facets({ attribute: 'brand' }); // With options const topCategories = list.catalog.fn.facets({ attribute: 'category', sort: 'size-desc', // without sort, count takes the first values in index order count: 5, excludeZeros: false }); ``` --- # catalog.fn.aggregate() Source: https://docs.dcupl.com/sdk/fn/aggregate Description: Compute numeric statistics (sum, avg) and count, min/max, distinct values and groups on any attribute, for analytics dashboards, KPI displays, and data… # catalog.fn.aggregate() Compute numeric statistics (sum, avg) and count, min/max, distinct values and groups on any attribute, for analytics dashboards, KPI displays, and data summaries. min/max compare strings lexically and skip dates. ## Syntax ```typescript list.catalog.fn.aggregate(options) ``` ## Parameters - `attribute` (`string`, required) — The attribute to aggregate (supports deep queries like "product.price") - `types` (`AggregationType[]`, required) — Array of aggregation types: "sum", "avg", "count", "min", "max", "distinct", "group" - `excludeUndefineds` (`boolean`, optional) — Leave the entry for undefined values out of distinct (default: false). sum, avg, min, max and count always skip missing values - `distinctOptions.limit` (`number`, optional) — Return at most this many distinct values: the first ones in index order, not the most frequent - `distinctOptions.skipKeys` (`boolean`, optional) — Skip returning item keys for distinct values (default: false) - `groupOptions.groups` (`AggregationGroupOption[]`, optional) — Groups for the "group" type: ranges ({ from: 0, to: 50 }, both bounds exclusive) or value lists ({ value: \["a", "b"\] }) - `groupOptions.calculateNotFound` (`boolean`, optional) — Append an entry (option.value: \["$dcupl\_not\_found"\]) counting the items that match no group ## Return type Returns an Aggregation object with the requested metrics: ```typescript // Can be imported from @dcupl/common type Aggregation = { attribute: string; types: AggregationType[]; sum?: number; // Total of all numeric values avg?: number; // Average (mean) of numeric values count?: number; // Number of non-null values (array values counted individually) min?: number; // Minimum value (numbers, strings and booleans; Date values are skipped) max?: number; // Maximum value (same types as min) distinct?: Array<{ // Unique values with counts value: string; // The raw value (a number for numeric attributes) count: number; keys?: string[]; // Omitted with distinctOptions.skipKeys }>; group?: Array<{ // One entry per groupOptions.groups entry option: { from?: any; to?: any; value?: any[] }; // The group definition count: number; keys: string[]; }>; } ``` ## Usage notes - Aggregations respect current query filters - metrics update as filters change - Use distinctOptions.limit for large datasets to limit returned distinct values - Supports deep property queries (e.g., "product.brand.rating") - Performance: Request only the aggregation types you need - each type adds computation cost - Performance: Use distinctOptions.skipKeys: true when you only need values, not item references - Performance: For attributes with many distinct values, set distinctOptions.limit to keep the result small - Groups may overlap: an item is counted in every group whose range or value list it matches - Troubleshooting: If sum/avg are undefined, no numeric value was found - verify the attribute is a numeric type. min/max are undefined when no number, string or boolean was found (Date values are skipped); count, distinct and group are always set (an empty list gives count: 0 and distinct: \[\]) - Missing values never skew sum/avg/min/max/count; excludeUndefineds only removes the undefined entry from distinct - excludeZeros, excludeUnresolved and groupOptions.limit are not aggregate() options: SDK 2.0 removed them from AggregationOptions (they never had an effect), so passing one is a type error. facets() keeps excludeZeros and excludeUnresolved ## Related methods - `catalog.fn.facets()` — Get unique values with counts for filter UIs - `catalog.fn.groupBy()` — Group items with aggregations per group - `catalog.fn.pivot()` — Create multi-dimensional pivot tables ## Related guides/docs - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Dashboard KPIs** Calculate key metrics for a dashboard including revenue, order count, average order value, and unique customers. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Filter to last 30 days const thirtyDaysAgo = new Date(); thirtyDaysAgo.setDate(thirtyDaysAgo.getDate() - 30); orderList.catalog.query.addCondition({ operator: 'gte', attribute: 'orderDate', value: thirtyDaysAgo, }); // Revenue metrics const revenue = orderList.catalog.fn.aggregate({ attribute: 'totalAmount', types: ['sum', 'avg', 'count'], }); // Customer metrics const customers = orderList.catalog.fn.aggregate({ attribute: 'customer', types: ['distinct'], }); console.log('Total Revenue:', revenue.sum); console.log('Orders:', revenue.count); console.log('Avg Order Value:', revenue.avg); // count, distinct and group are always set; sum and avg are undefined // when no numeric value matched (e.g. no orders in the window) const uniqueCustomers = customers.distinct!.length; console.log('Unique Customers:', uniqueCustomers); console.log('Revenue per Customer:', uniqueCustomers ? (revenue.sum ?? 0) / uniqueCustomers : 0); ``` **Customer lifetime value** Sum the orders per customer in one call with groupBy() and its aggregates, instead of one filtered aggregate() per customer. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // One group per customer, with the sum of its orders const byCustomer = orderList.catalog.fn.groupBy({ attribute: 'customer', aggregates: [{ attribute: 'totalAmount', types: ['sum'] }], }); const customerLTV = byCustomer.items.map((group) => ({ customer: group.key, ltv: group._aggregates[0]?.sum ?? 0, orders: group._meta.size, })); // Sort by LTV descending customerLTV.sort((a, b) => b.ltv - a.ltv); ``` **Discount analysis** Analyze discounts given across orders, calculate total discount amount and effective discount rate. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Filter to orders with discounts orderList.catalog.query.addCondition({ operator: 'gt', attribute: 'discountAmount', value: 0, }); const discountAgg = orderList.catalog.fn.aggregate({ attribute: 'discountAmount', types: ['sum', 'avg', 'count'], }); const revenueAgg = orderList.catalog.fn.aggregate({ attribute: 'totalAmount', types: ['sum'], }); console.log('Orders with Discount:', discountAgg.count); console.log('Total Discounts Given:', discountAgg.sum); console.log('Avg Discount per Order:', discountAgg.avg); console.log('Revenue (after discount):', revenueAgg.sum); // sum is undefined when no order matched, so default to 0 const discounts = discountAgg.sum ?? 0; const gross = (revenueAgg.sum ?? 0) + discounts; console.log('Discount Rate:', gross ? (discounts / gross) * 100 : 0, '%'); ``` **Deep property aggregation** Aggregate values from referenced models using dot notation for deep property access. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Aggregate on a deep property (product's brand rating) const brandRatings = orderList.catalog.fn.aggregate({ attribute: 'product.brand.rating', types: ['avg', 'min', 'max'], }); console.log('Brand Ratings (from orders):'); console.log(' Average:', brandRatings.avg); console.log(' Lowest:', brandRatings.min); console.log(' Highest:', brandRatings.max); // Distinct brands ordered (limit keeps the first 10 in index order, not the 10 most frequent) const distinctBrands = orderList.catalog.fn.aggregate({ attribute: 'product.brand.name', types: ['distinct'], distinctOptions: { limit: 10 }, }); console.log('Brands Ordered (first 10):', distinctBrands.distinct!.map(b => b.value)); ``` **Price bands with group** Count items per custom range. Both range bounds are exclusive: a price of exactly 100 is in neither band below. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const bands = productList.catalog.fn.aggregate({ attribute: 'price', types: ['group'], groupOptions: { groups: [ { from: 0, to: 100 }, // 0 < price < 100 { from: 100, to: 500 }, // 100 < price < 500 ], calculateNotFound: true, // extra entry for items in no band }, }); bands.group?.forEach((g) => { const label = g.option.value ? 'other' : g.option.from + ' < price < ' + g.option.to; console.log(label + ': ' + g.count + ' products'); }); ``` **SDK** ```typescript // Get all numeric stats for price const stats = list.catalog.fn.aggregate({ attribute: 'price', types: ['sum', 'avg', 'min', 'max', 'count'] }); console.log('Sum:', stats.sum); console.log('Average:', stats.avg); console.log('Min:', stats.min); console.log('Max:', stats.max); console.log('Count:', stats.count); ``` --- # catalog.fn.suggest() Source: https://docs.dcupl.com/sdk/fn/suggest Description: Provides autocomplete suggestions for an attribute. Returns the distinct values that match the user input, with the keys of the items that have each value… # catalog.fn.suggest() Provides autocomplete suggestions for an attribute. Returns the distinct values that match the user input, with the keys of the items that have each value, for search-as-you-type and autocomplete. ## Syntax ```typescript list.catalog.fn.suggest(options) ``` ## Parameters - `attribute` (`string`, required) — The attribute to get suggestions for. A dotted path such as 'brand.name' resolves through the reference - `value` (`string | number`, required) — The search term. A bare string is a case-insensitive prefix match; wrap it in slashes ('/pattern/') for a regular expression; '' returns every value (up to max) - `regexFlags` (`string`, optional) — Flags for a '/pattern/' value, e.g. 'i' for case-insensitive. Ignored for any other value; invalid flags return no suggestions - `max` (`number`, optional) — Maximum number of suggestions to return (default: 20) - `transform` (`('lowercase' | 'removeWhitespace' | 'trim')[]`, optional) — Normalize values before matching - `calculateResultKeys` (`boolean`, optional) — Include item keys in results (default: true) - `excludeUndefineds` (`boolean`, optional) — Exclude undefined values (default: true) - `excludeNulls` (`boolean`, optional) — Exclude null values (default: true) - `relevantData` (`'filtered' | 'all' | 'excludeQuery'`, optional) — Which items to suggest from: 'filtered' (default, the list's current result), 'all' (ignore the query), or 'excludeQuery' (the current query without excludeQuery) - `excludeQuery` (`Partial`, optional) — Condition or group (e.g. { groupKey: 'category' }) to leave out of the query when relevantData is 'excludeQuery' ## Return type Returns an array of Suggestion objects: ```typescript // Can be imported from @dcupl/common type Suggestion = { value: string; // The raw value: typed string; at runtime a number or boolean for numeric and boolean attributes, the ISO string for dates keys: string[]; // Item keys with this value ([] if calculateResultKeys: false) } ``` ## Usage notes - A bare string matches the start of the value, ignoring case: 'lap' suggests "Laptop Pro" but not "Gaming Laptop" - Wrap the term in slashes ('/pro/') to match it anywhere; the pattern is case-sensitive unless you set regexFlags: 'i' (flags never go after the closing slash: '/pro/i' is not a regex, just a plain prefix). Escape regex characters in user input - Empty search term returns all unique values (up to max) - Dotted paths work: attribute: "brand.name" suggests the names of the referenced brands, with the keys of the items that reference them - Suggestions respect current query filters by default (relevantData: "filtered") - Use transform options for flexible matching (lowercase, removeWhitespace, trim) - Performance: Set calculateResultKeys: false when you only need values, not item references - Performance: Use max parameter to limit results - default is 20 - Performance: Avoid regex patterns for simple prefix matching - prefix is faster - Troubleshooting: Use relevantData: "all" if you want suggestions regardless of current filters ## Related methods - `catalog.fn.facets()` — Get all unique values with counts for building filters - `catalog.query.addCondition()` — Apply filters based on selected suggestion - `catalog.query.many()` — Get the items behind the result keys ## Related guides/docs - [Search & autocomplete](/sdk/advanced/search-autocomplete) - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Product search autocomplete** Basic autocomplete implementation with prefix matching and transform options. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); function handleSearchInput(inputValue: string) { const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: inputValue, max: 10, transform: ['lowercase', 'trim'], }); // Render suggestions const dropdown = document.getElementById('suggestions')!; dropdown.innerHTML = suggestions .map((s) => '
' + s.value + ' (' + s.keys.length + ')
') .join(''); } // Attach to input document.getElementById('search')!.addEventListener('input', (e) => { handleSearchInput((e.target as HTMLInputElement).value); }); ``` **Category filter suggestions** Get all category values for a filter dropdown, showing counts based on current filters. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // An empty search term returns every value (up to max) const categories = productList.catalog.fn.suggest({ attribute: 'category', value: '', // Empty = all values max: 50, // default is 20 relevantData: 'filtered', // Respect current filters }); // Build filter options with counts const filterOptions = categories.map((s) => ({ label: s.value, count: s.keys.length, })); console.log('Available Categories:'); filterOptions.forEach((opt) => { console.log(' ' + opt.label + ' (' + opt.count + ')'); }); ``` **Brand search with icons** Enrich suggestions with additional metadata like counts and custom icons. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Brand icon mapping const brandIcons: Record = { Nike: '👟', Apple: '🍎', Samsung: '📱', Sony: '🎮', }; function searchBrands(query: string) { const suggestions = productList.catalog.fn.suggest({ attribute: 'brand', value: query, max: 8, transform: ['lowercase'], }); return suggestions.map((s) => ({ brand: s.value, count: s.keys.length, icon: brandIcons[s.value] || '🏷️', })); } const results = searchBrands('s'); results.forEach((r) => { console.log(r.icon + ' ' + r.brand + ' (' + r.count + ' products)'); }); ``` **Color suggestions with mapping** Map color suggestions to hex codes or other visual representations. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const colorHex: Record = { red: '#FF0000', blue: '#0000FF', green: '#00FF00', black: '#000000', white: '#FFFFFF', }; function getColorSuggestions(query: string) { const suggestions = productList.catalog.fn.suggest({ attribute: 'color', value: query, transform: ['lowercase'], }); return suggestions.map((s) => ({ name: s.value, hex: colorHex[s.value.toLowerCase()] || '#CCCCCC', count: s.keys.length, })); } const colors = getColorSuggestions('bl'); colors.forEach((c) => { console.log(c.hex + ' ' + c.name + ' (' + c.count + ')'); }); ``` **Country suggestions with flags** Enrich country suggestions with flag emojis for visual display. ```typescript const customerList = dcupl.lists.create({ modelKey: 'Customer' }); const countryFlags: Record = { 'United States': '🇺🇸', Germany: '🇩🇪', Japan: '🇯🇵', France: '🇫🇷', 'United Kingdom': '🇬🇧', }; function searchCountries(query: string) { const suggestions = customerList.catalog.fn.suggest({ attribute: 'country', value: query, max: 10, transform: ['trim'], }); return suggestions.map((s) => ({ country: s.value, flag: countryFlags[s.value] || '🏳️', customers: s.keys.length, })); } const results = searchCountries('un'); results.forEach((r) => { console.log(r.flag + ' ' + r.country + ' - ' + r.customers + ' customers'); }); ``` **Course title with full item lookup** Use suggestion keys to fetch complete item details for rich autocomplete. ```typescript const courseList = dcupl.lists.create({ modelKey: 'Course' }); // suggest() and query.one() are synchronous function searchCourses(query: string) { const suggestions = courseList.catalog.fn.suggest({ attribute: 'title', value: query, max: 5, calculateResultKeys: true, }); // Look up the first matching course of each suggestion return suggestions.map((s) => { const course = courseList.catalog.query.one({ itemKey: s.keys[0] }); return { title: s.value, instructor: course?.instructor, duration: course?.duration, rating: course?.rating, matchCount: s.keys.length, }; }); } const results = searchCourses('java'); results.forEach((c) => { console.log(c.title); console.log(' By: ' + c.instructor + ' | ' + c.duration + 'h | ⭐' + c.rating); }); ``` **Hashtag suggestions** Handle special characters in input by preprocessing before suggestion. ```typescript const postList = dcupl.lists.create({ modelKey: 'Post' }); function suggestHashtags(input: string) { // Remove # prefix if present const query = input.startsWith('#') ? input.slice(1) : input; const suggestions = postList.catalog.fn.suggest({ attribute: 'hashtags', value: query, max: 10, transform: ['lowercase', 'removeWhitespace'], }); return suggestions.map((s) => ({ tag: '#' + s.value, posts: s.keys.length, })); } const tags = suggestHashtags('#java'); tags.forEach((t) => { console.log(t.tag + ' (' + t.posts + ' posts)'); }); ``` **Campaign code suggestions (uppercase)** Prefix matching ignores case, so lowercase input finds uppercase codes without any transform. ```typescript const campaignList = dcupl.lists.create({ modelKey: 'Campaign' }); function suggestCampaignCodes(input: string) { // A bare string is a case-insensitive prefix match const suggestions = campaignList.catalog.fn.suggest({ attribute: 'code', value: input, max: 10, }); return suggestions.map((s) => ({ code: s.value, usageCount: s.keys.length, })); } const codes = suggestCampaignCodes('sum'); codes.forEach((c) => { console.log(c.code + ' (used ' + c.usageCount + ' times)'); }); // Typing "sum" matches codes like SUMMER2024 and SUMMERSALE ``` **Substring suggestions (regex + regexFlags)** Wrap the term in slashes to match it anywhere in the value. The pattern is case-sensitive; set regexFlags for flags. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const escapeRegExp = (text: string) => text.replace(/[.*+?^${}()|[\]\\]/g, (char) => '\\' + char); function suggestContaining(input: string) { return productList.catalog.fn.suggest({ attribute: 'name', value: `/${escapeRegExp(input.trim())}/`, regexFlags: 'i', // flags go here: '/phone/i' is not a regex (plain prefix text) max: 10, }); } // '/phone/' matches "Smartphone Case"; with regexFlags 'i' also "Galaxy Phone" const results = suggestContaining('phone'); console.log(results.map((s) => s.value)); ``` **SDK** ```typescript // Basic suggestions for productName attribute const suggestions = list.catalog.fn.suggest({ attribute: 'productName', value: '/pro/', // regex for 'pro' anywhere in the value max: 10, transform: ['lowercase'], }); ``` --- # catalog.fn.groupBy() Source: https://docs.dcupl.com/sdk/fn/group-by Description: Group items by an attribute value and compute aggregations for each group. Perfect for creating categorized lists, calculating group statistics, and building… # catalog.fn.groupBy() Group items by an attribute value and compute aggregations for each group. Perfect for creating categorized lists, calculating group statistics, and building hierarchical data displays. ## Syntax ```typescript list.catalog.fn.groupBy(options) ``` ## Parameters - `attribute` (`string`, required) — The attribute to group items by - `aggregates` (`AggregationOptions[]`, optional) — Aggregations to compute per group: any AggregationOptions, as in aggregate() (including distinct and group). A dotted attribute such as customer.rating resolves through the reference - `items` (`DcuplSectionItemOption[]`, optional) — Which groups include their items: { key: "\*" } for all, or one entry per group key (compared with the raw value). Each entry takes start, count, sort and projection for that group's items - `start / count` (`number`, optional) — Page through the groups - `sort` (`{ attributes: string[], order: ("ASC" | "DESC")[] }`, optional) — Sort the items inside each group. When grouping by a reference, it also sorts the groups by the referenced model's attributes; for a plain attribute, the groups follow the order of their first item under this sort. Only properties can be sort attributes (references are ignored). Order values must be uppercase - `projection` (`Projection`, optional) — Projection for the group entries when grouping by a reference (for the items inside a group, set projection on the items entry). Without $: true, \_meta is dropped from the groups; \_aggregates and \_items stay when you request them. Ignored when grouping by a plain property ## Return type Returns a GroupByResponse object: ```typescript // Can be imported from @dcupl/common type GroupByResponse = { _meta: { currentSize: number; // Number of groups (before start/count) initialSize: number; // Number of groups (before start/count) }; items: SectionItem[]; // Array of groups } type SectionItem = { key: string | number | boolean | null; // Group identifier: the raw attribute value // (declared as string in @dcupl/common) value?: any; // The attribute value (when grouping by a reference: // the referenced item's properties instead) _meta: { size: number }; // Number of items in this group _aggregates: Aggregation[]; // One per entry of options.aggregates, same order _items: ListItem[]; // Items of the group ([] unless requested via items) } ``` ## Usage notes - Groups respect current query filters - group counts update as filters change - Use aggregates to compute statistics like sum, avg, min, max per group - Items are only included if explicitly requested via the items option - Supports deep queries for grouping by related model attributes (e.g., "category.parent") - Performance: Only request items when needed - use { key: "specific-key" } instead of { key: "\*" } - Performance: Limit items per group with count parameter to avoid loading thousands of items - Performance: Use projection to return only needed properties for items - Items whose value for the grouping attribute is undefined are not part of any group (null values form a group with the key null), and a group without items never appears. When grouping by a reference, a key that does not resolve still forms its own group - There are no exclude\* options: SDK 2.0 removed excludeZeros, excludeUndefineds and excludeUnresolved from DcuplGroupByOptions (groupBy() never read them), so passing one is a type error - Troubleshooting: If \_items stays empty for a specific key, check the type: items\[\].key is compared with the raw value, so "5" does not match a numeric 5 ## Related methods - `catalog.fn.facets()` — Get unique values with counts for building filter UIs - `catalog.fn.aggregate()` — Compute aggregations across all items - `catalog.fn.pivot()` — Create multi-dimensional pivot tables ## Related guides/docs - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Product catalog by category** Group products by category with multiple aggregations and include actual items with pagination. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const grouped = productList.catalog.fn.groupBy({ attribute: 'category', aggregates: [ { attribute: 'price', types: ['avg', 'min', 'max'] }, { attribute: 'rating', types: ['avg'] }, ], items: [{ key: '*', count: 5 }], // Include first 5 items per group }); grouped.items.forEach((group) => { console.log(group.key + ':'); console.log(' Products:', group._meta.size); console.log(' Avg Price:', group._aggregates[0].avg); console.log(' Price Range:', group._aggregates[0].min, '-', group._aggregates[0].max); console.log(' Avg Rating:', group._aggregates[1].avg); console.log(' Sample Products:', group._items.map(p => p.name).join(', ')); }); ``` **Sales by region with sorting** Group orders by region, calculate revenue metrics, and sort by aggregated values. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Filter to this year orderList.catalog.query.addCondition({ operator: 'eq', attribute: 'year', value: 2024, }); const grouped = orderList.catalog.fn.groupBy({ attribute: 'region', aggregates: [ { attribute: 'totalAmount', types: ['sum', 'avg', 'count'] }, { attribute: 'customer', types: ['distinct'] }, ], }); // Sort by revenue (sum) descending const sorted = grouped.items.sort( (a, b) => b._aggregates[0].sum! - a._aggregates[0].sum! ); sorted.forEach((group) => { console.log(group.key + ':'); console.log(' Revenue:', group._aggregates[0].sum); console.log(' Orders:', group._aggregates[0].count); console.log(' Avg Order:', group._aggregates[0].avg); console.log(' Unique Customers:', group._aggregates[1].distinct!.length); }); ``` **Tasks by project with selective items** Group tasks by project and load items only for the projects you name in items. ```typescript const taskList = dcupl.lists.create({ modelKey: 'Task' }); const grouped = taskList.catalog.fn.groupBy({ attribute: 'project', aggregates: [ { attribute: 'estimatedHours', types: ['sum'] }, { attribute: 'status', types: ['distinct'] }, ], // Items are loaded only for these groups; each entry takes start, count, sort, projection items: [ { key: 'Project Alpha', count: 10 }, { key: 'Project Beta', count: 10 }, ], }); grouped.items.forEach((group) => { const statusBreakdown = group._aggregates[1].distinct ?.map(s => s.value + ': ' + s.count) .join(', '); console.log(group.key + ':'); console.log(' Tasks:', group._meta.size); console.log(' Total Hours:', group._aggregates[0].sum); console.log(' Status:', statusBreakdown); if (group._items?.length > 0) { console.log(' First Tasks:', group._items.map(t => t.name).join(', ')); } }); ``` **Content by tag (multi-valued)** Group articles by tags (multi-valued attribute) with sorting and item inclusion. ```typescript const articleList = dcupl.lists.create({ modelKey: 'Article' }); const grouped = articleList.catalog.fn.groupBy({ attribute: 'tags', // Multi-valued attribute aggregates: [ { attribute: 'views', types: ['sum', 'avg'] }, ], items: [{ key: '*', count: 3, sort: { attributes: ['views'], order: ['DESC'] }, }], }); // Sort groups by total views const sorted = grouped.items.sort( (a, b) => b._aggregates[0].sum! - a._aggregates[0].sum! ); sorted.slice(0, 10).forEach((group) => { console.log('#' + group.key + ':'); console.log(' Articles:', group._meta.size); console.log(' Total Views:', group._aggregates[0].sum); console.log(' Avg Views:', group._aggregates[0].avg); console.log(' Top Articles:', group._items.map(a => a.title).join(', ')); }); ``` **Orders by customer segment (deep property)** Group orders by customer segment using deep property access with dot notation. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); const grouped = orderList.catalog.fn.groupBy({ attribute: 'customer.segment', // Deep property aggregates: [ { attribute: 'totalAmount', types: ['sum', 'avg'] }, { attribute: 'customer', types: ['distinct'] }, ], }); grouped.items.forEach((group) => { const uniqueCustomers = group._aggregates[1].distinct!.length; const revenuePerCustomer = group._aggregates[0].sum! / uniqueCustomers; console.log('Segment: ' + group.key); console.log(' Revenue:', group._aggregates[0].sum); console.log(' Avg Order:', group._aggregates[0].avg); console.log(' Customers:', uniqueCustomers); console.log(' Revenue/Customer:', revenuePerCustomer.toFixed(2)); }); ``` **Campaigns by channel with KPIs** Group marketing campaigns by channel and calculate conversion metrics from aggregations. ```typescript const campaignList = dcupl.lists.create({ modelKey: 'Campaign' }); const grouped = campaignList.catalog.fn.groupBy({ attribute: 'channel', aggregates: [ { attribute: 'impressions', types: ['sum'] }, { attribute: 'clicks', types: ['sum'] }, { attribute: 'conversions', types: ['sum'] }, { attribute: 'spend', types: ['sum'] }, ], }); grouped.items.forEach((group) => { const impressions = group._aggregates[0].sum!; const clicks = group._aggregates[1].sum!; const conversions = group._aggregates[2].sum!; const spend = group._aggregates[3].sum!; const ctr = (clicks / impressions) * 100; const conversionRate = (conversions / clicks) * 100; const cpc = spend / clicks; console.log(group.key + ':'); console.log(' Campaigns:', group._meta.size); console.log(' CTR:', ctr.toFixed(2) + '%'); console.log(' Conversion Rate:', conversionRate.toFixed(2) + '%'); console.log(' CPC:', '$' + cpc.toFixed(2)); }); ``` **Feedback by rating with sample items** Group feedback by a numeric rating and show sample comments for low ratings. items\[\].key must equal the raw value, so a string key like "1" does not match a numeric rating; use "\*" with a count instead. ```typescript const feedbackList = dcupl.lists.create({ modelKey: 'Feedback' }); const grouped = feedbackList.catalog.fn.groupBy({ attribute: 'rating', items: [{ key: '*', count: 5 }], // Up to 5 items per rating }); // Sort by rating value const sorted = grouped.items.sort( (a, b) => Number(a.key) - Number(b.key) ); sorted.forEach((group) => { console.log(group.key + ' Star:'); console.log(' Count:', group._meta.size); if (Number(group.key) <= 2) { console.log(' Sample Comments:'); group._items.forEach(item => { console.log(' -', item.comment.substring(0, 50) + '...'); }); } }); ``` **Products low on stock by category** Pre-filter to low-stock products, then group by category to identify inventory issues. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Filter to low stock items productList.catalog.query.addCondition({ operator: 'lte', attribute: 'stock', value: 10, }); const grouped = productList.catalog.fn.groupBy({ attribute: 'category', aggregates: [ { attribute: 'stock', types: ['sum'] }, { attribute: 'price', types: ['sum'] }, // Value at risk ], items: [{ key: '*', count: 3 }], }); console.log('Low Stock Alert:'); grouped.items.forEach((group) => { const valueAtRisk = group._aggregates[1].sum!; console.log(group.key + ':'); console.log(' Products:', group._meta.size); console.log(' Total Stock:', group._aggregates[0].sum); console.log(' Value at Risk:', '$' + valueAtRisk.toFixed(2)); console.log(' Items:', group._items.map(p => p.name + ' (' + p.stock + ')').join(', ')); }); ``` **SDK** ```typescript // Group articles by category const grouped = list.catalog.fn.groupBy({ attribute: 'category' }); // With aggregations per group const groupedWithStats = list.catalog.fn.groupBy({ attribute: 'category', aggregates: [ { attribute: 'price', types: ['sum', 'avg'] } ] }); // Include items in each group const withItems = list.catalog.fn.groupBy({ attribute: 'category', items: [{ key: '*' }] }); ``` --- # catalog.fn.pivot() Source: https://docs.dcupl.com/sdk/fn/pivot Description: Create multi-dimensional pivot tables for cross-tabulation analysis. Organize data into rows and columns with aggregated values at each intersection. # catalog.fn.pivot() Create multi-dimensional pivot tables for cross-tabulation analysis. Organize data into rows and columns with aggregated values at each intersection. ## Syntax ```typescript list.catalog.fn.pivot(options) ``` ## Parameters - `rows` (`Array<{ attribute: string; calculateTotals?: boolean }>`, required) — Row dimensions; several entries nest (first = outermost); \[\] for none. calculateTotals: false leaves the row totals empty (first row dimension only) - `columns` (`Array<{ attribute: string; calculateTotals?: boolean }>`, required) — Column dimensions (pass \[\] for none; without columns the result is { key: "invalid\_options" }). calculateTotals: false leaves the column totals empty - `values` (`AggregationOptions[]`, required) — Aggregations ({ attribute, types, ... } as in aggregate()) computed for every row, column and cell. 'distinct' counts and keys only the items of that row, column or cell. A dotted value attribute (customer.rating) resolves through the reference - `options.calculateTotals` (`boolean`, optional) — Compute the grand totals in pivot.values (default: true) ## Return type Returns a nested PivotResponse. Each dimension adds a level: pivot.rows\[0\] is the first row dimension, its rows are the row values, and in each row columns\[0\].columns are the cells: ```typescript // PivotResponse can be imported from @dcupl/common. PivotResponseItem is not exported: // type PivotResponseItem = NonNullable[number]; type PivotResponse = { key: string; // 'root' values?: Aggregation[]; // Grand totals ([] with options.calculateTotals: false) rows?: PivotResponseItem[]; // [0] = first row dimension columns?: Omit[]; // [0] = first column dimension (column totals) } type PivotResponseItem = { key: string; // Dimension level: the attribute; value level: the raw value // (a number for a numeric attribute, e.g. 2024) value?: any; // The row/column value (value level only) values?: Aggregation[]; // Aggregations for this row, column or cell rows?: PivotResponseItem[]; // Dimension level: one entry per value; // value level: the next row dimension columns?: Omit[]; // Dimension level: one entry per value; // row value: the first column dimension with its cells; // column value: the next column dimension (never rows) } // Example: rows [region], columns [product] // pivot.rows[0] -> { key: 'region', rows: [...] } // pivot.rows[0].rows[0] -> { key: 'North', value: 'North', values, columns } // pivot.rows[0].rows[0].columns[0].columns[0] -> cell { key, value, values } ``` ## Usage notes - Pivot tables respect current query filters - results update as filters change - Supports multiple row and column dimensions for hierarchical grouping - With a second column dimension, each cell of the first one nests the cells of the second in its columns\[0\].columns, forming one column tree A > B; column cells never carry a rows entry - Items without a value for a row or column dimension are collected in extra entries: one whose key and value are undefined and, for null values, one whose key and value are null. Like any other entry, each carries values plus its cells and nested dimensions. Skip both (key != null) when you render the table. Items whose array value is empty (\[\]) appear in no entry of that dimension and count only in the totals - Dotted paths work for row and column dimensions (e.g., "product.category" or "customer.region") - Set options.calculateTotals: false to skip the grand totals, or calculateTotals: false on the first row dimension or the first column dimension to skip its totals (cells are always computed) - Ideal for sales reports, time-based analysis, and cross-tabulations - Performance: Limit row/column dimensions - each dimension multiplies result size - Performance: Pre-filter data before pivoting to reduce the dataset - Performance: Request only needed aggregation types in values array - Combinations without items are left out: a row only lists the column values that occur in it, so look cells up by value instead of by index ## Related methods - `catalog.fn.groupBy()` — Simpler single-dimension grouping - `catalog.fn.aggregate()` — Compute aggregations without grouping - `catalog.fn.facets()` — Get unique values before pivoting ## Related guides/docs - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Sales by region and product** Classic pivot table with regions as rows, products as columns, and revenue/count as values. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); const pivot = orderList.catalog.fn.pivot({ rows: [{ attribute: 'region' }], columns: [{ attribute: 'product' }], values: [ { attribute: 'totalAmount', types: ['sum'] }, { attribute: 'key', types: ['count'] }, ], }); // pivot.rows[0] is the 'region' dimension; its rows are the regions. // In each region, columns[0] is the 'product' dimension; its columns are the cells. // Entries without a key (undefined or null) collect the orders without a region or product: skip them. const regions = (pivot.rows?.[0]?.rows ?? []).filter((row) => row.key != null); regions.forEach((row) => { console.log('Region: ' + row.value); const cells = (row.columns?.[0]?.columns ?? []).filter((col) => col.key != null); cells.forEach((col) => { const revenue = col.values?.[0]?.sum ?? 0; const orders = col.values?.[1]?.count ?? 0; console.log(' ' + col.value + ': $' + revenue + ' (' + orders + ' orders)'); }); }); // Grand totals console.log('Grand Total:', pivot.values?.[0]?.sum); ``` **Navigating pivot results** Each dimension adds a level to the result. A row only has cells for the column values that occur in it, so look cells up by value, not by index. ```typescript const pivot = list.catalog.fn.pivot({ rows: [{ attribute: 'category' }], columns: [{ attribute: 'year' }], values: [{ attribute: 'revenue', types: ['sum'] }], }); // Grand total over all data const grandTotal = pivot.values?.[0]?.sum; console.log('Grand Total:', grandTotal); // Column headers: pivot.columns[0] is the 'year' dimension (its entries carry the column totals). // Entries without a key (undefined or null) collect the items without a value: skip them. const years = (pivot.columns?.[0]?.columns ?? []) .filter((col) => col.key != null) .map((col) => col.value); console.log('Columns:', years); // Rows: pivot.rows[0] is the 'category' dimension const categories = (pivot.rows?.[0]?.rows ?? []).filter((row) => row.key != null); // Extract data for chart visualization const chartData = categories.map((row) => { const cells = row.columns?.[0]?.columns ?? []; return { category: row.value, rowTotal: row.values?.[0]?.sum ?? 0, byYear: Object.fromEntries( years.map((year) => [year, cells.find((cell) => cell.value === year)?.values?.[0]?.sum ?? 0]) ), }; }); console.log(chartData); // Access a specific cell: first category, second year const cell = categories[0]?.columns?.[0]?.columns?.find((c) => c.value === years[1]); console.log('First category, second year:', cell?.values?.[0]?.sum ?? 0); ``` **Two column dimensions** A second column dimension nests in the first: each channel cell holds its status cells, so the columns form one tree, channel > status. Column cells never hold rows. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); const pivot = orderList.catalog.fn.pivot({ rows: [{ attribute: 'region' }], columns: [{ attribute: 'channel' }, { attribute: 'status' }], values: [{ attribute: 'totalAmount', types: ['sum'] }], }); // The cells of an item's next column dimension, without the entries for missing values const cellsOf = (item) => (item?.columns?.[0]?.columns ?? []).filter((col) => col.key != null); const regions = (pivot.rows?.[0]?.rows ?? []).filter((row) => row.key != null); regions.forEach((row) => { console.log(row.value + ': ' + row.values?.[0]?.sum); cellsOf(row).forEach((channel) => { console.log(' ' + channel.value + ': ' + channel.values?.[0]?.sum); cellsOf(channel).forEach((status) => { console.log(' ' + status.value + ': ' + status.values?.[0]?.sum); }); }); }); // pivot.columns holds the same channel > status tree over all regions: the column totals const web = cellsOf(pivot).find((col) => col.value === 'web'); const webOpen = cellsOf(web).find((col) => col.value === 'open'); console.log('All regions, web > open: ' + webOpen?.values?.[0]?.sum); // For the orders North/web/open 100, North/web/closed 200, North/store/closed 50 // and South/web/open 300, this prints: // North: 350 // web: 300 // open: 100 // closed: 200 // store: 50 // closed: 50 // South: 300 // web: 300 // open: 300 // All regions, web > open: 400 ``` **Deep property pivot** Pivot using referenced model properties via dot notation for cross-model analysis. ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); // Pivot by customer country and product brand (deep properties) const pivot = orderList.catalog.fn.pivot({ rows: [{ attribute: 'customer.country' }], columns: [{ attribute: 'product.brand.name' }], values: [ { attribute: 'totalAmount', types: ['sum'] }, { attribute: 'key', types: ['count'] }, ], }); console.log('Sales by Country and Brand:'); (pivot.rows?.[0]?.rows ?? []).forEach((row) => { console.log('\n' + row.value + ':'); (row.columns?.[0]?.columns ?? []).forEach((col) => { const revenue = col.values?.[0]?.sum ?? 0; const orders = col.values?.[1]?.count ?? 0; if (revenue > 0) { console.log(' ' + col.value + ': $' + revenue + ' (' + orders + ' orders)'); } }); }); ``` **SDK** ```typescript // Basic pivot: category rows × brand columns const pivot = list.catalog.fn.pivot({ rows: [{ attribute: 'category' }], columns: [{ attribute: 'brand' }], values: [{ attribute: 'key', types: ['count'] }] }); // Multi-dimensional pivot with multiple aggregations const salesPivot = list.catalog.fn.pivot({ rows: [{ attribute: 'sport' }, { attribute: 'category' }], columns: [{ attribute: 'brand' }], values: [ { attribute: 'price', types: ['sum', 'avg'] }, { attribute: 'inStock', types: ['sum'] } ] }); ``` --- # catalog.fn.metadata() Source: https://docs.dcupl.com/sdk/fn/metadata Description: Returns metadata about the list and its data. Provides information about the model, current query state, item counts, available attributes, and views. # catalog.fn.metadata() Returns metadata about the list and its data. Provides information about the model, current query state, item counts, available attributes, and views. ## Syntax ```typescript list.catalog.fn.metadata() ``` ## Return type Returns a ListMetadata object: ```typescript // Can be imported from @dcupl/common type ListMetadata = { key: string; // Model key currentSize: number; // Items matching the current query (ignores start/count) initialSize: number; // Items in the list attributes: ModelAttribute[]; // Model properties and references views: ViewDefinition[]; // Views defined for the model appliedQuery: DcuplGlobalQueryOptions; // Current query, including sort/start/count } ``` ## Usage notes - Use currentSize vs initialSize to show "X of Y items" in filter UIs - currentSize is the total number of matches even when start/count are applied; query.count() returns the length of the current page then - attributes array describes all available fields for filtering and display - appliedQuery reflects the current filter state for debugging or persistence - No parameters required - returns metadata for the list in its current state ## Related methods - `catalog.query.count()` — Get count of matching items (respects applied start/count) - `catalog.fn.facets()` — Get filter options for an attribute - `catalog.query.current()` — Get current query state ## Related guides/docs - [Functions API reference: global dcupl.fn.\* form and errors](/sdk/api/functions) ## Examples **Building a pagination component** Calculate current page, total pages, and page range using metadata for custom pagination UI. ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Set page size productList.catalog.query.applyOptions({ count: 20, start: 0 }); function getPaginationInfo() { const meta = productList.catalog.fn.metadata(); const { currentSize, appliedQuery } = meta; const { start = 0, count = 20 } = appliedQuery; const totalPages = Math.ceil(currentSize / count); const currentPage = Math.floor(start / count) + 1; const from = start + 1; const to = Math.min(start + count, currentSize); return { currentPage, totalPages, pageSize: count, totalItems: currentSize, from, to, hasPrevious: currentPage > 1, hasNext: currentPage < totalPages, }; } // Navigate to a specific page function goToPage(page: number) { const meta = productList.catalog.fn.metadata(); const count = meta.appliedQuery.count || 20; const start = (page - 1) * count; productList.catalog.query.applyOptions({ start, count }); } // Usage const info = getPaginationInfo(); console.log(`Page ${info.currentPage} of ${info.totalPages}`); console.log(`Showing ${info.from}-${info.to} of ${info.totalItems}`); ``` **SDK** ```typescript // Get list metadata const meta = list.catalog.fn.metadata(); console.log(meta.key); // Model key console.log(meta.currentSize); // Filtered count console.log(meta.initialSize); // Total count console.log(meta.attributes); // Model attributes console.log(meta.appliedQuery); // Current query ``` --- # Derived properties Source: https://docs.dcupl.com/sdk/advanced/derived-properties Description: Automatically compute property values from references # Derived properties Derived properties automatically populate values from referenced models, enabling denormalization and computed fields without manual data duplication. They're perfect for pulling data across relationships to improve query performance and simplify data access. ## What are derived properties? A derived property is a computed field that automatically extracts values from a referenced model and stores them locally. Think of it as automatic denormalization. **Benefits:** - Query local properties instead of deep queries - Faster filtering and searching - Simplified data access - Automatic updates when references change ```typescript { key: 'customerName', type: 'string', derive: { localReference: 'customer', // Navigate through this reference remoteProperty: 'name', // Get this property value }, } ``` **How it works:** ``` Order (key: 'o1') ├─ customer: 'c1' // Reference to Customer └─ customerName: 'Alice' // Derived from customer.name ``` ## Reference | `derive` key | Meaning | | ---------------- | -------------------------------------------------------------------------------------------------------- | | `localReference` | A reference of this model (single- or multi-valued). If it doesn't exist: `MissingReference` error | | `remoteProperty` | A property of the referenced model, not a dotted path. If it doesn't exist: `MissingProperty` error | | `separator` | Joins the values over a multi-valued reference into a string (default `', '`); ignored for single-valued | | Situation | Derived value | | ---------------------------------------------------------------------- | --------------------------------------------------------------------------------- | | Single-valued reference | The remote value as it is; the declared `type` doesn't convert it | | Multi-valued reference, `string`, `int`, `float` or `date` type | The values joined with `separator`, as a string | | Multi-valued reference, `Array<…>` (or any other) type | The list of values | | Falsy remote values (`0`, `false`, `''`) over a multi-valued reference | Left out | | Single-valued reference is empty, or its key has no item | The property is left out (a missing item is a `RemoteReferenceKeyNotFound` error) | | Multi-valued reference is empty, or none of its keys has an item | `''` for `string`, `int`, `float` and `date` types, `[]` for `Array<…>` types | | Some keys of a multi-valued reference have no item | Those keys are skipped; the others are joined or listed | | A referenced item changes on `dcupl.update()` | The derived value is recomputed | `index: true` builds an inverse index for derived properties over single-valued references (see [Indexing Derived Properties](#indexing-derived-properties)). To derive a reference instead of a value, use a derived reference (`derive: { localReference, remoteReference }`, see [Reference types](/sdk/api/models#reference)); the `derive` option itself is listed in the [Models API](/sdk/api/models#derive). ## Basic derived property ### Single-valued reference derivation Pull a property from a single referenced item: ```typescript import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; const dcupl = new Dcupl(); // Customer model const customerModel: ModelDefinition = { key: 'Customer', properties: [ { key: 'name', type: 'string' }, { key: 'email', type: 'string' }, { key: 'country', type: 'string' }, ], data: [ { key: 'c1', name: 'Alice Johnson', email: 'alice@example.com', country: 'USA' }, { key: 'c2', name: 'Bob Smith', email: 'bob@example.com', country: 'Canada' }, ], }; // Order model with derived property const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderId', type: 'string' }, { key: 'total', type: 'int' }, // Derived property - automatically populated from customer.name { key: 'customerName', type: 'string', derive: { localReference: 'customer', remoteProperty: 'name', }, }, ], references: [{ key: 'customer', type: 'singleValued', model: 'Customer' }], data: [ { key: 'o1', orderId: 'order1', total: 150, customer: 'c1' }, { key: 'o2', orderId: 'order2', total: 200, customer: 'c1' }, { key: 'o3', orderId: 'order3', total: 100, customer: 'c2' }, ], }; dcupl.models.set(customerModel); dcupl.models.set(orderModel); await dcupl.init(); const orderList = dcupl.lists.create({ modelKey: 'Order' }); const orders = orderList.catalog.query.items(); console.log(orders[0].customerName); // 'Alice Johnson' (derived!) // Filter on the derived property instead of a deep query on customer.name orderList.catalog.query.addCondition({ attribute: 'customerName', operator: 'eq', value: 'Alice Johnson', }); ``` The examples below reuse these imports and start from a fresh `const dcupl = new Dcupl()` each. ## Multi-valued reference derivation Derive properties from multi-valued references (arrays): ```typescript const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'int' }, ], data: [ { key: 'p1', name: 'Laptop', price: 999 }, { key: 'p2', name: 'Mouse', price: 29 }, { key: 'p3', name: 'Keyboard', price: 79 }, ], }; const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderId', type: 'string' }, // Derive product names from multi-valued reference { key: 'productNames', type: 'string', derive: { localReference: 'products', remoteProperty: 'name', separator: ', ', // Join multiple values }, }, ], references: [{ key: 'products', type: 'multiValued', model: 'Product' }], data: [ { key: 'o1', orderId: 'order1', products: ['p1', 'p2'] }, { key: 'o2', orderId: 'order2', products: ['p3'] }, ], }; dcupl.models.set(productModel); dcupl.models.set(orderModel); await dcupl.init(); const orderList = dcupl.lists.create({ modelKey: 'Order' }); const orders = orderList.catalog.query.items(); console.log(orders[0].productNames); // 'Laptop, Mouse' console.log(orders[1].productNames); // 'Keyboard' ``` Over a multi-valued reference, a `string`, `int`, `float` or `date` property gets the values joined with `separator` (default `', '`; `separator: ' & '` gives `'Laptop & Mouse'`); an `Array<…>` property (or any other type) gets the list of values. Falsy values (`0`, `false`, `''`) are left out. Over a single-valued reference, the remote value is copied as it is and `separator` is ignored. ## Chained derivations `derive` follows exactly one reference, and `remoteProperty` is a property of the referenced model (not a dotted path). To reach further, derive the value on the intermediate model first and derive that property: ```typescript const countryModel: ModelDefinition = { key: 'Country', properties: [ { key: 'name', type: 'string' }, { key: 'code', type: 'string' }, ], data: [ { key: 'us', name: 'United States', code: 'US' }, { key: 'ca', name: 'Canada', code: 'CA' }, ], }; const customerModel: ModelDefinition = { key: 'Customer', properties: [ { key: 'name', type: 'string' }, // Derive country name { key: 'countryName', type: 'string', derive: { localReference: 'country', remoteProperty: 'name', }, }, ], references: [{ key: 'country', type: 'singleValued', model: 'Country' }], data: [ { key: 'c1', name: 'Alice', country: 'us' }, { key: 'c2', name: 'Bob', country: 'ca' }, ], }; const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderId', type: 'string' }, // Derive customer's country name (2-level derivation) { key: 'customerCountryName', type: 'string', derive: { localReference: 'customer', remoteProperty: 'countryName', // This is already derived! }, }, ], references: [{ key: 'customer', type: 'singleValued', model: 'Customer' }], data: [ { key: 'o1', orderId: 'order1', customer: 'c1' }, { key: 'o2', orderId: 'order2', customer: 'c2' }, ], }; dcupl.models.set(countryModel); dcupl.models.set(customerModel); dcupl.models.set(orderModel); await dcupl.init(); const orderList = dcupl.lists.create({ modelKey: 'Order' }); const orders = orderList.catalog.query.items(); console.log(orders[0].customerCountryName); // 'United States' console.log(orders[1].customerCountryName); // 'Canada' // Fast filtering on derived property orderList.catalog.query.addCondition({ attribute: 'customerCountryName', operator: 'eq', value: 'United States', }); ``` ## Examples ### Filter by a referenced name Filter products by their category's name without a deep query: ```typescript const categoryModel: ModelDefinition = { key: 'Category', properties: [ { key: 'name', type: 'string' }, { key: 'slug', type: 'string' }, ], data: [ { key: 'c1', name: 'Electronics', slug: 'electronics' }, { key: 'c2', name: 'Clothing', slug: 'clothing' }, ], }; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'int' }, // Derive category name for easy filtering { key: 'categoryName', type: 'string', index: true, // eq/in on categoryName are index lookups derive: { localReference: 'category', remoteProperty: 'name', }, }, ], references: [{ key: 'category', type: 'singleValued', model: 'Category' }], data: [ { key: 'p1', name: 'Laptop', price: 999, category: 'c1' }, { key: 'p2', name: 'Mouse', price: 29, category: 'c1' }, { key: 'p3', name: 'T-Shirt', price: 19, category: 'c2' }, ], }; dcupl.models.set(categoryModel); dcupl.models.set(productModel); await dcupl.init(); const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'categoryName', operator: 'eq', value: 'Electronics', }); const electronics = productList.catalog.query.items(); // Returns: Laptop, Mouse ``` ### Search a multi-valued derivation ```typescript const tagModel: ModelDefinition = { key: 'Tag', properties: [ { key: 'name', type: 'string' }, { key: 'color', type: 'string' }, ], data: [ { key: 't1', name: 'JavaScript', color: '#f7df1e' }, { key: 't2', name: 'TypeScript', color: '#3178c6' }, { key: 't3', name: 'React', color: '#61dafb' }, ], }; const articleModel: ModelDefinition = { key: 'Article', properties: [ { key: 'title', type: 'string' }, // Derive tag names (multi-valued) { key: 'tagNames', type: 'string', derive: { localReference: 'tags', remoteProperty: 'name', separator: ', ', }, }, ], references: [{ key: 'tags', type: 'multiValued', model: 'Tag' }], data: [ { key: 'art1', title: 'Getting Started with TS', tags: ['t1', 't2'] }, { key: 'art2', title: 'React Hooks Guide', tags: ['t1', 't3'] }, ], }; dcupl.models.set(tagModel); dcupl.models.set(articleModel); await dcupl.init(); const articleList = dcupl.lists.create({ modelKey: 'Article' }); const articles = articleList.catalog.query.items(); console.log(articles[0].tagNames); // 'JavaScript, TypeScript' console.log(articles[1].tagNames); // 'JavaScript, React' // Search in derived tag names ('/TypeScript/' matches anywhere in the value) articleList.catalog.query.addCondition({ attribute: 'tagNames', operator: 'find', value: '/TypeScript/', }); // → Getting Started with TS ``` ### Date derived properties ```typescript const projectModel: ModelDefinition = { key: 'Project', properties: [ { key: 'name', type: 'string' }, { key: 'deadline', type: 'date' }, ], data: [{ key: 'proj1', name: 'Website Redesign', deadline: '2025-12-31' }], }; const taskModel: ModelDefinition = { key: 'Task', properties: [ { key: 'title', type: 'string' }, // Derive project deadline { key: 'projectDeadline', type: 'date', derive: { localReference: 'project', remoteProperty: 'deadline', }, }, ], references: [{ key: 'project', type: 'singleValued', model: 'Project' }], data: [{ key: 't1', title: 'Design mockups', project: 'proj1' }], }; dcupl.models.set(projectModel); dcupl.models.set(taskModel); await dcupl.init(); const taskList = dcupl.lists.create({ modelKey: 'Task' }); // Filter by project deadline (compare dates with a Date; an ISO string matches nothing) taskList.catalog.query.addCondition({ attribute: 'projectDeadline', operator: 'lte', value: new Date('2025-12-31'), }); ``` ### Array derived properties ```typescript const vendorModel: ModelDefinition = { key: 'Vendor', properties: [ { key: 'name', type: 'string' }, { key: 'certifications', type: 'Array' }, ], data: [{ key: 'v1', name: 'TechCorp', certifications: ['ISO9001', 'ISO27001'] }], }; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, // Derive array property (copied as it is over a single-valued reference) { key: 'vendorCertifications', type: 'Array', derive: { localReference: 'vendor', remoteProperty: 'certifications', }, }, ], references: [{ key: 'vendor', type: 'singleValued', model: 'Vendor' }], data: [{ key: 'p1', name: 'Widget', vendor: 'v1' }], }; dcupl.models.set(vendorModel); dcupl.models.set(productModel); await dcupl.init(); const productList = dcupl.lists.create({ modelKey: 'Product' }); const products = productList.catalog.query.items(); console.log(products[0].vendorCertifications); // ['ISO9001', 'ISO27001'] ``` ## Performance considerations ### Query performance **Before (Deep Query):** ```typescript title="Resolves the reference for every evaluated item" orderList.catalog.query.addCondition({ attribute: 'customer.name', operator: 'eq', value: 'Alice', }); ``` **After (Derived Property):** ```typescript title="Reads the value stored on the item" orderList.catalog.query.addCondition({ attribute: 'customerName', // Derived property operator: 'eq', value: 'Alice', }); ``` A deep query resolves the reference on every evaluated item; a derived property is stored on the item when the data is processed, and with `index: true` an `eq` condition becomes an index lookup. The gap grows with the number of items and the depth of the path. Measure with your own data (see [Profiling](/sdk/advanced/performance#profiling-and-debugging)). ### Memory vs speed trade-off A derived property stores one more value on every item and is recomputed when the referenced items change, so it costs memory and some processing time in `init()` and `update()`. Derive the values you query often, not every field of the referenced model. ### When to use derived properties **Use derived properties when:** - You frequently query on referenced properties - Query performance is critical - The referenced data changes infrequently - You need fast facets/aggregations **Avoid derived properties when:** - The referenced data changes constantly - Memory is very constrained - You rarely query the property - Deep queries are fast enough for your use case ### Indexing derived properties Set `index: true` on a derived property over a single-valued reference to build its inverse index while the data is processed, so `eq` and `in` conditions on it are index lookups. Derived properties over multi-valued references are not indexed. (`filter: true` doesn't index; it only lists the property in `catalog.filters` for filter UIs.) ```typescript { key: 'customerName', type: 'string', index: true, derive: { localReference: 'customer', remoteProperty: 'name', }, } ``` ## Best practices ### 1. Index derived properties you filter by exact value ```typescript // ✅ Good: indexed, so eq/in conditions are index lookups { key: 'categoryName', type: 'string', index: true, derive: { localReference: 'category', remoteProperty: 'name', }, } // ❌ Bad: no index, so every eq/in condition scans the items { key: 'categoryName', type: 'string', derive: { localReference: 'category', remoteProperty: 'name', }, } ``` ### 2. Declare the remote property's type The derived value is copied as it is (see [Type Mismatch Errors](#type-mismatch-errors)), so give the derived property the type of the remote property: a `float` remote property gets a `float` derived property. ### 3. Use descriptive names Prefix the derived property with the reference it comes from, such as `customerName`, `categoryTitle` or `authorFullName`, rather than `name`, `title` or `value`. ### 4. Derive only what you need Each derived property adds a value to every item. Derive the fields you filter, sort, facet or display (for example `customerName` and `customerEmail`); read the rest with a [projection](/sdk/api/types#projection) when you need it. ## Troubleshooting ### Derived property is undefined **Problem:** Derived property shows as undefined ```typescript console.log(order.customerName); // undefined ``` **Solutions:** 1. Check reference exists. Over a single-valued reference, dcupl leaves the derived property out when the reference is empty or its key has no item (over a multi-valued reference you get `''` or `[]` instead): ```typescript console.log(order.customer); // { key: 'c1' }, not undefined ``` 2. Verify reference is resolved: ```typescript await dcupl.init(); // Must call init() ``` 3. Check remote property exists. Items list a reference as `{ key }` only, so project it to see the remote values: ```typescript const [order] = orderList.catalog.query.items({ projection: { $: true, customer: { $: true } } }); console.log(order.customer.name); // Verify source property ``` `dcupl.quality.errors.summary()` also lists the cause: `RemoteReferenceKeyNotFound` (the referenced key has no item), `MissingProperty` (`remoteProperty` isn't a property of the referenced model) or `MissingReference` (`localReference` isn't a reference of this model). ### Type mismatch errors **Problem:** The derived value doesn't have the type the property declares **Solution:** dcupl copies the remote value as it is: the derived property's `type` doesn't convert it (over a multi-valued reference it only decides between a joined string and a list). Declare the remote property's type: ```typescript // Remote property { key: 'price', type: 'float' } // Derived property (must match) { key: 'categoryPrice', type: 'float', derive: {...} } ``` ### Multi-valued derivation not working **Problem:** Multi-valued derivation returns incorrect results **Solution:** Ensure reference is multi-valued: ```typescript references: [ { key: 'tags', type: 'multiValued', model: 'Tag' }, // Must be multiValued ]; properties: [ { key: 'tagNames', type: 'string', derive: { localReference: 'tags', remoteProperty: 'name', separator: ', ', }, }, ]; ``` ### Performance issues **Problem:** Derived properties not improving performance **Solution:** Set `index: true` (derived properties over single-valued references) for properties you filter with `eq` or `in`: ```typescript { key: 'customerName', type: 'string', index: true, derive: { localReference: 'customer', remoteProperty: 'name', }, } ``` ## Next steps - [Expression properties](/sdk/advanced/expression-properties) — Template-based computed fields - [Models](/sdk/core-concepts/models) — Properties and reference relationships - [Deep queries](/sdk/queries/query-deep) — Querying across references - [Performance](/sdk/advanced/performance) — Optimization strategies - [Models API](/sdk/api/models#reference) — Derived, resolved and query references --- # Expression properties Source: https://docs.dcupl.com/sdk/advanced/expression-properties Description: Create dynamic computed fields using template expressions # Expression properties Expression properties are template-based computed fields that combine multiple properties into dynamic, formatted strings. Perfect for display names, labels, descriptions, and any scenario where you need to compose values dynamically. ## What are expression properties? An expression property uses a template string with variable interpolation to create computed values: ```typescript { key: 'displayName', type: 'string', expression: '${firstName} ${lastName}', } ``` **How it works:** ``` Input: { firstName: 'Alice', lastName: 'Johnson' } Output: { displayName: 'Alice Johnson' } ``` **Key features:** - Variable interpolation with `${variable}` - String concatenation - Access to all properties in the same model - Access to referenced model properties - Automatic updates when source values change ## Evaluation rules Expressions are text templates, not formulas: | Rule | Behaviour | | ---------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Placeholders | Each `${path}` is replaced with the value at that path. Any other syntax, such as `{{name}}`, stays literal text | | Result | Always a string; the declared `type` is not applied. There is no arithmetic, comparison or condition | | Missing values | A path without a value becomes an empty string; a `null` value renders as `'null'` | | Value conversion | JavaScript string conversion: numbers as numbers (`8.0` becomes `'8'`), arrays joined with commas (`'red,blue'`), `json` objects as `'[object Object]'`, `date` values as `Date.toString()`, which depends on the time zone (`'Wed Jan 15 2025 00:00:00 GMT+0000 (…)'`) | | References | Single-valued references resolve two levels deep (`${brand.name}`, `${brand.country.name}`); `${tags.0.name}` picks an entry of a multi-valued reference of the item. One level further a reference holds only its key: `${brand.country.region.key}` resolves, `${brand.country.region.name}` is empty | | Order | Expressions are evaluated in the order of `properties`, so declare an expression before another expression that uses it | | Updates | Computed in `init()`, and recomputed on `dcupl.update()` for items whose values (or items they reach through single-valued references) changed. A path through a multi-valued reference (`${tags.0.name}`) is not recomputed when the referenced item gets a partial update; it stays stale until a full rebuild (for example after a `data.set()`) | To show a date in an expression, keep it as a string property or format it before loading, for example in a [loader transformer](/sdk/api/loader#transformers). ## Basic expression ### Simple variable interpolation ```typescript import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; const dcupl = new Dcupl(); const customerModel: ModelDefinition = { key: 'Customer', properties: [ { key: 'firstName', type: 'string' }, { key: 'lastName', type: 'string' }, // Expression property { key: 'fullName', type: 'string', expression: '${firstName} ${lastName}', }, ], data: [ { key: 'c1', firstName: 'Alice', lastName: 'Johnson' }, { key: 'c2', firstName: 'Bob', lastName: 'Smith' }, ], }; dcupl.models.set(customerModel); await dcupl.init(); const customerList = dcupl.lists.create({ modelKey: 'Customer' }); const customers = customerList.catalog.query.items(); console.log(customers[0].fullName); // 'Alice Johnson' console.log(customers[1].fullName); // 'Bob Smith' // Filter on expression property ('/Alice/' matches anywhere in the value) customerList.catalog.query.addCondition({ attribute: 'fullName', operator: 'find', value: '/Alice/', }); // → Alice Johnson ``` ## String concatenation ### With static text ```typescript const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'sku', type: 'string' }, { key: 'price', type: 'float' }, // Product title with SKU { key: 'displayTitle', type: 'string', expression: '${name} (${sku})', }, // Price label { key: 'priceLabel', type: 'string', expression: 'Price: $${price}', }, ], data: [ { key: 'p1', name: 'Laptop', sku: 'LPT-001', price: 999 }, { key: 'p2', name: 'Mouse', sku: 'MSE-002', price: 29 }, ], }; dcupl.models.set(productModel); await dcupl.init(); const productList = dcupl.lists.create({ modelKey: 'Product' }); const products = productList.catalog.query.items(); console.log(products[0].displayTitle); // 'Laptop (LPT-001)' console.log(products[0].priceLabel); // 'Price: $999' ``` ### Multi-line templates A `\n` in the expression is a line break in the value: ```typescript const employeeModel: ModelDefinition = { key: 'Employee', properties: [ { key: 'firstName', type: 'string' }, { key: 'lastName', type: 'string' }, { key: 'title', type: 'string' }, { key: 'department', type: 'string' }, { key: 'email', type: 'string' }, // Display name with title { key: 'displayName', type: 'string', expression: '${firstName} ${lastName}, ${title}', }, // Email signature, one value per line { key: 'emailSignature', type: 'string', expression: '${firstName} ${lastName}\n${title}\n${department}\n${email}', }, ], data: [ { key: 'e1', firstName: 'Alice', lastName: 'Johnson', title: 'Senior Engineer', department: 'Engineering', email: 'alice@example.com', }, ], }; dcupl.models.set(employeeModel); await dcupl.init(); const employeeList = dcupl.lists.create({ modelKey: 'Employee' }); const employees = employeeList.catalog.query.items(); console.log(employees[0].displayName); // 'Alice Johnson, Senior Engineer' console.log(employees[0].emailSignature); // Alice Johnson // Senior Engineer // Engineering // alice@example.com ``` ## Complex expressions ### Using reference properties Expression properties can access properties from referenced models: ```typescript const brandModel: ModelDefinition = { key: 'Brand', properties: [ { key: 'name', type: 'string' }, { key: 'tagline', type: 'string' }, ], data: [{ key: 'b1', name: 'Apple', tagline: 'Think Different' }], }; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'model', type: 'string' }, // Expression using reference property { key: 'fullTitle', type: 'string', expression: '${brand.name} ${name} - ${model}', }, ], references: [{ key: 'brand', type: 'singleValued', model: 'Brand' }], data: [{ key: 'p1', name: 'iPhone', model: '14 Pro', brand: 'b1' }], }; dcupl.models.set(brandModel); dcupl.models.set(productModel); await dcupl.init(); const productList = dcupl.lists.create({ modelKey: 'Product' }); const products = productList.catalog.query.items(); console.log(products[0].fullTitle); // 'Apple iPhone - 14 Pro' ``` The value resolves, but SDK 2.0 also records an `UnknownExpressionVariable` quality error for `${brand.name}`, because the validator only knows the model's own properties and references. To avoid the error, derive the referenced value into a property first and use that property in the expression. ### Combining derived and expression properties Derived properties are filled before expressions, so an expression can use them: ```typescript const brandModel: ModelDefinition = { key: 'Brand', properties: [{ key: 'name', type: 'string' }], data: [{ key: 'b1', name: 'Apple' }], }; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, // Derived from the brand reference { key: 'brandName', type: 'string', derive: { localReference: 'brand', remoteProperty: 'name' } }, // Uses the derived property: no quality error { key: 'fullTitle', type: 'string', expression: '${brandName} ${name}' }, ], references: [{ key: 'brand', type: 'singleValued', model: 'Brand' }], data: [{ key: 'p1', name: 'iPhone', brand: 'b1' }], }; dcupl.models.set(brandModel); dcupl.models.set(productModel); await dcupl.init(); const productList = dcupl.lists.create({ modelKey: 'Product' }); console.log(productList.catalog.query.items()[0].fullTitle); // 'Apple iPhone' ``` See [Derived properties](/sdk/advanced/derived-properties) for the derive options. ### Expressions using other expressions An expression can use another expression property declared before it: ```typescript const softwareModel: ModelDefinition = { key: 'Software', properties: [ { key: 'name', type: 'string' }, { key: 'major', type: 'int' }, { key: 'minor', type: 'int' }, { key: 'patch', type: 'int' }, // Semantic version { key: 'version', type: 'string', expression: '${major}.${minor}.${patch}', }, // Full software name: uses version, so it comes after it { key: 'fullName', type: 'string', expression: '${name} v${version}', }, ], data: [{ key: 's1', name: 'dcupl', major: 2, minor: 5, patch: 3 }], }; dcupl.models.set(softwareModel); await dcupl.init(); const softwareList = dcupl.lists.create({ modelKey: 'Software' }); const software = softwareList.catalog.query.items(); console.log(software[0].version); // '2.5.3' console.log(software[0].fullName); // 'dcupl v2.5.3' ``` If `fullName` were declared before `version`, it would render as `'dcupl v'`. ## Best practices ### 1. Filter on expression properties like any property You can use an expression property in `addCondition` without any flag. Add `filter: true` only to list it in `catalog.filters` for a filter UI. `index: true` has no effect on expression properties, so `eq` conditions on them scan the list's items. ```typescript { key: 'fullName', type: 'string', filter: true, // only needed for catalog.filters expression: '${firstName} ${lastName}', } ``` ### 2. Keep templates simple Prefer short templates over long ones like `'${category.type} > ${category.name} > ${brand.name} ${name} (${sku})'`. Split a long template into intermediate expressions (see [Expression Too Complex](#expression-too-complex)). ### 3. Consider performance Expressions are computed in `init()` and recomputed for changed items on `dcupl.update()`. For every item, each expression property resolves all of the model's references, not only the ones its template uses, so the cost grows with items × expression properties × references. On models with many references, keep the number of expression properties small. ### 4. Use for display, not logic ```typescript // ✅ Good: Display purposes { key: 'displayName', type: 'string', expression: '${firstName} ${lastName}', } // ❌ Bad: Don't use for calculations { key: 'total', type: 'string', expression: '${price} * ${quantity}', // Won't calculate! } ``` ## Troubleshooting ### Expression not evaluating **Problem:** Expression shows literal template ```typescript console.log(product.displayName); // '{{brand}} {{name}}' ``` **Solution:** Write placeholders as `${property}`; any other syntax, such as `{{property}}`, stays literal text. Put the expression in single or double quotes: in a backtick template literal, JavaScript fills in `${…}` itself before dcupl sees the expression. ```typescript expression: '${brand} ${name}'; // ✅ dcupl placeholders ``` ### Empty value in expression **Problem:** A placeholder renders as an empty string ```typescript console.log(product.fullTitle); // 'Apple ' ``` **Solution:** A placeholder whose path has no value becomes an empty string (a `null` value renders as `'null'`). Check source properties exist on the item and the placeholder spells the key as the model does: ```typescript console.log(product.name); // Verify source property ``` ### Reference property not accessible **Problem:** Can't access reference properties ```typescript expression: '${brand.name} ${name}'; // brand.name renders as '' ``` **Solution:** A reference path resolves only when it follows single-valued references at most two levels deep (or picks an entry of a multi-valued reference, `${tags.0.name}`) and the referenced item exists. One level further, only `.key` resolves. A key without an item is reported as a `RemoteReferenceKeyNotFound` quality error: ```typescript console.log(dcupl.quality.errors.summary()); // look for RemoteReferenceKeyNotFound ``` For a longer path, derive the value onto an intermediate model with [Derived properties](/sdk/advanced/derived-properties) and use that property. ### Expression too complex **Problem:** Very long expression is hard to maintain **Solution:** Break into multiple expressions. Declare the intermediate expressions first: expressions are evaluated in the order of `properties`, and a placeholder for an expression that comes later renders as an empty string. ```typescript // Instead of one huge expression { key: 'description', expression: '${a} ${b} ${c} ${d} ${e} ${f} ${g}', } // Create intermediate expressions { key: 'partOne', expression: '${a} ${b} ${c}', }, { key: 'partTwo', expression: '${d} ${e}', }, { key: 'description', expression: '${partOne} - ${partTwo} ${f} ${g}', } ``` ## Next steps - [Derived properties](/sdk/advanced/derived-properties) — Compute values from references - [Models](/sdk/core-concepts/models) — Properties and references configuration - [Quality & validation](/sdk/advanced/quality-validation) — Validate expressions - [Performance](/sdk/advanced/performance) — Optimization strategies --- # Quality & validation Source: https://docs.dcupl.com/sdk/advanced/quality-validation Description: Validate data quality and enforce business rules # Quality & validation dcupl provides a comprehensive quality and validation system to ensure data integrity, enforce business rules, and catch errors early. Configure validation rules at both the model and property level. ## What is quality & validation? Quality configuration enables: - **Required fields** - Ensure critical data is present - **Nullable control** - Allow or prevent `null` values - **Type enforcement** - Strict or loose data type checking - **Validators** - Built-in email, min/max, pattern, enum, unique and more - **Error tracking** - Collect and report validation errors - **Error summary** - Error counts per model, attribute and error type ## Enabling quality validation Quality validation has two levels, the instance and the model. Both are **enabled by default**, and checks run only when both are enabled: ```typescript import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; const dcupl = new Dcupl({ quality: { enabled: true, // Default: true. false turns off all quality checks maxErrorsPerGroup: 1000, // Default: 1000. Stored error records per (model, attribute, error type) }, }); const userModel: ModelDefinition = { key: 'User', quality: { enabled: true }, // Default: true. false skips all checks for this model properties: [ // ... properties with validation rules ], }; ``` The older `errorTracking: { enabled }` option is deprecated and can't turn checks off: checks run when either option is enabled, and `quality.enabled` defaults to `true`. Use `quality: { enabled: false }` instead. ## Property quality configuration ### Quality options Instance options go into `new Dcupl({ quality })`: | Option (default) | Effect | | ---------------------------- | -------------------------------------------------------------------------------------------------------------- | | `enabled` (`true`) | `false` turns off all quality checks. Required by the TypeScript type when you pass `quality` | | `maxErrorsPerGroup` (`1000`) | Stored error records per (model, attribute, error type), see [Limiting Stored Errors](#limiting-stored-errors) | Model options go into the model definition's `quality`: | Option (default) | Effect | | ------------------ | --------------------------------------------------------------------------------- | | `enabled` (`true`) | `false` skips all checks for this model | | `attributes` | Defaults for every attribute of the model: all attribute options but `validators` | Attribute options go into a property's (or reference's) `quality`: | Option (default) | Effect | | ------------------------------- | ----------------------------------------------------------------------- | | `required` (`true`) | A missing value is reported as `UndefinedValue` | | `nullable` (`false`) | A `null` value is reported as `NullValue` | | `forceStrictDataType` (`false`) | No coercion: a value of another type is reported and dropped | | `validators` | Validation rules, see [Validators](#validators) | | `validatorHandling` (`'loose'`) | A failed validator keeps the value (`'loose'`) or drops it (`'strict'`) | What happens to the value when a check fails: | Failure | Error and value on the item | | --------------------------------------------------------- | ------------------------------------------ | | Value missing (`undefined`) and `required` is `true` | `UndefinedValue`; the value is absent | | Value is `null` and `nullable` is `false` | `NullValue`; the value is kept as `null` | | Value of another type and `forceStrictDataType` is `true` | `WrongDataType`; the value is dropped | | Value can't be coerced to the declared type | `WrongDataType`; the value is dropped | | Number coerced with a loss (`5.5` on an `int`) | `WrongDataType`; the coerced value is kept | | Validator fails and `validatorHandling` is `'loose'` | `InvalidValidator`; the value is kept | | Validator fails and `validatorHandling` is `'strict'` | `InvalidValidator`; the value is dropped | ### Model-wide defaults Attributes are **required by default**: a declared property or reference that is missing on an item is reported as `UndefinedValue`. Mark optional fields with `required: false`, or change the default for a whole model with `quality.attributes`. It accepts `required`, `nullable`, `forceStrictDataType` and `validatorHandling`; validators are set per attribute only. ```typescript const productModel: ModelDefinition = { key: 'Product', quality: { attributes: { required: false }, // every attribute optional unless it says otherwise }, properties: [ { key: 'name', type: 'string', quality: { required: true } }, { key: 'description', type: 'string' }, // optional ], }; ``` ### Required fields A required field without a value is reported as `UndefinedValue`: ```typescript const userModel: ModelDefinition = { key: 'User', properties: [ { key: 'email', type: 'string', quality: { required: true } }, { key: 'bio', type: 'string', quality: { required: false } }, // optional ], data: [ { key: 'u1', email: 'alice@example.com' }, // ✅ Valid { key: 'u2' }, // ❌ Missing required email ], }; dcupl.models.set(userModel); await dcupl.init(); const errors = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'model', operator: 'eq', value: 'User' }], }); console.log(errors.map((e) => `${e.itemKey}.${e.attribute}: ${e.errorType.key}`)); // [ 'u2.email: UndefinedValue' ] ``` ### Nullable fields A `null` on a non-nullable field is reported as `NullValue`; the value itself is kept on the item. ```typescript const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string', quality: { nullable: false } }, // the default { key: 'description', type: 'string', quality: { required: false, nullable: true } }, ], data: [ { key: 'p1', name: 'Laptop', description: null }, // ✅ Valid { key: 'p2', name: null }, // ❌ name cannot be null (NullValue) ], }; ``` ### Strict data type checking By default, values are coerced to the declared type (`'5'` becomes `5`). Values that can't be coerced are dropped and reported as `WrongDataType`, and so are lossy coercions (`5.5` becomes `5` on an `int`, the coerced value is kept). This happens without any strict setting. A `string` property accepts any value (`42` becomes `'42'`), and a `boolean` property accepts the strings `'true'`/`'false'`/`'1'`/`'0'` (any case) and the numbers `1`/`0`; any other string, number, object or array is reported as `WrongDataType` and the value stays undefined. CSV cells follow the same rule, except that an empty cell is no value, not a `WrongDataType`. `forceStrictDataType: true` turns coercion off: every value that isn't already of the declared type is reported as `WrongDataType` and dropped. `date` properties ignore the setting. ```typescript const orderModel: ModelDefinition = { key: 'Order', properties: [{ key: 'quantity', type: 'int', quality: { forceStrictDataType: true } }], data: [ { key: 'o1', quantity: 5 }, // ✅ Valid int { key: 'o2', quantity: '5' }, // ❌ String not allowed { key: 'o3', quantity: 5.5 }, // ❌ Float not allowed ], }; ``` > **Watch out:** Validators run on `string`, `int` and `float` properties (and their `Array<…>` types), never on `boolean`, `json` or `date` properties. In SDK 2.0 they are also skipped on `int` and `float` properties that use `forceStrictDataType: true`, and a strict `float` rejects whole numbers such as `100`. `unique` is the exception: it runs on every type. Don't combine strict mode with validators on numbers. ### References References are required by default too, so an item without a value for a reference is reported as `UndefinedValue` (error group `ReferenceDataError`). An absent or empty (`''`) value counts as missing. An explicit `null` is kept as `null` on the item (SDK 2.0; `eq null` matches it, `typeof 'undefined'` does not) and is reported as `NullValue` unless `nullable` is set, as for a property. Of the other attribute options, `forceStrictDataType` and `validators` have no effect on references. | Situation | Error and reference on the item | | ------------------------------------------------------------------ | ------------------------------------------------------ | | No value and `required` is `true` (the default) | `UndefinedValue`; the reference is absent | | `null` and `nullable` is `false` (the default) | `NullValue`; the reference is kept as `null` | | Key not in the referenced model, `validity` is `'loose'` (default) | `RemoteReferenceKeyNotFound`; kept as `{ key }` | | Key not in the referenced model, `validity` is `'strict'` | `RemoteReferenceKeyNotFound`; the reference is dropped | | Wrong shape (an array on a `singleValued` reference) | `WrongDataType`; the reference is absent | `validity` is set on the reference itself (`{ key: 'category', type: 'singleValued', model: 'Category', validity: 'strict' }`). A referenced model with `supportsAutoCreation` creates the missing item instead of reporting it. See the reference options in the [Models API](/sdk/api/models#reference). ## Validators A failed validator is reported as `InvalidValidator`. With the default `validatorHandling: 'loose'` the value is kept; with `'strict'` it is dropped from the item. Validators take `{ value }`; `email` and `unique` take `{}`. The available validators are `email`, `min`, `max`, `minLength`, `maxLength`, `pattern`, `enum`, `startsWith`, `endsWith`, `includes` and `unique`. ### Email validator Validate email format: ```typescript const contactModel: ModelDefinition = { key: 'Contact', properties: [ { key: 'email', type: 'string', quality: { required: true, validators: { email: {}, // Email format validation }, }, }, ], data: [ { key: 'c1', email: 'alice@example.com' }, // ✅ Valid { key: 'c2', email: 'invalid-email' }, // ❌ Invalid format { key: 'c3', email: 'bob@domain' }, // ❌ Invalid format ], }; dcupl.models.set(contactModel); await dcupl.init(); const errors = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [ { attribute: 'model', operator: 'eq', value: 'Contact' }, { attribute: 'errorType', operator: 'eq', value: 'InvalidValidator' }, ], }); console.log(errors.map((e) => e.description)); // [ // 'The value "invalid-email" is not a valid email address', // 'The value "bob@domain" is not a valid email address' // ] ``` ### Min/max validators Validate numeric ranges on `int` and `float` properties: ```typescript const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'rating', type: 'float', quality: { validators: { min: { value: 1 }, max: { value: 5 } } }, }, ], data: [ { key: 'p1', rating: 4.5 }, // ✅ Valid { key: 'p2', rating: 0 }, // ❌ 'The value "0" is smaller than the minimum value of "1"' { key: 'p3', rating: 6 }, // ❌ 'The value "6" is larger than the maximum value of "5"' ], }; ``` ### Unique validator Ensure unique values across all items of the model: ```typescript const userModel: ModelDefinition = { key: 'User', properties: [{ key: 'email', type: 'string', quality: { validators: { unique: {} } } }], data: [ { key: 'u1', email: 'alice@example.com' }, // ✅ Valid { key: 'u2', email: 'alice@example.com' }, // ❌ Duplicate email ], }; ``` A duplicate is reported as `InvalidValidator` (title "Validator Error - Unique") on the second and later items with that value. In SDK 2.0, items without a value count as duplicates of each other too (`The value "undefined" is not unique`). `NonUniqueKey` is a different error: two items with the same **key**. ### Common validations The other validators work like `min` and `max`: one `{ value }` each, and each failure is its own `InvalidValidator` record with the description shown. Several validators on one attribute go into the same `validators` object (`{ email: {}, unique: {}, maxLength: { value: 254 } }`) and are checked independently. | Validator | Typical field | Configuration | Fails when (description) | | ----------------------- | ---------------------------- | -------------------------------------------------------------------- | ----------------------------------------------------------------------------------- | | `minLength`/`maxLength` | Username, file name | `minLength: { value: 3 }`, `maxLength: { value: 20 }` | The value has fewer / more characters ("is shorter than the minimum length of") | | `pattern` | SKU, order number | `pattern: { value: '^[A-Z]{3}-[0-9]{4}$' }` | The value doesn't match the regular expression ("does not match the pattern") | | `pattern` | Phone number (E.164), US ZIP | `pattern: { value: '^\\+?[1-9]\\d{1,14}$' }`, `'^\\d{5}(-\\d{4})?$'` | As above; escape backslashes in the string | | `enum` | Status, priority | `enum: { value: ['pending', 'shipped', 'delivered'] }` | The value isn't one of the listed values, exact match ("is not in the enum values") | | `startsWith`/`endsWith` | File name, report prefix | `startsWith: { value: 'report_' }`, `endsWith: { value: '.pdf' }` | The value doesn't start / end with the text ("does not start with") | | `includes` | Comma-separated tags | `includes: { value: 'approved' }` | The value doesn't contain the text ("does not include") | | `min`/`max` | Price, age, quantity | `min: { value: 0 }`, `max: { value: 120 }` | See [Min/max validators](#minmax-validators) | | `email`, `unique` | Email address, SKU | `email: {}`, `unique: {}` | See [Email validator](#email-validator) and [Unique validator](#unique-validator) | Attributes are required by default, so `required: true` is only needed where the model sets `quality: { attributes: { required: false } }`. ## Error handling ### Getting errors `dcupl.quality.errors.summary()` returns one entry per (model, attribute, error type) group with its count. The individual error records live in the built-in model `$DcuplErrorTrackingErrors`, so you query them like any other model. With `quality: { enabled: false }` these models are not created, and `summary()` lists only loader errors. Using the `User` model from [Required Fields](#required-fields): ```typescript await dcupl.init(); // One entry per (model, attribute, errorType) group const summary = dcupl.quality.errors.summary(); console.log(summary); // [ // { // key: 'User::email::UndefinedValue', // model: 'User', // attribute: 'email', // errorGroup: 'PropertyDataError', // errorType: 'UndefinedValue', // type: 'model', // totalCount: 1, // storedCount: 1, // truncated: false // } // ] // Individual error records for one model const userErrors = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'model', operator: 'eq', value: 'User' }], }); console.log(userErrors); // [ // { // key: '4YfyXvsPa2', // title: 'Undefined Value', // itemKey: 'u2', // attribute: 'email', // description: 'The required string property "email" in the "User" model is undefined (no value)', // meta: { rawValue: undefined, declaredType: 'string', origin: 'inline' }, // groupTotal: 1, // truncated: false, // model: { key: 'User' }, // errorGroup: { key: 'PropertyDataError' }, // errorType: { key: 'UndefinedValue' } // } // ] ``` In error records, `model`, `errorGroup` and `errorType` are references: filter them with a plain string (`value: 'User'`) and read them as `error.errorType.key`. The record `key` is generated. ### Error types **Model errors** (`type: 'model'`): - `UndefinedValue` - A required field has no value on an item - `NullValue` - A non-nullable field is `null` on an item - `WrongDataType` - A value can't be coerced to the declared type, or the coercion is lossy. Reported without any strict setting - `InvalidValidator` - A validator failed (including `unique`) - `UndefinedAttribute` - A required attribute has no column or field at all in the loaded data (one record without an `itemKey`) - `UnknownColumn` - A column in a loaded CSV file is not declared on the model; its values are dropped. Only reported for CSV resources, not for JSON or inline data - `NonUniqueKey` - Two rows loaded together (with `data.set()`, a model's `data` or a loader resource) share the same key. The last row wins as a whole, at the first row's position; no values are merged from the replaced rows, and the replaced rows record no further errors. One `NonUniqueKey` is recorded per replaced row, also when the rows arrive in separate containers (`data.set()` after `init()`, model data plus a staged `upsert`). `upsert` and `update` rows with the same key still merge without an error - `MissingKey` - An item has no key: no `key`, a custom `keyProperty` whose value is `undefined`/`null`, or a composite key with every part missing (a single missing part keeps its `'undefined'`/`'null'` part). The item is dropped. On a model with derived, grouped, resolved or query references, or derived or expression properties, a keyless row currently also makes `init()`, or a `dcupl.update()` that rebuilds (for example after `data.set()`), fail: it logs "Initialization failed" or "Update failed" and resolves `false`. This also happens with quality checks disabled. Remove such rows before loading - `RemoteReferenceKeyNotFound` - A reference value points to a key that doesn't exist in the referenced model - `MissingModel` - A reference points to a model that isn't registered - `MissingProperty` - A derived property's `remoteProperty` doesn't exist on the remote model - `MissingReference` - A derived, resolved or grouped attribute names a reference that doesn't exist (a derived property's `localReference`, a derived reference's `localReference` or `remoteReference`, `resolve.reference`, `groupBy.reference`) - `DuplicatePropertyKey` - A model declares the same property key twice - `UnknownExpressionVariable` - An expression uses a variable that is neither a property nor a reference - `InvalidModelDefinition` - The model definition is invalid **Loader errors** (`type: 'loader'`, recorded even when quality checks are disabled): - `InvalidResource` - A resource could not be fetched or parsed - `TransformerError` - A transformer threw while processing a resource - `ScriptingDisabled` - A `script`, `operator` or `transformer` resource was skipped because the instance runs with `scripting: { enabled: false }` - `InvalidConfig` - Typed, but never recorded: a loader configuration that can't be fetched or parsed makes `loader.config.fetch()` throw instead - `EndpointScriptMissing`, `EndpointDuplicate` - Reported by [REST API instances](/console/rapi/scripted-endpoints), not by the SDK: an endpoint's `scriptKey` names a script that isn't registered, or the same method and path is configured twice ### Errors after updates There is no public method to reset errors. Errors follow the data instead: when you fix the data and apply the update, the errors for those items are removed. ```typescript dcupl.data.update([{ key: 'u2', email: 'bob@example.com' }], { model: 'User' }); await dcupl.update(); console.log(dcupl.quality.errors.summary()); // [] ``` > **Note:** After `dcupl.update()`, partial or full, `summary()`, the `$DcuplErrorTrackingErrors` and `$DcuplErrorTrackingSummary` models, and lists on them hold the same records a fresh `init()` would produce for the current data: the changed rows are re-validated (`unique` included), the errors of removed rows are gone, and a value of the wrong type in `data.update()` is recorded as `WrongDataType` and dropped. The item then has no value for that attribute; its previous value is not kept. One exception: a keyless row in a `data.upsert()` that a partial update applies is currently dropped without a `MissingKey` record; `init()` and a full rebuild record it. ## Quality reports `$DcuplErrorTrackingErrors` and `$DcuplErrorTrackingSummary` are built the first time something reads them (`query.one`/`many`/`execute`, `fn.*` or a list on them) rather than during every `init()` and `update()`; they still describe the error set of the last `init()` or `update()`, and every read returns the same records. The first read pays for the build. Once a list over an error-tracking model exists (`lists.create` until it is destroyed; a `query.execute` or `fn.*` call on one keeps such a list internally), every `init()` and `update()` builds them again right away. Errors whose description does not depend on the row (`UndefinedValue`, `NullValue`, rows without a key) share one description string per (model, attribute, error type) group, which keeps the heap lower on error-heavy datasets. Measured on a 23,754-row customer catalog with about 344,000 recorded errors in 887 groups, default options: building them at the first read and sharing the descriptions took the retained heap after `init()` from 285 MB to 110 MB, and `init()` from 2763 ms to 2030 ms (2026-10-16). ### Error summary Each `summary()` entry describes one (model, attribute, error type) group: | Field | Description | | ------------------------- | ------------------------------------------------------------------- | | `key` | `'Model::attribute::ErrorType'` | | `model`, `attribute` | Affected model and attribute (if any) | | `errorGroup`, `errorType` | Error classification | | `type` | `'model'` or `'loader'` | | `totalCount` | Number of occurrences, also beyond `maxErrorsPerGroup` | | `storedCount` | Records kept in `$DcuplErrorTrackingErrors` (≤ `maxErrorsPerGroup`) | | `truncated` | `true` when `totalCount` is greater than `storedCount` | The same totals are available as the built-in model `$DcuplErrorTrackingSummary`. ### Limiting stored errors `quality.maxErrorsPerGroup` (default `1000`) caps how many error records are stored per group. The totals keep counting: ```typescript const dcupl = new Dcupl({ quality: { enabled: true, maxErrorsPerGroup: 2 }, }); dcupl.models.set({ key: 'Product', properties: [{ key: 'price', type: 'float', quality: { validators: { min: { value: 0 } } } }], data: [1, 2, 3, 4, 5].map((i) => ({ key: `p${i}`, price: -i })), }); await dcupl.init(); const [group] = dcupl.quality.errors.summary(); console.log(group.totalCount, group.storedCount, group.truncated); // 5 2 true ``` The stored records of a capped group carry `truncated: true` and the group's `groupTotal`. Failed type coercions are the exception. They are consolidated into one `WrongDataType` record per attribute, without an `itemKey`, with the number of affected rows in `meta.rowCount` and one sample value in `meta.rawValue`. When more values of one attribute fail than `maxErrorsPerGroup`, the counts around that consolidation aren't exact in SDK 2.0 (for inline data the consolidated record can even be missing), so keep `maxErrorsPerGroup` above the number of failures you need to count. ### Browsing errors in a list Errors are a model, so lists, queries and facets work on them, for example to count errors by type: ```typescript dcupl.models.set({ key: 'User', properties: [ { key: 'email', type: 'string', quality: { validators: { email: {} } } }, { key: 'age', type: 'int' }, ], data: [ { key: 'u1', email: 'not-an-email', age: 'old' }, { key: 'u2', age: 30 }, { key: 'u3', email: 'also-wrong', age: 41 }, ], }); await dcupl.init(); const errorList = dcupl.lists.create({ modelKey: '$DcuplErrorTrackingErrors' }); const byType = errorList.catalog.fn.facets({ attribute: 'errorType' }); console.log(byType.map((facet) => `${facet.key}: ${facet.size}`)); // [ 'InvalidValidator: 2', 'UndefinedValue: 1', 'WrongDataType: 1' ] errorList.catalog.query.addCondition({ attribute: 'errorType', operator: 'eq', value: 'InvalidValidator' }); console.log(errorList.catalog.query.items().map((e) => e.itemKey)); // [ 'u1', 'u3' ] ``` In SDK 2.0 the `errorType` facet leaves out `UnknownColumn`, `MissingKey`, `DuplicatePropertyKey`, `MissingProperty`, `MissingReference`, `UnknownExpressionVariable`, `ScriptingDisabled` and `TransformerError`. Filtering by these types works; for complete counts use `summary()`. ## Data import validation Check rows before you accept them, and report every problem at once: ```typescript import { Dcupl } from '@dcupl/core'; async function importProducts(rows: Record[]) { const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'sku', type: 'string', quality: { validators: { unique: {}, pattern: { value: '^[A-Z]{3}-[0-9]{4}$' }, }, }, }, { key: 'price', type: 'float', quality: { validators: { min: { value: 0 } } } }, { key: 'description', type: 'string', quality: { required: false } }, ], data: rows, }); await dcupl.init(); const summary = dcupl.quality.errors.summary(); const records = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'model', operator: 'eq', value: 'Product' }], }); return { success: summary.length === 0, summary, records }; } const result = await importProducts([ { key: 'p1', sku: 'LPT-0001', price: 999 }, { key: 'p2', sku: 'LPT-0001', price: 899 }, // duplicate sku { key: 'p3', sku: 'mouse', price: -5 }, // wrong pattern, negative price { key: 'p4', price: 'free' }, // no sku, price not a number ]); console.log(result.success); // false console.log(result.summary.map((s) => `${s.attribute}: ${s.errorType} (${s.totalCount})`)); // [ // 'sku: InvalidValidator (2)', // 'price: InvalidValidator (1)', // 'sku: UndefinedValue (1)', // 'price: WrongDataType (1)' // ] console.log(result.records.map((e) => `${e.itemKey ?? '-'}.${e.attribute}: ${e.description}`)); // [ // 'p3.sku: The value "mouse" does not match the pattern "^[A-Z]{3}-[0-9]{4}$"', // 'p3.price: The value "-5" is smaller than the minimum value of "0"', // 'p4.sku: The required string property "sku" in the "Product" model is undefined (no value)', // '-.price: 1 value(s) in column "price" of model "Product" failed to coerce to the declared type "float" (e.g. "free")', // 'p2.sku: The value "LPT-0001" is not unique' // ] ``` The failed coercion is one consolidated `WrongDataType` record per attribute, without an `itemKey`. These rows are inline data: the CSV-only checks (`UnknownColumn`, parsing of `true` / `false` cells) apply only to CSV files loaded through the [loader](/sdk/api/loader). ## Form validation integration Validate a single form entry with a model that describes the form: ```typescript import { Dcupl } from '@dcupl/core'; type FieldError = { key: string; description: string }; async function validateUser(formData: Record): Promise { // A fresh instance per check: its error records are complete after init() const dcupl = new Dcupl(); dcupl.models.set({ key: 'User', properties: [{ key: 'email', type: 'string', quality: { validators: { email: {} } } }], data: [{ key: 'form', ...formData }], // every item needs a key }); await dcupl.init(); return dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'model', operator: 'eq', value: 'User' }], }); } ``` Show the returned descriptions next to the field and submit only when the list is empty. Key the messages by the error record's `key`: one field can fail several validators. With a single item, a field that is missing altogether is reported as `UndefinedAttribute` (`The Attribute "email" is not present in the dataset for model "User"`), not as `UndefinedValue`, so map it to your own message. ## Troubleshooting ### Validation not running **Problem:** No validation errors despite invalid data **Solution:** Both levels are enabled by default, so check that neither `new Dcupl({ quality: { enabled: false } })` nor a model's `quality: { enabled: false }` switches them off. Validators never run on `boolean`, `json` and `date` properties, and don't run on `int` and `float` properties that use `forceStrictDataType: true` (see [Strict Data Type Checking](#strict-data-type-checking)). ### Too many errors **Problem:** Overwhelming number of validation errors **Solution:** Look at the summary first: one entry per group instead of one record per item. Many `UndefinedValue` errors usually mean optional fields that aren't marked `required: false` (attributes are required by default). `maxErrorsPerGroup` limits how many records are stored. ```typescript console.table(dcupl.quality.errors.summary()); // Mark optional fields, or make all attributes of a model optional { key: 'bio', type: 'string', quality: { required: false } } { key: 'LogEntry', quality: { attributes: { required: false } }, properties: [...] } // Validate critical fields first, add more validation gradually { key: 'email', quality: { required: true, validators: { email: {} } } } ``` ## Next steps - [Models](/sdk/core-concepts/models) — Complete property configuration - [Data management](/sdk/core-concepts/data-management) — Loading and updating data - [Derived properties](/sdk/advanced/derived-properties) — Computed values - [Expression properties](/sdk/advanced/expression-properties) — Template-based fields - [Performance](/sdk/advanced/performance) — Optimization strategies - [dcupl validate](/cli/validate) — Check a whole project's quality errors from the CLI --- # Performance optimization Source: https://docs.dcupl.com/sdk/advanced/performance Description: Optimize dcupl for speed and efficiency with large datasets # Performance optimization Learn how to optimize dcupl for large datasets, complex queries, and high-frequency updates. This guide covers the performance options, indexing, partial updates, batch operations, and profiling. ## Performance overview dcupl is designed to handle large datasets efficiently. With proper optimization: - **Initialize once** - `init()` processes all data; afterwards, apply changes with `dcupl.update()` - **Cached reads** - Each list caches its query results, facets and aggregations until the next update that affects it - **Partial updates** - `data.update()`, `upsert()` and `remove()` reprocess only the changed attributes and what depends on them - **Batched applies** - Stage many changes, then apply them with one `dcupl.update()` - **Efficient memory usage** - With list management ## Performance options These options go into `new Dcupl({ performance })`: | Option (default) | Effect | | ------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `cache.enabled` (`true`) | Per-list caches for query results, facets and aggregations (see [Caching](/sdk/advanced/caching)) | | `cache.maxSize` (`64 MiB`) | Bound, in estimated bytes, for each list's cached results and for the indices dcupl builds per model (or per scoped list) as it answers queries. Least recently used entries are dropped and rebuilt on their next use, so results are unchanged; `Infinity` removes the bound. The index bound also applies with `cache.enabled: false`. Lower it to cap memory in long-running processes | | `cache.maxProjectionSize` (`256 MiB`) | Bound, in estimated bytes per list, for the projected copies that deep facets and deep filters build (a dotted path such as `product.color` in a facet attribute, a list's query or an `fn.*` query), one per distinct set of dotted attributes. Least recently used copies are dropped and projected again on their next use; a copy larger than the bound is built and served but not kept, so raise the bound for deep facets on large lists. Results are unchanged; `Infinity` removes the bound. | | `clone.enabled` (`true`) | Every read returns a deep copy. `false` returns the stored items, which you must not mutate | | `containerBatchSize` (unset) | How often `init()` and full rebuilds yield to the event loop: unset after every data container (one staged `data.*` call), `0` never, `N` every N containers. Fewer yields finish sooner but block the UI longer | | `partialUpdates` (`0.5` and `0.5`) | Two thresholds for the share of changed models and items: above both, `dcupl.update()` runs a full rebuild instead of a partial update (see [When dcupl Falls Back](#when-dcupl-falls-back-to-a-full-update)) | | `retainStagedData` (`true`) | `false` makes `init()` drop the staged raw rows after a successful build, the same as `dcupl.seal()` (measured 15.6–21.8 % less retained heap after `init()` on a 135,880-row fixture on 2026-10-15, and 2–24 % on customer catalogs of 1,000–44,000 rows on 2026-10-16). For instances that load once and never update: afterwards every data or model write throws `DcuplSealedError`, `init()`/`update()` resolve `false`, and reads are unchanged (see [seal()](/sdk/api/dcupl#seal)) | | `autoUpdateLists.enabled` (`false`) | `query`-scoped lists re-run their seed query on every update that affects them | Quality checks also cost time in `init()`: `new Dcupl({ quality: { enabled: false } })` turns them off for all models, and a model's own `quality: { enabled: false }` for that model. ```typescript const dcupl = new Dcupl({ performance: { clone: { enabled: false }, // your code never mutates returned items containerBatchSize: 0, // process synchronously; blocks the UI while init() runs }, }); ``` The full option types are in the [Dcupl API](/sdk/api/dcupl#dcuplinitperformanceconfig). ## Indexing strategies ### Automatic indexing dcupl builds the indices it needs for facets, aggregations and grouping on first use, so you don't need to configure anything special for most use cases. References are indexed while data is processed, so `eq` and `in` conditions on a reference are index lookups. ```typescript const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'category', type: 'string' }, // Filterable, facet index built on first use { key: 'brand', type: 'string' }, // Filterable, facet index built on first use { key: 'price', type: 'float' }, // Filterable + aggregatable ], }; ``` ### Manual index property `index: true` builds an inverse index for a property while data is processed, so `eq` and `in` conditions on it are index lookups instead of a pass over the items. The index costs processing time and memory in `init()` and `update()`, so use it for properties you often filter by exact value (SKUs, IDs, status codes): ```typescript { key: 'sku', type: 'string', index: true, // Build an inverse index for eq/in lookups } ``` ## Initialization ### Initialize once, update incrementally `init()` processes every model and all staged data. Call it once at startup; afterwards stage changes and apply them with `await dcupl.update()`. Calling `init()` again re-processes everything. ```typescript dcupl.models.set(productModel); dcupl.data.set(initialProducts, { model: 'Product' }); await dcupl.init(); // Once // Later: stage and apply changes dcupl.data.update([{ key: 'p1', price: 150 }], { model: 'Product' }); await dcupl.update(); ``` ### Load critical data first To make the app usable sooner, initialize with the data the first screen needs, then load the rest in the background. The second load replaces the model with `data.set()`, so it is a full rebuild: ```typescript const firstPage = await fetch('/api/products?limit=100').then((r) => r.json()); dcupl.data.set(firstPage, { model: 'Product' }); await dcupl.init(); renderUI(); const allProducts = await fetch('/api/products').then((r) => r.json()); dcupl.data.set(allProducts, { model: 'Product' }); await dcupl.update(); ``` ## Partial updates A partial update reprocesses only the changed attributes and the attributes that depend on them, instead of rebuilding every model. All data calls are staged: nothing changes until `await dcupl.update()` applies them. ### Full update (slow) ```typescript // ❌ Slow: Replaces the model's data and rebuilds everything dcupl.data.set(updatedData, { model: 'Product' }); await dcupl.update(); // data.set() turns this into a full rebuild ``` ### Partial update (fast) ```typescript // ✅ Fast: Updates only changed items dcupl.data.update( [ { key: 'p1', price: 999 }, // Only update price { key: 'p2', stock: 50 }, // Only update stock ], { model: 'Product' }, ); await dcupl.update(); // Apply the staged changes ``` ### When dcupl falls back to a full update - A `data.set()` (or `data.reset()`) anywhere in the batch turns the whole update into a full rebuild. - A model definition changed after `init()`: a [`models.set()`](/sdk/api/models#set) or `models.update()` that registers a definition (even an unchanged one), or a `models.remove()` of a registered model, makes the next `dcupl.update()` rebuild all data like `init()` (the `dcupl_partial_update` event reports `isPartialUpdate: false`). The rebuild runs in the same instance, so lists created earlier serve the new definition, and data staged in the same batch is applied. Only that one update is affected; the next one follows the other rules again. - `dcupl.update()` with nothing staged runs a full rebuild, so don't call it "just in case". - If more than half of the models **and** more than half of their items changed, a full rebuild is cheaper and dcupl switches to it. Tune this with `modelRatioThreshold` and `itemRatioThreshold` in `performance.partialUpdates` (both default `0.5`). - When the data a [query reference](/sdk/api/models#reference) depends on changes — the rows of the model that defines it, its target model, or a model the target reaches through its references — `dcupl.update()` falls back to a full rebuild so the reference is recomputed (the `dcupl_partial_update` event then reports `isPartialUpdate: false`). Other changes stay on the partial path. ### Partial update with specific attributes Update only specific attributes. Every attribute you send replaces the stored value (arrays and JSON values are replaced, not merged); attributes you leave out keep their value. `data.update()` only changes existing items and ignores unknown keys; use `data.upsert()` to add items. ```typescript dcupl.data.update([{ key: 'p1', price: 999 }], { model: 'Product' }); await dcupl.update(); ``` ### Batch partial updates Update multiple items efficiently: stage all changes, then apply them with a single `dcupl.update()`. Every `dcupl.update()` call has a fixed cost (it checks every affected list and emits events), so one call per batch is far cheaper than one per item. ```typescript title="Update 1000 items" const updates = []; for (let i = 0; i < 1000; i++) { updates.push({ key: `p${i}`, stock: newStock[i], }); } dcupl.data.update(updates, { model: 'Product' }); await dcupl.update(); // One apply for all 1000 changes ``` ### Remove items efficiently ```typescript title="Remove specific items" dcupl.data.remove([{ key: 'p1' }, { key: 'p2' }, { key: 'p3' }], { model: 'Product' }); await dcupl.update(); ``` ## Query optimization ### Use indexed properties Every property is queryable, but only `eq` and `in` conditions on references and `index: true` properties are answered from an index. Other conditions evaluate the list's items. ```typescript // sku has index: true (see Manual Index Property): an index lookup productList.catalog.query.addCondition({ attribute: 'sku', operator: 'eq', value: 'LPT-001', }); // Range operators such as gte always evaluate the list's items productList.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 100, }); ``` ### Put the most selective condition first In an `and` group, each condition only evaluates the items the previous conditions left. Put the condition that matches the fewest items first, ideally one that is answered from an index: ```typescript productList.catalog.query.addGroup({ groupKey: 'filters', groupType: 'and', queries: [ { attribute: 'sku', operator: 'eq', value: 'LPT-001' }, // Index lookup, one item { attribute: 'inStock', operator: 'eq', value: true }, // Evaluates only what is left ], }); ``` ### Limit deep queries Each level of a dotted path resolves one more reference for every evaluated item, so the cost grows with the depth. A list that queries through other models is also invalidated whenever any model on the path changes. ```typescript // ⚠️ Slower: 3-level deep query orderList.catalog.query.addCondition({ attribute: 'customer.address.city', operator: 'eq', value: 'San Francisco', }); // ✅ Faster: Use derived properties // Customer model: derive the city from the address reference { key: 'city', type: 'string', derive: { localReference: 'address', remoteProperty: 'city' } } // Order model: derive it again from the customer reference { key: 'customerCity', type: 'string', derive: { localReference: 'customer', remoteProperty: 'city', // The Customer's derived city }, } // Then query directly orderList.catalog.query.addCondition({ attribute: 'customerCity', operator: 'eq', value: 'San Francisco', }); ``` ### Optimize query groups Every condition in an `or` group is evaluated against all items the group receives, and the results are merged. For several values of one attribute, use one `in` condition instead: it is answered from the index when there is one, and with a single pass over the items when there isn't. ```typescript // ⚠️ Slower: one evaluation per value productList.catalog.query.addGroup({ groupKey: 'category', groupType: 'or', queries: [ { attribute: 'category', operator: 'eq', value: 'Electronics' }, { attribute: 'category', operator: 'eq', value: 'Computers' }, { attribute: 'category', operator: 'eq', value: 'Phones' }, ], }); // ✅ Faster: one condition, same result productList.catalog.query.addCondition({ attribute: 'category', operator: 'in', value: ['Electronics', 'Computers', 'Phones'], }); ``` ### Use pagination For large result sets, request a page instead of the whole result. Pass paging as a **read option**: `start` (0-based index) and `count` (maximum items) are passed to the read itself, so the list holds the full filtered result and each call slices it. ```typescript const PAGE_SIZE = 50; const productList = dcupl.lists.create({ modelKey: 'Product' }); // First page const page1 = productList.catalog.query.items({ start: 0, count: PAGE_SIZE }); // Any page — no cursor to advance, just compute the offset function getPage(page: number) { return productList.catalog.query.items({ start: page * PAGE_SIZE, count: PAGE_SIZE, }); } // Total number of matching items, independent of paging const total = productList.catalog.query.count(); const pageCount = Math.ceil(total / PAGE_SIZE); ``` Because the page is resolved at read time, filters and paging compose without re-creating the list — apply a query, then read page 0 again: ```typescript productList.catalog.query.apply({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const firstPage = productList.catalog.query.items({ start: 0, count: PAGE_SIZE }); const filteredTotal = productList.catalog.query.count(); ``` > `count` means two different things here: as an **option** it is the page size, as a **method** > (`query.count()`) it is the number of matching items. The same `start`/`count` options work on > `query.execute()`, which takes the same `DcuplListQueryOptions`. They do **not** apply to > `query.many()`, which selects by `itemKeys` and accepts only `projection`. If you store paging > in the list's query with `query.applyOptions({ start, count })` instead, `items()` and `count()` > both return only that page while those options are applied (`query.clear()` removes them). ### Filter in dcupl, not on the result Let dcupl filter instead of filtering the returned array: `items()` followed by `Array.filter()` copies every item first (with cloning on), skips the indexes, and the filtered result isn't cached. ```typescript // ✅ Indexed where possible, cached per query productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); const electronics = productList.catalog.query.items(); // ❌ Copies every product, then filters in JavaScript const filtered = productList.catalog.query.items().filter((p) => p.category === 'Electronics'); ``` ## List management ### Reuse lists Every `lists.create()` registers a list that stays, with its caches, until you destroy it, and `dcupl.update()` checks every registered list. A list created per call therefore piles up. ```typescript // ❌ Bad: Create new list for each query function getProducts(category) { const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category, }); return list.catalog.query.items(); } // ✅ Good: Reuse list const productList = dcupl.lists.create({ modelKey: 'Product' }); function getProducts(category) { productList.catalog.query.clear(); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category, }); return productList.catalog.query.items(); } ``` ### Destroy lists Destroy lists when no longer needed: ```typescript const tempList = dcupl.lists.create({ modelKey: 'Product' }); // ... use list // Clean up tempList.destroy(); ``` ### List pooling For frequently created/destroyed lists: ```typescript class ListPool { private pool: Map = new Map(); get(modelKey: string): DcuplList { const pool = this.pool.get(modelKey) || []; return pool.pop() || dcupl.lists.create({ modelKey }); } release(list: DcuplList) { list.catalog.query.clear(); const modelKey = list.modelKey; const pool = this.pool.get(modelKey) || []; pool.push(list); this.pool.set(modelKey, pool); } } const pool = new ListPool(); const list = pool.get('Product'); // ... use list pool.release(list); ``` ## Memory optimization ### Limit data loading Load only needed data: ```typescript // ❌ Bad: Load all data const allProducts = await fetch('/api/products').then((r) => r.json()); dcupl.data.set(allProducts, { model: 'Product' }); // ✅ Good: Load filtered data const activeProducts = await fetch('/api/products?active=true').then((r) => r.json()); dcupl.data.set(activeProducts, { model: 'Product' }); ``` ### Remove unused properties Don't define properties you won't use. dcupl stores only the attributes the model declares (unless the model uses `autoGenerateProperties`), so every declared property costs memory on every item: ```typescript // ❌ Bad: Include everything properties: [ { key: 'id', type: 'string' }, { key: 'name', type: 'string' }, { key: 'legacyField1', type: 'string' }, { key: 'legacyField2', type: 'string' }, { key: 'internalNote', type: 'string' }, // ... ]; // ✅ Good: Only needed properties properties: [ { key: 'id', type: 'string' }, { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ]; ``` ### Clear unused data Remove data that's no longer needed: ```typescript title="Remove specific items" dcupl.data.remove([{ key: 'old1' }, { key: 'old2' }, { key: 'old3' }], { model: 'Product' }); await dcupl.update(); // Clear all data for a model dcupl.data.reset({ model: 'Product' }); await dcupl.update(); ``` ## Large dataset handling ### Load data in chunks When an API returns the data in pages, stage every page with `upsert()` and call `init()` once at the end. Staged data is processed in one pass, so the chunks cost one processing run instead of one per chunk: ```typescript async function loadLargeDataset() { const chunkSize = 10000; let offset = 0; while (true) { const chunk = await fetch(`/api/products?offset=${offset}&limit=${chunkSize}`).then((r) => r.json()); if (chunk.length === 0) break; // upsert adds to the staged data - set() would replace the previous chunks dcupl.data.upsert(chunk, { model: 'Product' }); offset += chunkSize; } await dcupl.init(); } ``` ### Virtualization For rendering large lists: ```tsx title="Use virtual scrolling libraries" import { FixedSizeList } from 'react-window'; // Create the list once, not on every render const productList = dcupl.lists.create({ modelKey: 'Product' }); function ProductList() { const products = productList.catalog.query.items(); return ( {({ index, style }) =>
{products[index].name}
}
); } ``` ## Profiling and debugging ### Measure init performance ```typescript console.time('init'); await dcupl.init(); console.timeEnd('init'); ``` ### Timings from events The events dcupl emits for its methods carry a `duration` in milliseconds. Subscribe with `dcupl.on()` for instance-wide events and `list.on()` for a list's queries and functions: ```typescript dcupl.on((msg) => { if (msg.type === 'dcupl_initialized' || msg.type === 'dcupl_updated_manually') { console.log(msg.type, msg.duration); } }); productList.on((msg) => { if (msg.type === 'query_execute' || msg.type === 'fn_facets') { console.log(msg.type, msg.duration); } }); ``` ### Measure query performance ```typescript const list = dcupl.lists.create({ modelKey: 'Product' }); console.time('query'); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const results = list.catalog.query.items(); console.timeEnd('query'); ``` ### Measure update performance ```typescript console.time('update'); dcupl.data.update([{ key: 'p1', price: 999 }], { model: 'Product' }); await dcupl.update(); // The work happens here - data calls only stage changes console.timeEnd('update'); ``` ## Practical examples ### Example 1: e-commerce product catalog ```typescript title="Catalog model and queries" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'sku', type: 'string', index: true }, { key: 'name', type: 'string' }, { key: 'brand', type: 'string' }, { key: 'price', type: 'float' }, { key: 'inStock', type: 'boolean' }, // Derived for faster queries; index: true makes eq/in on it index lookups { key: 'categoryName', type: 'string', index: true, derive: { localReference: 'category', remoteProperty: 'name', }, }, // Display only { key: 'description', type: 'string' }, { key: 'images', type: 'json' }, ], references: [{ key: 'category', type: 'singleValued', model: 'Category' }], }; // Reuse list const productList = dcupl.lists.create({ modelKey: 'Product' }); const PAGE_SIZE = 50; // Fast queries function searchProducts(filters, page = 0) { productList.catalog.query.clear(); if (filters.category) { productList.catalog.query.addCondition({ attribute: 'category', // Reference - matched by key, answered from its index operator: 'eq', value: filters.category, }); } if (filters.inStock) { productList.catalog.query.addCondition({ attribute: 'inStock', operator: 'eq', value: true, }); } return productList.catalog.query.items({ start: page * PAGE_SIZE, count: PAGE_SIZE }); } // Efficient updates // data.update matches items by their key, not by sku async function updatePrice(productKey, newPrice) { dcupl.data.update([{ key: productKey, price: newPrice }], { model: 'Product' }); await dcupl.update(); } ``` ### Example 2: real-time dashboard ```typescript title="Optimized for frequent updates" const metricsModel: ModelDefinition = { key: 'Metric', properties: [ { key: 'timestamp', type: 'date' }, { key: 'type', type: 'string' }, { key: 'value', type: 'float' }, ], }; dcupl.models.set(metricsModel); await dcupl.init(); // Fast aggregation const metricsList = dcupl.lists.create({ modelKey: 'Metric' }); // gte/lte on a date property match Date objects or timestamps, not ISO strings function getMetrics(type: string, startDate: Date, endDate: Date) { metricsList.catalog.query.clear(); metricsList.catalog.query.addGroup({ groupKey: 'filters', groupType: 'and', queries: [ { attribute: 'type', operator: 'eq', value: type }, { attribute: 'timestamp', operator: 'gte', value: startDate.getTime() }, { attribute: 'timestamp', operator: 'lte', value: endDate.getTime() }, ], }); return metricsList.catalog.fn.aggregate({ attribute: 'value', types: ['sum', 'avg', 'min', 'max'], }); } // Efficient real-time updates: buffer incoming metrics, // add them with upsert and apply them in one update per second type Metric = { key: string; timestamp: Date; type: string; value: number }; const pending: Metric[] = []; function addMetric(metric: Metric) { pending.push(metric); } setInterval(async () => { if (pending.length === 0) return; dcupl.data.upsert(pending.splice(0), { model: 'Metric' }); await dcupl.update(); }, 1000); ``` ### Example 3: order management ```typescript const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderId', type: 'string', index: true }, { key: 'status', type: 'string' }, { key: 'total', type: 'float' }, { key: 'createdAt', type: 'date' }, // Derived for fast queries; index: true makes eq/in on it index lookups { key: 'customerEmail', type: 'string', index: true, derive: { localReference: 'customer', remoteProperty: 'email', }, }, ], references: [{ key: 'customer', type: 'singleValued', model: 'Customer' }], }; // Efficient status updates async function updateOrderStatus(orderKey, newStatus) { dcupl.data.update([{ key: orderKey, status: newStatus }], { model: 'Order' }); await dcupl.update(); } // Fast customer order lookup const orderList = dcupl.lists.create({ modelKey: 'Order' }); function getCustomerOrders(email) { orderList.catalog.query.clear(); orderList.catalog.query.addCondition({ attribute: 'customerEmail', // Derived + indexed: an index lookup, no reference resolution operator: 'eq', value: email, }); return orderList.catalog.query.items(); } ``` ### Example 4: search performance ```typescript title="Optimized search model" const articleModel: ModelDefinition = { key: 'Article', properties: [ { key: 'title', type: 'string' }, { key: 'slug', type: 'string', index: true }, { key: 'tags', type: 'Array' }, { key: 'authorName', type: 'string' }, { key: 'status', type: 'string' }, { key: 'publishedAt', type: 'date' }, // Content fields { key: 'content', type: 'string' }, { key: 'excerpt', type: 'string' }, ], }; const articleList = dcupl.lists.create({ modelKey: 'Article' }); const escapeRegex = (text: string) => text.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); function searchArticles(query: string) { articleList.catalog.query.clear(); // Conditions on different attributes are combined with AND, // so group them with groupType 'or'. // '/term/' = substring search; regexFlags 'i' = case-insensitive const pattern = `/${escapeRegex(query)}/`; articleList.catalog.query.addGroup({ groupKey: 'search', groupType: 'or', queries: [ { attribute: 'title', operator: 'find', value: pattern, options: { regexFlags: 'i' } }, { attribute: 'tags', operator: 'find', value: pattern, options: { regexFlags: 'i' } }, { attribute: 'authorName', operator: 'find', value: pattern, options: { regexFlags: 'i' } }, ], }); return articleList.catalog.query.items({ start: 0, count: 20 }); } ``` `find` evaluates every item of the list, so text search cost grows with the list. See [Text search](/sdk/queries/query-search) for the matching rules. ### Example 5: inventory management ```typescript const inventoryModel: ModelDefinition = { key: 'Inventory', properties: [ { key: 'sku', type: 'string', index: true }, { key: 'warehouse', type: 'string' }, { key: 'location', type: 'string' }, { key: 'quantity', type: 'int' }, { key: 'reorderPoint', type: 'int' }, { key: 'lastUpdated', type: 'date' }, ], }; // Batch stock updates async function updateStock(updates) { const data = updates.map((u) => ({ key: u.key, // The item key - data.update ignores keys it doesn't know quantity: u.quantity, lastUpdated: new Date().toISOString(), })); dcupl.data.update(data, { model: 'Inventory' }); await dcupl.update(); } // Fast low-stock query const inventoryList = dcupl.lists.create({ modelKey: 'Inventory' }); function getLowStock() { inventoryList.catalog.query.clear(); inventoryList.catalog.query.addCondition({ attribute: 'quantity', operator: 'lte', value: 10, }); return inventoryList.catalog.query.items(); } ``` ## Performance benchmarks Timings depend on your models (references, derived and expression properties), your data and the device, so measure with your own data using the profiling snippets above. As a reference point: measured with 1 million items, indexed queries took 25–65 ms, facets about 2.7 s. What holds in general: - **One `dcupl.update()` per batch.** Staging 100 changes and applying them with a single `dcupl.update()` is far faster than calling `dcupl.update()` after each change. - **Partial beats full.** `data.update()`, `upsert()` and `remove()` reprocess only the changed attributes; `data.set()` rebuilds the whole model and everything that depends on it. - **Repeated reads are cached.** Re-reading a list's result for an unchanged query skips query evaluation; what remains is the per-read deep copy, which you can turn off with `performance.clone.enabled: false` if you never mutate returned items. ## Performance checklist ### Model setup - ✅ Use derived properties for common deep queries - ✅ Remove unused properties - ✅ Use appropriate data types - ✅ Use `index: true` for frequently looked-up keys ### Querying - ✅ Filter by exact value on references or `index: true` properties - ✅ Put the most selective condition first in `and` groups - ✅ Use `in` instead of an `or` group of `eq` conditions on one attribute - ✅ Avoid deep queries (use derived properties) - ✅ Use pagination for large results - ✅ Reuse lists instead of creating new ones - ✅ Keep query groups simple ### Updates - ✅ Call `init()` once, then apply changes with `dcupl.update()` - ✅ Use partial updates (`data.update()`, `upsert()`, `remove()`) - ✅ Stage a batch of changes, then call `await dcupl.update()` once - ✅ Avoid full `init()` after updates ### Memory - ✅ Load only needed data - ✅ Remove unused data - ✅ Destroy unused lists - ✅ Use virtualization for large lists ## Troubleshooting ### Slow queries **Problem:** Queries take too long **Solution:** 1. Measure query time: ```typescript console.time('query'); const results = list.catalog.query.items(); console.timeEnd('query'); ``` 2. Simplify the query: put indexed, selective conditions first and replace `or` groups of `eq` conditions with one `in` condition: ```typescript // Instead of { groupType: 'or', queries: [{ attribute: 'status', operator: 'eq', value: 'new' }, { ..., value: 'sale' }] } list.catalog.query.addCondition({ attribute: 'status', operator: 'in', value: ['new', 'sale'] }); ``` 3. Use derived properties for deep queries ### Slow updates **Problem:** Updates are slow **Solution:** Use partial updates: ```typescript // ❌ Slow: full rebuild dcupl.data.set(allData, { model: 'Product' }); await dcupl.update(); // ✅ Fast dcupl.data.update([{ key: 'p1', price: 999 }], { model: 'Product' }); await dcupl.update(); ``` ### High memory usage **Problem:** Memory usage is too high **Solution:** 1. Remove unused properties 2. Load filtered data 3. Clear unused data 4. Destroy unused lists ### Slow init **Problem:** `dcupl.init()` takes too long **Solution:** 1. Reduce data size 2. Remove unused models 3. Disable quality validation for non-critical models (`quality: { enabled: false }` on the model), or for all models with `new Dcupl({ quality: { enabled: false } })` 4. Try `performance: { containerBatchSize: 0 }`: `init()` no longer yields to the event loop between data containers, so it finishes sooner but blocks the UI while it runs 5. Profile with [event timings](#timings-from-events) ## Next steps - [Caching](/sdk/advanced/caching) — Caching strategies - [Data management](/sdk/core-concepts/data-management) — Loading and updating data - [Lists](/sdk/core-concepts/lists) — Working with lists - [Queries](/sdk/queries/query-filter) — Query optimization - [Models](/sdk/core-concepts/models) — Property indexing - [Dcupl API](/sdk/api/dcupl) — All constructor options --- # Caching strategies Source: https://docs.dcupl.com/sdk/advanced/caching Description: How dcupl caches query results per list, when it drops those caches, and the options that control them # Caching strategies dcupl caches query results per list. This page explains what is cached, when a cache is dropped, and which options control it. dcupl keeps all data in memory; for keeping data across page loads, see [Caching Outside dcupl](#caching-outside-dcupl). ## Understanding dcupl caching ```mermaid flowchart LR Q[Read on a list] --> C{Cached for this query?} C -->|Yes| R[Return Cached] C -->|No| E[Execute Query] E --> S[Store in the list's cache] S --> R2[Return Results] U["dcupl.update()"] --> I[Drop caches of affected lists] ``` dcupl caches query results **per list**. Each list keeps the results of the queries it has executed, keyed by the query, together with the facets and aggregations computed on them. Global reads (`dcupl.query.execute()`, `dcupl.fn.*`) are cached the same way, per model, and refreshed on every `dcupl.update()` that affects that model. A list's cache is dropped when: - `dcupl.update()` (or `init()`) applies a data change to the list's model, to a model it depends on through a reference or a derived or resolved attribute, or to a model the list reads through a dotted path such as `customer.city` - you call `list.invalidate()` or `list.update()` Changing the query does not drop the cache: the new query gets its own entry, and switching back to an earlier query reuses its cached result. The cache key holds the full query, so two queries share an entry only when they are equal. **Key concepts:** - **Automatic caching** - Query results, facets, aggregations, groupBy, pivot and suggest results are cached by default - **Cache invalidation** - Every `dcupl.update()` invalidates the affected lists - **Manual control** - `list.invalidate()` drops a list's caches - **List-level caching** - Each list maintains its own cache ## Automatic query caching ### How it works ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // First read - the result is computed and stored in the list's cache const results1 = productList.catalog.query.items(); // Second read - served from the cache, the query is not evaluated again const results2 = productList.catalog.query.items(); // Every read returns a deep copy by default console.log(results1 === results2); // false console.log(results1.length === results2.length); // true ``` By default every read returns a deep copy (`performance.clone.enabled: true`), so changing a returned item cannot corrupt the data inside dcupl. The copy is made on every read, cache hit or not. If your application never mutates returned items, you can turn cloning off; repeated reads then return the same cached array: ```typescript title="Without cloning" const dcupl = new Dcupl({ performance: { clone: { enabled: false } } }); // ... models, data, await dcupl.init(), list and query as above const results1 = productList.catalog.query.items(); const results2 = productList.catalog.query.items(); console.log(results1 === results2); // true ``` ### Cache scope Each list maintains its own query cache: ```typescript const list1 = dcupl.lists.create({ modelKey: 'Product' }); const list2 = dcupl.lists.create({ modelKey: 'Product' }); list1.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); list2.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Clothing', }); // Each list has independent cache const electronics = list1.catalog.query.items(); // Cached for list1 const clothing = list2.catalog.query.items(); // Cached for list2 ``` ## Cache invalidation ### Automatic invalidation Data changes are staged and take effect on the next `await dcupl.update()`. That call invalidates the caches of every affected list: ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); productList.catalog.query.items().map((p) => p.key); // ['p1', 'p2'] - cached // Stage a change - nothing is applied yet dcupl.data.update([{ key: 'p1', category: 'Clothing' }], { model: 'Product' }); productList.catalog.query.items().map((p) => p.key); // ['p1', 'p2'] // Apply it - the list's caches are dropped await dcupl.update(); productList.catalog.query.items().map((p) => p.key); // ['p2'] - re-computed ``` On each `dcupl.update()` every affected list drops its query and result caches and recomputes lazily on the next read; lists nobody reads do no work. An unscoped list like this one therefore always shows current data, with or without `autoUpdate`. Scoped lists are different, see [list.update()](#listupdate). ### Manual cache invalidation You rarely need this, because `dcupl.update()` already invalidates every affected list. Call `list.invalidate()` when a list's results depend on something dcupl cannot see, for example a custom operator that reads application state: ```typescript import type { DcuplQuery } from '@dcupl/common'; let maxPrice = 1000; dcupl.query.registerCustomOperator('belowMaxPrice', (query, item) => item.price < maxPrice); const productList = dcupl.lists.create({ modelKey: 'Product' }); // DcuplQuery.operator only lists the built-in operators, so cast the condition productList.catalog.query.addCondition({ attribute: 'price', operator: 'belowMaxPrice', value: true, } as unknown as DcuplQuery); productList.catalog.query.items(); // p1, p2, p3 - cached maxPrice = 100; productList.catalog.query.items(); // p1, p2, p3 - still the cached result // Drop the list's caches; the next read recomputes productList.invalidate(); productList.catalog.query.items(); // p3 ``` `query.clear()` is not a way to invalidate: it removes the conditions, but re-applying the same query afterwards returns the cached result again. `registerCustomOperator` still works but is deprecated; for new code, register an `operator` script instead (see [Scripts](/sdk/guides/scripts)). ### Partial cache invalidation Invalidation is selective per list, not per attribute. `dcupl.update()` only touches the lists affected by the change: lists on a changed model, lists on a model that depends on it (through a reference, or a derived or resolved attribute), and lists whose dotted paths (such as `customer.city`) read through a changed model. An affected list drops all of its cached results, even if the changed attribute is not part of its query: ```typescript title="Only affected lists are invalidated" const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const categoryList = dcupl.lists.create({ modelKey: 'Category' }); productList.catalog.query.items(); // Cached categoryList.catalog.query.items(); // Cached // Update a property the query does not use dcupl.data.update([{ key: 'p1', stock: 100 }], { model: 'Product' }); await dcupl.update(); productList.catalog.query.items(); // Re-computed: Product changed categoryList.catalog.query.items(); // Still cached: Category is not affected ``` > **Note:** The `cacheInvalidationThreshold` option in `performance.partialUpdates` was removed in SDK 2.0: passing it is a type error, and at runtime it is ignored. The per-list caches and the indices dcupl builds per model are bounded by `performance.cache.maxSize` instead (default 64 MiB; least recently used entries are dropped and rebuilt on their next use; see [DcuplInitPerformanceConfig](/sdk/api/dcupl#dcuplinitperformanceconfig)). ## List update and cache ### list.update() `dcupl.update()` already gives every unscoped list fresh results. `list.update()` re-derives a **scoped** list, one created with a seed `query` or with `listItemKeys`. A scoped list keeps its key set (the `listItemKeys` you pass, or the keys the query matched at creation) but always serves the model's current items: after any `dcupl.update()`, partial or full rebuild, changed values show, removed items drop out, and an item added under one of the keys is served. For a `listItemKeys` list that is all there is, so `list.update()` and `autoUpdate` change nothing for its items. For a `query`-scoped list without `autoUpdate`, items that start matching later only join when you call `list.update()`. ```typescript const electronics = dcupl.lists.create({ modelKey: 'Product', query: { modelKey: 'Product', queries: [{ attribute: 'category', operator: 'eq', value: 'Electronics' }], }, }); electronics.catalog.query.items().map((p) => p.key); // ['p1', 'p2'] // Add a new Electronics product dcupl.data.upsert([{ key: 'p5', name: 'Tablet', category: 'Electronics', price: 399 }], { model: 'Product' }); await dcupl.update(); electronics.catalog.query.items().map((p) => p.key); // ['p1', 'p2'] - scope unchanged // Re-run the seed query: new matches join (current values already show) electronics.update(); electronics.catalog.query.items().map((p) => p.key); // ['p1', 'p2', 'p5'] ``` `list.update()` is synchronous and emits a `list_updated` event on the list. ### autoUpdate option With `autoUpdate: true`, a `query`-scoped list re-runs its seed query on every `dcupl.update()` that affects it, so new matches join and items that stop matching leave (current values show either way; a `listItemKeys` list serves the same items with or without it): ```typescript const electronics = dcupl.lists.create({ modelKey: 'Product', query: { modelKey: 'Product', queries: [{ attribute: 'category', operator: 'eq', value: 'Electronics' }], }, autoUpdate: true, // Re-run the seed query on every update }); dcupl.data.upsert([{ key: 'p5', name: 'Tablet', category: 'Electronics', price: 399 }], { model: 'Product' }); await dcupl.update(); electronics.catalog.query.items().map((p) => p.key); // ['p1', 'p2', 'p5'] ``` On an unscoped list, `autoUpdate` makes no visible difference: it already sees every change after `dcupl.update()`. To enable it for all lists, use `new Dcupl({ performance: { autoUpdateLists: { enabled: true } } })` (default `false`); for a single update, use `dcupl.update({ autoUpdateLists: true })`. The per-update value overrides the list's setting, which overrides the global one. > **Watch out:** `autoUpdate` on `lists.create()` works at runtime, but the TypeScript type of `create()` does not declare it, so TypeScript reports an error for it. **Note:** Invalidation is lazy, but `autoUpdate` re-runs the seed query eagerly on every update that affects the list. Use it only on `query`-scoped lists that must pick up new matches. ## Facet caching Facets are also cached automatically: ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // First call - computed and cached const facets1 = productList.catalog.fn.facets({ attribute: 'category' }); // Second call with the same options - served from the list's cache const facets2 = productList.catalog.fn.facets({ attribute: 'category' }); ``` As with `items()`, each call returns a deep copy unless `performance.clone.enabled` is `false`. ### Facet cache invalidation ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); productList.catalog.fn.facets({ attribute: 'category' }).map((f) => `${f.value}: ${f.size}`); // ['Electronics: 2', 'Clothing: 1'] // Update data - the facet cache is dropped by dcupl.update() dcupl.data.update([{ key: 'p1', category: 'Computers' }], { model: 'Product' }); await dcupl.update(); productList.catalog.fn.facets({ attribute: 'category' }).map((f) => `${f.value}: ${f.size}`); // ['Computers: 1', 'Electronics: 1', 'Clothing: 1'] - re-computed ``` ## Aggregation caching Aggregations are cached like queries: ```typescript const orderList = dcupl.lists.create({ modelKey: 'Order' }); orderList.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'completed', }); // First call - computed const stats1 = orderList.catalog.fn.aggregate({ attribute: 'total', types: ['sum', 'avg', 'count'], }); // Second call with the same options - cached const stats2 = orderList.catalog.fn.aggregate({ attribute: 'total', types: ['sum', 'avg', 'count'], }); ``` ## Cache configuration The built-in caches are controlled by the `performance` options of the `Dcupl` constructor: ```typescript title="Cache-related options (defaults shown)" const dcupl = new Dcupl({ performance: { cache: { enabled: true, // Per-list query, facet and aggregation caches maxSize: 64 * 1024 * 1024, // Bound in estimated bytes per list cache and per model's indices; Infinity = none maxProjectionSize: 256 * 1024 * 1024, // Bound per list for the projected copies of deep facets and deep filters; Infinity = none }, clone: { enabled: true }, // Return a deep copy on every read autoUpdateLists: { enabled: false }, // true: re-run query-scoped lists' seed queries on every update }, }); ``` `cache.enabled` applies to every list of the instance; there is no per-list switch. See [Dcupl options](/sdk/api/dcupl) for the other `performance` options. The built-in cache has no TTL. Its size is bounded by `performance.cache.maxSize` (default 64 MiB, estimated; `Infinity` for no bound): when a new entry takes a list's cache over the bound, the least recently used entries are dropped and rebuilt on their next use, so results don't change, only repeated queries may recompute. The same bound applies, separately, to the indices dcupl builds per model (or per scoped list), also with the cache disabled. Within the bound, cached results live until the next `dcupl.update()` that affects the list (or `list.invalidate()`), and they are released together with the list when you call `list.destroy()`. Every distinct query adds entries (its result and its filtered data, plus any facets and aggregations computed on it), so a list driven by free text, such as search-as-you-type over data that rarely changes, fills its cache up to the bound. Lower `maxSize` to cap that memory, call `list.invalidate()` on such a list from time to time (for example when the search field loses focus), or turn the cache off with `performance.cache.enabled: false`. Deep facets and deep filters (a dotted path such as `product.color` in a facet attribute, a list's query or an `fn.*` query) project a copy of the list per distinct set of dotted attributes. These copies have their own bound, `performance.cache.maxProjectionSize` (default 256 MiB per list, estimated; `Infinity` for no bound), separate from `maxSize` so that broad query results don't push a projection out: the least recently used copies are dropped and projected again on their next use, and a copy larger than the bound is built and served but not kept. Like the result cache, projections are dropped on every update that affects the list, and with `cache.enabled: false` every deep facet or deep filter projects again. ## Keeping your own caches in sync `dcupl.on()` receives every event of the instance. `dcupl_partial_update` is emitted after every `dcupl.update()` that applied staged changes, including a full rebuild after `data.set()` or `data.reset()`; `msg.args[0].changedModels` lists the models that changed. The next `dcupl.update()` after a model definition change (`models.set()`, `models.update()` or `models.remove()` after `init()`) emits it too, even with no data staged. It is not emitted by `init()` or by an `update()` with nothing staged and no model definition changed since the last build. Use it to drop entries of your own caches: ```typescript title="Invalidate an application cache per model" // Your own cache of derived results, keyed by model const appCache = new Map(); dcupl.on((msg) => { if (msg.type !== 'dcupl_partial_update') return; for (const model of msg.args[0].changedModels) { appCache.delete(model); } }); ``` ## Caching outside dcupl dcupl holds its data in memory only, so nothing survives a page reload. To start faster on the next visit, store the raw data you loaded (IndexedDB for large datasets, `localStorage` for small settings) and stage it before `init()`; then fetch fresh data in the background and apply it with `dcupl.update()`. Expiry, versioning and offline handling are up to your application. ```typescript title="Start from stored data, refresh in the background" // loadStored / saveStored / fetchProducts are your own storage and API helpers const stored = await loadStored('products'); dcupl.data.set(stored ?? (await fetchProducts()), { model: 'Product' }); await dcupl.init(); const fresh = await fetchProducts(); dcupl.data.set(fresh, { model: 'Product' }); if (await dcupl.update()) { await saveStored('products', fresh); } ``` `init()` and `update()` don't throw when processing fails: they log the error and resolve to `false`, so check the result before you rely on the new data. ## Best practices ### 1. Leverage automatic caching ```typescript // Good: Reuse lists const list = dcupl.lists.create({ modelKey: 'Product' }); function getProducts(category: string) { list.catalog.query.clear(); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category, }); return list.catalog.query.items(); // Cached } // Bad: Create new lists function getProducts(category: string) { const list = dcupl.lists.create({ modelKey: 'Product' }); // ... query // No caching benefit, and every list stays registered (and is checked // on every dcupl.update()) until you call list.destroy() } ``` ### 2. Use autoUpdate sparingly Unscoped lists see every data change after `dcupl.update()`. `autoUpdate` re-runs a `query`-scoped list's seed query, eagerly, on every update that affects it; current values show without it, and a `listItemKeys` list serves the same items with or without it. ```typescript // Good: Scoped list that must pick up newly matching items const alertList = dcupl.lists.create({ modelKey: 'Metric', query: { modelKey: 'Metric', queries: [{ attribute: 'status', operator: 'eq', value: 'alert' }], }, autoUpdate: true, }); // Bad: Unscoped list - no visible effect, it is always up to date const categoryList = dcupl.lists.create({ modelKey: 'Category', autoUpdate: true, // Unnecessary }); ``` ### 3. Clear data when needed ```typescript // Good: Clear on logout async function logout() { // Destroys every list. dcupl.update() no longer refreshes a destroyed list, // so re-create the lists your UI reads instead of keeping the old objects dcupl.lists.destroy(); for (const model of ['Order', 'Customer']) { dcupl.data.reset({ model }); // Staged - applied by dcupl.update() } await dcupl.update(); // Also clear what your application stored outside dcupl } // Bad: Never clear function logout() { // Stale data remains } ``` ## Troubleshooting ### Stale data **Problem:** Cache returns outdated data **Solution:** Apply staged changes with `dcupl.update()` (it invalidates every affected list), and invalidate your own caches: ```typescript dcupl.data.update([{ key: 'p1', price: 999 }], { model: 'Product' }); // Apply the change - affected lists recompute on the next read await dcupl.update(); // Query-scoped lists without autoUpdate: re-run the seed query to pick up // new matches (current values and listItemKeys lists need nothing) list.update(); // Caches outside dcupl: drop them yourself, for example on dcupl_partial_update ``` ### Cache growing too large **Problem:** Memory usage increases although the data doesn't change **Solution:** A list keeps one cache entry per distinct query until the next update that affects it, up to `performance.cache.maxSize` (default 64 MiB; see [Cache Configuration](#cache-configuration)). Lower the bound to cap memory, drop the caches of lists driven by free text from time to time, destroy lists you no longer read, or turn caching off: ```typescript // A smaller bound: 16 MiB per list cache const dcupl = new Dcupl({ performance: { cache: { enabled: true, maxSize: 16 * 1024 * 1024 } } }); searchList.invalidate(); // drop this list's cached results tempList.destroy(); // release a list and its caches // No caching at all: new Dcupl({ performance: { cache: { enabled: false } } }) ``` ### Inconsistent results **Problem:** Different lists show different data **Solution:** Ensure single source of truth: ```typescript // Good: Single dcupl instance const dcupl = new Dcupl(); const list1 = dcupl.lists.create({ modelKey: 'Product' }); const list2 = dcupl.lists.create({ modelKey: 'Product' }); // Bad: Multiple instances const dcupl1 = new Dcupl(); const dcupl2 = new Dcupl(); ``` ## Next steps - [Performance](/sdk/advanced/performance) — Performance optimization - [Data management](/sdk/core-concepts/data-management) — Loading and updating data - [Lists](/sdk/core-concepts/lists) — Working with lists - [Queries](/sdk/queries/query-filter) — Query optimization --- # Faceted search Source: https://docs.dcupl.com/sdk/advanced/faceted-search Description: Use facets() for filter options and addCondition() for filtering # Faceted search Faceted search lets users narrow down results by selecting filter values. dcupl provides `facets()` to get available options with counts and `addCondition()` to filter results. ## Key dcupl functions | Function | Purpose | | ------------------------- | ---------------------------------------------------------- | | `catalog.fn.facets()` | Get filter options with counts for an attribute | | `query.addCondition()` | Add a value; values on the same attribute are ORed | | `query.setCondition()` | Replace the conditions on that condition's attribute | | `query.removeCondition()` | Remove one value that `addCondition()` added | | `query.removeGroup()` | Remove a whole group, for example every value of one facet | | `query.addGroup()` | Add a group of conditions with its own key and AND/OR type | | `query.clear()` | Remove all active filters (and sort/paging options) | ## Example data The outputs on this page come from this setup: ```typescript title="Setup" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'category', type: 'string' }, { key: 'brand', type: 'string' }, ], }); dcupl.data.set( [ { key: 'p1', name: 'Laptop', category: 'Electronics', brand: 'Apple' }, { key: 'p2', name: 'Phone', category: 'Electronics', brand: 'Samsung' }, { key: 'p3', name: 'Tablet', category: 'Electronics', brand: 'Apple' }, { key: 'p4', name: 'Jacket', category: 'Clothing', brand: 'Patagonia' }, { key: 'p5', name: 'Novel', category: 'Books', brand: 'Penguin' }, { key: 'p6', name: 'Cookbook', category: 'Books', brand: 'Penguin' }, { key: 'p7', name: 'Headphones', category: 'Electronics', brand: 'Sony' }, ], { model: 'Product' }, ); await dcupl.init(); ``` ## Getting facets Use `facets()` to get available filter values with their counts. ```typescript const list = dcupl.lists.create({ modelKey: 'Product' }); const categoryFacets = list.catalog.fn.facets({ attribute: 'category', }); // categoryFacets[0]: // { key: 'Electronics', value: 'Electronics', selected: false, enabled: true, size: 4, // entry: { key: 'Electronics' }, resultKeys: ['p1', 'p2', 'p3', 'p7'] } ``` Each entry's count is `size`. Counts are disjunctive: each value is counted under every _other_ group of the list's query. The group keyed by the faceted attribute is left out, so after selecting "Electronics" the category facet still lists every category with its count, while a filter on another attribute (for example `brand`) reduces the counts. Values without matching items are left out unless you pass `excludeZeros: false`. ## Applying filters Use `addCondition()` to filter results based on user selections. ```typescript // Single filter list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Electronics', }); const results = list.catalog.query.items(); // p1, p2, p3, p7 ``` ### Multi-select filters (OR logic) Call `addCondition()` once per selected value. Conditions on the same attribute go into one `or` group keyed by the attribute, so they are ORed, and `facets({ attribute })` leaves that group out without any option: ```typescript // User ticks "Electronics" and "Clothing" list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Electronics' }); list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Clothing' }); list.catalog.query.items(); // p1, p2, p3, p7, p4 list.catalog.fn.facets({ attribute: 'category' }).map((f) => `${f.value}: ${f.size}${f.selected ? ' ✓' : ''}`); // ['Electronics: 4 ✓', 'Clothing: 1 ✓', 'Books: 2'] ``` One `in` condition gives the same result and the same facets, and is handy when the UI hands you the whole selection at once: ```typescript list.catalog.query.setCondition({ operator: 'in', attribute: 'category', value: ['Electronics', 'Clothing'] }); ``` If the selection lives in a group with another key, name that group with `excludeGroup`: ```typescript list.catalog.query.addGroup({ groupKey: 'category_filter', groupType: 'or', queries: [ { operator: 'eq', attribute: 'category', value: 'Electronics' }, { operator: 'eq', attribute: 'category', value: 'Clothing' }, ], }); const categoryFacets = list.catalog.fn.facets({ attribute: 'category', excludeGroup: 'category_filter', }); ``` Without `excludeGroup`, the facet would count the selection against itself and list only the selected values (with `selected: false`). ### Deselecting a value To untick one value, remove its condition; the other selections stay. To reset one facet, remove its group: ```typescript // Untick "Clothing" (a condition added with addCondition) list.catalog.query.removeCondition({ operator: 'eq', attribute: 'category', value: 'Clothing' }); // Reset the whole category facet; filters on other attributes stay list.catalog.query.removeGroup('category'); ``` `removeCondition()` finds conditions in the group keyed by their attribute, which is where `addCondition()` and `setCondition()` put them. For a group with its own key, remove the condition from that group with `list.catalog.query.remove({ groupKey: 'category_filter', queries: [condition] })`, or remove the group with `removeGroup('category_filter')`. `clear()` removes every filter, including the other facets' selections. ### Combining filters (AND logic) Filters across different attributes combine with AND logic. ```typescript // Category filter list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Electronics' }); // Brand filter (OR within) - ANDed with category list.catalog.query.addCondition({ operator: 'eq', attribute: 'brand', value: 'Apple' }); list.catalog.query.addCondition({ operator: 'eq', attribute: 'brand', value: 'Samsung' }); // Results: Electronics items that are Apple OR Samsung (p1, p2, p3) ``` ## Complete flow ```typescript const list = dcupl.lists.create({ modelKey: 'Product' }); // 1. Get initial facets let categories = list.catalog.fn.facets({ attribute: 'category' }); let brands = list.catalog.fn.facets({ attribute: 'brand' }); // 2. User selects "Electronics" list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Electronics' }); // 3. Refresh facets - brand counts now reflect only Electronics, // categories still lists every category with "Electronics" selected categories = list.catalog.fn.facets({ attribute: 'category' }); brands = list.catalog.fn.facets({ attribute: 'brand' }); // 4. Get filtered results const results = list.catalog.query.items(); // 5. User unticks "Electronics" again list.catalog.query.removeCondition({ operator: 'eq', attribute: 'category', value: 'Electronics' }); // 6. Clear all filters list.catalog.query.clear(); ``` ## Next steps - [Facets Function](/sdk/fn/facets) — Complete facets() API reference - [Query Filter](/sdk/queries/query-filter) — Filter operators and syntax - [Query logic](/sdk/queries/query-logic) — AND/OR grouping details - [Product Catalog Tutorial](/learn/tutorials/product-catalog) — A faceted catalog end to end --- # Master-detail Source: https://docs.dcupl.com/sdk/advanced/master-detail Description: Look up the selected item by key and load related data with query conditions # Master-detail Display a list of items and show details for the selected item using dcupl queries. ## Key dcupl functions | Function | Purpose | | ------------------------------------ | ----------------------------------------------------------- | | `query.items()` | Get list of items for the master view | | `query.one({ itemKey })` | Get the selected item by key | | `query.one({ itemKey, projection })` | Get the selected item with fields of its referenced items | | `query.setCondition()` | Filter a related list (replaces that attribute's condition) | ## Example data The examples use products with a category, and reviews that reference a product: ```typescript title="Setup" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); dcupl.models.set({ key: 'Category', properties: [{ key: 'name', type: 'string' }], data: [{ key: 'cat-1', name: 'Electronics' }], }); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ], references: [{ key: 'category', type: 'singleValued', model: 'Category' }], data: [ { key: 'prod-123', name: 'Laptop', price: 999, category: 'cat-1' }, { key: 'prod-456', name: 'Phone', price: 599, category: 'cat-1' }, ], }); dcupl.models.set({ key: 'Review', properties: [ { key: 'rating', type: 'int' }, { key: 'text', type: 'string' }, ], references: [{ key: 'product', type: 'singleValued', model: 'Product' }], data: [ { key: 'r1', rating: 5, text: 'Fast', product: 'prod-123' }, { key: 'r2', rating: 4, text: 'Good camera', product: 'prod-456' }, { key: 'r3', rating: 3, text: 'Loud fan', product: 'prod-123' }, ], }); await dcupl.init(); ``` ## Getting the list Use `items()` to retrieve all items for the master list. ```typescript const list = dcupl.lists.create({ modelKey: 'Product' }); // Get all products for the list view const products = list.catalog.query.items(); ``` ## Selecting an item When user selects an item, look it up by its key with `one()`. This does not change the list's query, so the master view keeps its filters. The selection is your own state: to clear it or select another item, change that state and look up the new key. ```typescript function selectItem(itemKey: string) { // null if the list's query filters the item out (start/count paging is ignored) return list.catalog.query.one({ itemKey }); } const selected = selectItem('prod-999'); if (!selected) { // Not found: show an empty detail view } ``` Check the result with `if (!selected)` (or `=== null`): `one()` returns `null`, typed `T | null`, for a missing key or an item the list's query filters out. To read an item regardless of the list's filters, use the global lookup: `dcupl.query.one({ modelKey: 'Product', itemKey })` (returns `null` if the key does not exist). A reference comes back as `{ key }`. To show fields of referenced items in the detail view, pass a [projection](/sdk/api/types#projection): ```typescript const product = list.catalog.query.one({ itemKey: 'prod-123', projection: { $: true, category: { $: true } }, }); // { key: 'prod-123', name: 'Laptop', price: 999, category: { key: 'cat-1', name: 'Electronics' } } ``` ## Loading related data Load associated data for the selected item using a separate list. A condition on a reference matches the referenced item's key: ```typescript // Create the related list once and reuse it const reviewList = dcupl.lists.create({ modelKey: 'Review' }); function getReviews(productKey: string) { // setCondition replaces the previous condition on 'product' reviewList.catalog.query.setCondition({ attribute: 'product', operator: 'eq', value: productKey, }); return reviewList.catalog.query.items(); } getReviews('prod-123'); // r1, r3 ``` The same works with a plain string field that holds the key (for example `productId`), but a [reference](/sdk/api/models#reference) is indexed, can be projected, and lets dcupl check that the key exists. ## Complete flow ```typescript // 1. Create list and get all items const list = dcupl.lists.create({ modelKey: 'Product' }); const allProducts = list.catalog.query.items(); // 2. User clicks on product "prod-123": the detail with its category const selectedProduct = list.catalog.query.one({ itemKey: 'prod-123', projection: { $: true, category: { $: true } }, }); // 3. Load related data const reviewList = dcupl.lists.create({ modelKey: 'Review' }); reviewList.catalog.query.setCondition({ attribute: 'product', operator: 'eq', value: 'prod-123', }); const reviews = reviewList.catalog.query.items(); // r1, r3 // 4. User selects different product - look up the new key and refresh the reviews const newSelection = list.catalog.query.one({ itemKey: 'prod-456' }); reviewList.catalog.query.setCondition({ attribute: 'product', operator: 'eq', value: 'prod-456', }); const newReviews = reviewList.catalog.query.items(); // r2 ``` ## Next steps - [Query Filter](/sdk/queries/query-filter) — Filter operators and conditions - [Query results](/sdk/queries/query-results) — Retrieving items from queries - [Models API](/sdk/api/models#reference) — References and their types - [Projection](/sdk/api/types#projection) — Choosing the fields of returned items - [Core concepts](/sdk/core-concepts) — Core dcupl concepts and architecture --- # Pagination & infinite scroll Source: https://docs.dcupl.com/sdk/advanced/infinite-scroll Description: Use items() with start and count for pagination # Pagination & infinite scroll Load data in pages by passing `start` and `count` options to `items()`. ## Key dcupl functions | Function | Purpose | | -------------------------------------- | ------------------------------------------------------------- | | `query.items({ start, count })` | Get a slice of items | | `fn.metadata().currentSize` | Total number of matches (for calculating pages) | | `query.applyOptions({ start, count })` | Store the page in the list's query | | `query.count()` | Number of items `items()` returns (the page, once one is set) | ## Basic pagination Pass `start` and `count` to `items()` to load a specific page: ```typescript // Page 1: items 0-19 const page1 = productList.catalog.query.items({ start: 0, count: 20 }); // Page 2: items 20-39 const page2 = productList.catalog.query.items({ start: 20, count: 20 }); // Page 3: items 40-59 const page3 = productList.catalog.query.items({ start: 40, count: 20 }); ``` ## Calculate total pages Use `fn.metadata().currentSize` to determine how many pages exist. It counts every item that matches the current query, even when a page is stored with `applyOptions()` (then `query.count()` returns the length of that page): ```typescript const PAGE_SIZE = 20; const totalItems = productList.catalog.fn.metadata().currentSize; const totalPages = Math.ceil(totalItems / PAGE_SIZE); ``` ## Load more (infinite scroll) Accumulate items as user scrolls: ```typescript import type { ListItem } from '@dcupl/common'; const PAGE_SIZE = 20; let currentPage = 0; let allItems: ListItem[] = []; function loadMore() { const start = currentPage * PAGE_SIZE; const newItems = productList.catalog.query.items({ start, count: PAGE_SIZE }); allItems = [...allItems, ...newItems]; currentPage++; // More items exist while fewer than all matches are loaded return allItems.length < productList.catalog.fn.metadata().currentSize; } // Filters or data changed: start over, or the loaded pages mix old and new results productList.onResultChange(() => { currentPage = 0; allItems = []; loadMore(); }); ``` Comparing with `currentSize` stops exactly at the end; checking `newItems.length === PAGE_SIZE` would request one more, empty page when the total is a multiple of the page size. Reads with `start` and `count` don't change the list's query, so they don't trigger `onResultChange`. ## Pagination with sorting Combine pagination with sort options. Sort order values must be uppercase (`'ASC'` / `'DESC'`); with a lowercase value the whole sort is ignored and a warning is logged: ```typescript type Product = ListItem & { name: string; price: number }; const sortedPage = productList.catalog.query.items({ start: 0, count: 20, sort: { attributes: ['price'], order: ['ASC'], }, }); ``` Sorting doesn't change the result type: `items()` returns the list's item type, `Product[]` on a list created with `lists.create()` and `any[]` on an untyped list. Pass the type to the read (`items(…)`) to get typed items from an untyped list. ## Reset pagination after filter When filters change, reset to first page: ```typescript // Apply new filter productList.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); // Get filtered first page const results = productList.catalog.query.items({ start: 0, count: 20 }); ``` ## Keep the page in the list query Instead of passing `start` and `count` on every call, store them with `applyOptions()`. `items()` then returns the current page, and `list.onResultChange()` notifies you whenever the page, the filters or the data change. A filter change keeps `start`, so reset it yourself: ```typescript const PAGE_SIZE = 20; // Store the first page (other options such as sort are kept) productList.catalog.query.applyOptions({ start: 0, count: PAGE_SIZE }); // Re-render whenever the result changes const unsubscribe = productList.onResultChange(() => { renderProducts(productList.catalog.query.items()); }); // Next page productList.catalog.query.applyOptions({ start: PAGE_SIZE }); // New filter: go back to the first page first, then filter productList.catalog.query.applyOptions({ start: 0 }); productList.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); ``` ## Complete flow A paged product list with next/previous buttons, kept in the list's query: ```typescript import { Dcupl } from '@dcupl/core'; import type { ListItem } from '@dcupl/common'; type Product = ListItem & { name: string; category: string; price: number }; function renderProducts(items: Product[], page: number, pageCount: number) { console.log( `Page ${page + 1} of ${pageCount}:`, items.map((p) => p.name), ); } const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'category', type: 'string' }, { key: 'price', type: 'float' }, ], }); dcupl.data.set( Array.from({ length: 45 }, (_, i) => ({ key: `p${i}`, name: `Product ${i}`, category: i % 3 === 0 ? 'Books' : 'Electronics', price: i * 10, })), { model: 'Product' }, ); await dcupl.init(); const PAGE_SIZE = 20; const productList = dcupl.lists.create({ modelKey: 'Product' }); let page = 0; const pageCount = () => Math.ceil(productList.catalog.fn.metadata().currentSize / PAGE_SIZE); // 1. Re-render whenever the page, the filters or the data change productList.onResultChange(() => { renderProducts(productList.catalog.query.items(), page, pageCount()); }); // 2. Store the first page in the list's query (renders page 1 of 3) productList.catalog.query.applyOptions({ start: 0, count: PAGE_SIZE }); // 3. Next / previous page function goToPage(next: number) { const target = Math.max(0, Math.min(next, pageCount() - 1)); if (target === page) return; page = target; productList.catalog.query.applyOptions({ start: page * PAGE_SIZE }); } const nextPage = () => goToPage(page + 1); const prevPage = () => goToPage(page - 1); // 4. A new filter starts at the first page again function filterByCategory(category: string) { page = 0; // skipProcessing: store the option without recomputing; setCondition() recomputes once productList.catalog.query.applyOptions({ start: 0 }, { skipProcessing: true }); productList.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: category }); } nextPage(); // Page 2 of 3 filterByCategory('Books'); // Page 1 of 1 (15 books) ``` ## Next steps - [Query results](/sdk/queries/query-results) — items() and count() reference - [Query Filter](/sdk/queries/query-filter) — Combine pagination with filters - [Faceted search](/sdk/advanced/faceted-search) — Add facets to paginated results --- # Search & autocomplete Source: https://docs.dcupl.com/sdk/advanced/search-autocomplete Description: Use suggest() for autocomplete and query conditions for search # Search & autocomplete Build search with autocomplete using `suggest()` for suggestions and `setCondition()` for filtering results. ## Key dcupl functions | Function | Purpose | | ---------------------- | ---------------------------------------------------------------- | | `catalog.fn.suggest()` | Get autocomplete suggestions for an attribute | | `query.setCondition()` | Set filter condition (replaces the conditions on that attribute) | | `query.addCondition()` | Add filter condition (ORed with others on the same attribute) | ## Get suggestions Use `suggest()` to get autocomplete suggestions as the user types: ```typescript const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: 'lap', // User input max: 5, // Limit results }); // [{ value: 'Laptop Pro', keys: ['p1'] }, { value: 'Laptop Air', keys: ['p2'] }, { value: 'Laptop Stand', keys: ['p8'] }] ``` Each suggestion contains: - `value` - The suggested term - `keys` - Item keys that match this value A plain search term matches the start of the value, ignoring case: `'lap'` suggests "Laptop Pro" but not "Gaming Laptop". ## Match anywhere in the value Wrap the term in slashes for a regular expression and set `regexFlags: 'i'` to ignore case (`'/lap/i'` would be compared as a plain value and match nothing): ```typescript const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: '/lap/', regexFlags: 'i', max: 5, }); // Also suggests "Gaming Laptop" ``` Escape regex characters in user input before wrapping it in slashes. ## Suggest from multiple attributes Get suggestions from different attributes and combine them: ```typescript const nameSuggestions = productList.catalog.fn.suggest({ attribute: 'name', value: userInput, max: 5, }); const brandSuggestions = productList.catalog.fn.suggest({ attribute: 'brand', value: userInput, max: 3, }); // One dropdown: label each entry with the attribute it came from const combined = [ ...nameSuggestions.map((s) => ({ ...s, source: 'name' })), ...brandSuggestions.map((s) => ({ ...s, source: 'brand' })), ]; // [{ value: 'Laptop Pro', keys: ['p1'], source: 'name' }, ..., { value: 'Lenovo', keys: ['p4', 'p9'], source: 'brand' }] ``` The `source` tells you which condition to set when the user picks an entry. ## Handling input `max` defaults to `20`. In a search box, wait for a minimum length (for example 2 characters) and debounce the input (for example 150 ms) before you call `suggest()`. Each distinct term is cached on the list until the next update that affects it (see [Caching](/sdk/advanced/caching#cache-configuration)), so a debounce also keeps that cache small. ```typescript let timer: ReturnType | undefined; function onSearchInput(userInput: string) { clearTimeout(timer); if (userInput.length < 2) return; timer = setTimeout(() => { const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: userInput, max: 5 }); // render suggestions }, 150); } ``` ## Apply search filter When user selects a suggestion or submits search, filter the results. Wrap the term in slashes (`'/term/'`) for substring search; a bare string must match the whole value. Set `regexFlags: 'i'` for case-insensitive search: ```typescript // Text search with 'find' operator: names containing "laptop" in any case productList.catalog.query.setCondition({ operator: 'find', attribute: 'name', value: '/laptop/', options: { regexFlags: 'i' }, }); const results = productList.catalog.query.items(); ``` ## Filter by exact match When user selects a specific suggestion (e.g., a brand): ```typescript productList.catalog.query.setCondition({ operator: 'eq', attribute: 'brand', value: 'Apple', }); const results = productList.catalog.query.items(); ``` ## Clear and re-apply `setCondition()` replaces the previous condition on the same attribute, so a new search term needs no clearing. Call `clear()` only to drop all other filters as well (it also resets sort and paging): ```typescript // Optional: remove all conditions and options productList.catalog.query.clear(); // Apply new search (escape regex characters in user input) const escaped = newSearchTerm.replace(/[.*+?^${}()|[\]\\]/g, '\\$&'); productList.catalog.query.setCondition({ operator: 'find', attribute: 'name', value: `/${escaped}/`, options: { regexFlags: 'i' }, }); ``` ## Complete flow ```typescript // 1. User types "lap" const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: 'lap', max: 5, }); // 2. User selects "Laptop Pro" from suggestions (a bare string matches the whole value) productList.catalog.query.setCondition({ operator: 'find', attribute: 'name', value: 'Laptop Pro', }); // 3. Get filtered results const results = productList.catalog.query.items(); ``` `suggest()` draws from the list's current result by default (`relevantData: 'filtered'`), so once the search condition on `name` is set, the next `suggest()` on `name` only returns "Laptop Pro". To keep suggesting every name while a search is active, leave the search's group out: ```typescript const suggestions = productList.catalog.fn.suggest({ attribute: 'name', value: userInput, max: 5, relevantData: 'excludeQuery', excludeQuery: { groupKey: 'name' }, // the group setCondition() created for 'name' }); ``` ## Next steps - [suggest()](/sdk/fn/suggest) — Suggest function reference - [Query Filter](/sdk/queries/query-filter) — Filter operators - [Faceted search](/sdk/advanced/faceted-search) — Combine search with facets --- # Real-time updates Source: https://docs.dcupl.com/sdk/advanced/realtime-updates Description: Use list.onResultChange() and data.upsert() to handle live data changes # Real-time updates React to data changes with `list.onResultChange()` and push changes into dcupl with `data.upsert()`. ## Key dcupl functions | Function | Purpose | | ----------------------- | ---------------------------------------------------------------------------- | | `list.onResultChange()` | Run a callback when the list's result changes (returns unsubscribe function) | | `dcupl.data.upsert()` | Add new items or update existing ones (staged) | | `dcupl.data.remove()` | Remove data items (staged) | | `dcupl.update()` | Apply staged data changes; affected lists emit `result_updated` | ## Example data The snippets up to the Complete Flow use this `Product` model: ```typescript title="Setup" import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'stock', type: 'int' }, ], }); dcupl.data.set( [ { key: 'prod-1', name: 'Widget', stock: 50 }, { key: 'prod-2', name: 'Gadget', stock: 100 }, { key: 'prod-3', name: 'Gizmo', stock: 5 }, ], { model: 'Product' }, ); await dcupl.init(); ``` ## Listening to list events Subscribe to result changes on a list with `list.onResultChange()`. It fires once per `dcupl.update()` that can have changed the list (also for an update with nothing staged), and when the list's query changes — never for reads such as `items()`: ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); const unsubscribe = productList.onResultChange(() => { // Refresh your UI here console.log('Products:', productList.catalog.query.items()); }); // Later: cleanup unsubscribe(); ``` `list.on()` receives every event of the list, including reads; filter on `event.type === 'result_updated'` if you need it. See the [Events reference](/sdk/api/events). A list created with a `query` option keeps the keys that query selected: after `dcupl.update()` it serves their current values and removed items drop out, but a new match doesn't join it and an item that stops matching stays. Call `list.update()` to re-run the query once, or give the list `autoUpdate: true` (or set `performance.autoUpdateLists.enabled`) to re-run it on each `dcupl.update()`; the [Auto-Update Pattern](/sdk/api/lists#auto-update-pattern) shows how to pass it in TypeScript. ## Updating data with data.upsert() Push new or updated items into dcupl. `upsert()` inserts unknown keys and updates existing items; every attribute in the payload replaces the stored value, attributes you leave out keep theirs. Don't use `data.set()` for this: it replaces **all** data of the model. ```typescript // Add a new item or update an existing one dcupl.data.upsert([{ key: 'prod-4', name: 'Doohickey', stock: 20 }], { model: 'Product' }); // Apply changes await dcupl.update(); ``` Update multiple items at once: ```typescript dcupl.data.upsert( [ { key: 'prod-1', stock: 45 }, { key: 'prod-2', stock: 120 }, { key: 'prod-3', stock: 0 }, ], { model: 'Product' }, ); await dcupl.update(); ``` ## Removing data Remove items by key: ```typescript dcupl.data.remove([{ key: 'prod-3' }], { model: 'Product' }); await dcupl.update(); ``` ## Complete flow Combine event listening with data updates: ```typescript import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl(); dcupl.models.set({ key: 'Order', properties: [{ key: 'status', type: 'string' }], }); dcupl.data.set([], { model: 'Order' }); // Start empty await dcupl.init(); const orderList = dcupl.lists.create({ modelKey: 'Order' }); // 1. Listen for changes orderList.onResultChange(() => { const orders = orderList.catalog.query.items(); console.log('Orders updated:', orders.length); }); // 2. Add data (from your data source) async function handleNewOrder(order: { key: string; status: string }) { dcupl.data.upsert([order], { model: 'Order' }); await dcupl.update(); // orderList emits result_updated } // 3. Update existing item async function updateOrderStatus(orderId: string, status: string) { dcupl.data.update([{ key: orderId, status }], { model: 'Order' }); await dcupl.update(); } // 4. Remove item async function deleteOrder(orderId: string) { dcupl.data.remove([{ key: orderId }], { model: 'Order' }); await dcupl.update(); } ``` `data.update()` only changes items that already exist; an unknown key is ignored. Use `upsert()` when the item may be new. ## Batching updates For multiple changes, call `dcupl.update()` once at the end: ```typescript // Queue multiple changes dcupl.data.upsert([{ key: 'prod-1', stock: 10 }], { model: 'Product' }); dcupl.data.upsert([{ key: 'prod-2', stock: 20 }], { model: 'Product' }); dcupl.data.remove([{ key: 'prod-3' }], { model: 'Product' }); // A single update: each affected list emits result_updated once await dcupl.update(); ``` ## Next steps - [Lists](/sdk/core-concepts/lists) — Creating and managing lists - [Data management](/sdk/core-concepts/data-management) — Full data operations reference - [Query results](/sdk/queries/query-results) — Reading data from lists --- # TypeScript usage guide Source: https://docs.dcupl.com/sdk/guides/typescript Description: Type dcupl results, queries and events in TypeScript - the exported types, the methods that take a generic, and typed patterns # TypeScript usage guide This guide shows how to type your dcupl code: which types the SDK exports, which methods take your item type as a generic, what an untyped list returns, and patterns for typed queries, facets and aggregations. ## Installation The dcupl SDK includes built-in TypeScript definitions: ```bash group="install" tab="npm" npm install @dcupl/core @dcupl/common ``` ```bash group="install" tab="yarn" yarn add @dcupl/core @dcupl/common ``` ```bash group="install" tab="pnpm" pnpm add @dcupl/core @dcupl/common ``` `@dcupl/core` depends on exactly its own version of `@dcupl/common`. Install both in the same version, or your type imports come from a different version of `@dcupl/common` than the one the SDK uses. The type definitions use `NoInfer`, so they need TypeScript 5.4 or later. ```json filename="tsconfig.json" { "compilerOptions": { "target": "ES2020", "module": "ESNext", "lib": ["ES2020", "DOM"], "moduleResolution": "bundler", "strict": true, "esModuleInterop": true, "skipLibCheck": true, "forceConsistentCasingInFileNames": true, "resolveJsonModule": true, "declaration": true, "declarationMap": true } } ``` ## Key types `Dcupl` and `DcuplList` come from `@dcupl/core`, `DcuplAppLoader` from `@dcupl/loader`. The types come from `@dcupl/common`: | Type | What it types | | --------------------------------------- | ------------------------------------------------------------- | | `ModelDefinition`, `Property` | Models and their properties | | `ListItem` | A generic item: `{ key: string; [attribute: string]: any }` | | `DcuplQuery`, `DcuplQueryGroup` | A query condition and a group of conditions | | `DcuplQueryOperator` | The operator names: `'eq'`, `'in'`, `'find'`, `'gt'`, … | | `DcuplListQueryOptions` | The options of `items()`, `first()` and `count()` | | `SortDirection` | `'ASC'` or `'DESC'` | | `DcuplFacet` | One facet value: `{ key, value, size, selected, enabled, … }` | | `Aggregation`, `AggregationType` | The result of `fn.aggregate()` and its `types` | | `DcuplUpdateMessage`, `DcuplUpdateType` | An event and the union of its `type` | | `DcuplInitOptions` | The options of `new Dcupl()` | These methods take your item type as a generic parameter: `dcupl.lists.create()` (it types the list's items), `items()`, `execute()`, `first()` and `count()` on `list.catalog.query`, and `one()`, `many()` and `execute()` on `dcupl.query`. See [Types](/sdk/api/types) for the full reference. ### Untyped lists return `any` `dcupl.lists.create()` returns a `DcuplList` whose item type defaults to `any`. Without a generic, `items()` returns `any[]` and `first()` returns `any`, so TypeScript doesn't flag a misspelled attribute. The untyped call next to the ways to type the results: - `list.catalog.query.items()` returns `any[]`: a misspelled attribute isn't flagged. - `list.catalog.query.items()` returns `Product[]`: a misspelled attribute is an error. - `dcupl.lists.create(...)` returns a `DcuplList`, the same type as the annotation `const list: DcuplList = dcupl.lists.create(...)`. On that list `items()` returns `(Product & ListItem)[]` and `first()` `(Product & ListItem) | undefined`: a misspelled attribute is `any` and isn't flagged. `items()` is the stricter one: on a `DcuplList`, the known attributes are typed, but `ListItem`'s index signature makes every other attribute `any`. Generics and casts only tell TypeScript what shape to expect. Nothing checks the items at runtime: for that, use a type guard ([Custom Type Guards](#custom-type-guards)) or the model's quality checks. ## Basic type usage ### Importing types ```typescript import type { ModelDefinition, Property, ListItem, DcuplQuery, DcuplQueryOperator } from '@dcupl/common'; import { Dcupl, DcuplList } from '@dcupl/core'; ``` `@dcupl/core` exports the `Dcupl` and `DcuplList` classes; the types (models, queries, facets, events) come from `@dcupl/common`. A few helper types aren't exported, see [Types](/sdk/api/types). ### Type-safe model definition ```typescript import type { ModelDefinition } from '@dcupl/common'; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string' }, { key: 'inStock', type: 'boolean' }, { key: 'tags', type: 'Array' }, ], }; // TypeScript will catch errors const invalidModel: ModelDefinition = { key: 'Invalid', properties: [ // Error: 'type' is required { key: 'name' }, // Error: invalid type { key: 'price', type: 'invalid' }, ], }; ``` ## Custom type definitions ### Creating item interfaces Define TypeScript interfaces that match your models. Every dcupl item has a string `key`. `@dcupl/common` types a generic item as `ListItem` (`{ key: string; [attribute: string]: any }`); its index signature makes every attribute `any`, so declare your own interfaces to get checked attribute names: ```typescript // Every dcupl item has a string key interface DcuplItem { key: string; } interface Product extends DcuplItem { name: string; sku: string; price: number; category: string; brand: string; description: string; inStock: boolean; rating: number; tags: string[]; createdAt: Date; updatedAt: Date; } interface Category extends DcuplItem { name: string; slug: string; description: string; parentId: string | null; } interface Customer extends DcuplItem { email: string; firstName: string; lastName: string; phone: string; address: Address; } interface Address { street: string; city: string; state: string; zipCode: string; country: string; } ``` ### Complex nested types ```typescript interface Order extends DcuplItem { orderId: string; customerId: string; items: OrderItem[]; total: number; status: OrderStatus; shippingAddress: Address; billingAddress: Address; createdAt: Date; updatedAt: Date; } interface OrderItem { productId: string; quantity: number; price: number; discount?: number; } type OrderStatus = 'pending' | 'processing' | 'shipped' | 'delivered' | 'cancelled'; ``` ## Generic type usage ### DcuplList with generics `items()`, `first()` and `execute()` take the item type as a generic parameter, or you pass it once to `dcupl.lists.create()` ([Untyped Lists Return `any`](#untyped-lists-return-any)): ```typescript import { Dcupl, DcuplList } from '@dcupl/core'; const dcupl = new Dcupl(); // Setup: register the Product model and its data (dcupl.models.set / dcupl.data.set), // then initialize — lists.create() throws for a model that isn't registered await dcupl.init(); // Create a list const productList: DcuplList = dcupl.lists.create({ modelKey: 'Product' }); // The generic parameter types the results (Product is the interface defined above) const products = productList.catalog.query.items(); // TypeScript knows the shape products.forEach((product) => { console.log(product.name); // ✓ TypeScript knows 'name' exists console.log(product.price); // ✓ TypeScript knows 'price' is a number console.log(product.invalid); // ✗ Error: Property doesn't exist }); ``` ### Type-safe query functions Create reusable type-safe query functions: ```typescript function queryProducts( list: DcuplList, filters?: { category?: string; minPrice?: number; maxPrice?: number; inStock?: boolean; }, ): Product[] { // Reset query list.catalog.query.clear(); // Apply filters if (filters?.category) { list.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: filters.category, }); } // Conditions on the same attribute combine with OR, // so a price range needs its own 'and' group const priceRange: DcuplQuery[] = []; if (filters?.minPrice !== undefined) { priceRange.push({ operator: 'gte', attribute: 'price', value: filters.minPrice }); } if (filters?.maxPrice !== undefined) { priceRange.push({ operator: 'lte', attribute: 'price', value: filters.maxPrice }); } if (priceRange.length > 0) { list.catalog.query.addGroup({ groupKey: 'price', groupType: 'and', queries: priceRange }); } if (filters?.inStock !== undefined) { list.catalog.query.addCondition({ operator: 'eq', attribute: 'inStock', value: filters.inStock, }); } // Return typed results return list.catalog.query.items(); } // Usage with type safety const electronics = queryProducts(productList, { category: 'Electronics' }); const affordable = queryProducts(productList, { maxPrice: 100 }); const available = queryProducts(productList, { inStock: true }); ``` ### Generic repository pattern ```typescript class DcuplRepository { private list: DcuplList; constructor( private dcupl: Dcupl, private modelKey: string, ) { this.list = dcupl.lists.create({ modelKey }); } findAll(): T[] { this.list.catalog.query.clear(); return this.list.catalog.query.items() as T[]; } findById(id: string): T | undefined { this.list.catalog.query.clear(); // one() looks the item up by key in the list's current result; // it returns null when the key isn't there: ?? turns that into undefined return (this.list.catalog.query.one({ itemKey: id }) ?? undefined) as T | undefined; } findByAttribute(attribute: keyof T, value: any): T[] { this.list.catalog.query.clear(); this.list.catalog.query.addCondition({ operator: 'eq', attribute: attribute as string, value, }); return this.list.catalog.query.items() as T[]; } findWhere(predicate: (item: T) => boolean): T[] { const allItems = this.findAll(); return allItems.filter(predicate); } count(): number { return this.list.catalog.query.count(); } } // Usage const productRepo = new DcuplRepository(dcupl, 'Product'); const allProducts = productRepo.findAll(); // Type: Product[] const product = productRepo.findById('p1'); // Type: Product | undefined const electronics = productRepo.findByAttribute('category', 'Electronics'); // Type: Product[] const expensive = productRepo.findWhere((p) => p.price > 1000); // Type: Product[] ``` ## Typing results `execute()` is an alias of `items()`: both return the type you pass as the generic parameter: ```typescript // Generic parameter const products0 = productList.catalog.query.items(); // Explicit typing const products1: Product[] = productList.catalog.query.items() as Product[]; // Type assertion const products2 = productList.catalog.query.items() as Product[]; // With generic function function getProducts(list: DcuplList): T[] { return list.catalog.query.items(); } const products3 = getProducts(productList); // Type: Product[] ``` ## Type-safe queries ### Query builder with types ```typescript import type { DcuplQuery, DcuplQueryOperator } from '@dcupl/common'; // 'eq' | 'in' | 'find' | 'gt' | 'gte' | 'lt' | 'lte' | 'typeof' | 'isTruthy' | 'size' type QueryOperator = DcuplQueryOperator; interface TypedQuery { attribute: keyof T; operator: QueryOperator; value: any; } class TypedQueryBuilder { private queries: TypedQuery[] = []; constructor(private list: DcuplList) {} where(attribute: keyof T, operator: QueryOperator, value: any): this { this.queries.push({ attribute, operator, value }); return this; } execute(): T[] { this.list.catalog.query.clear(); // One 'and' group: every where() must match, also several on the same attribute // (addCondition would OR conditions that share an attribute) const queries: DcuplQuery[] = this.queries.map((query) => ({ operator: query.operator, attribute: query.attribute as string, value: query.value, })); this.list.catalog.query.addGroup({ groupKey: 'typed', groupType: 'and', queries }); return this.list.catalog.query.items(); } reset(): this { this.queries = []; this.list.catalog.query.clear(); return this; } } // Usage with type safety const builder = new TypedQueryBuilder(productList); const results = builder .where('category', 'eq', 'Electronics') // ✓ 'category' is valid .where('price', 'gte', 100) // ✓ 'price' is valid .where('inStock', 'eq', true) // ✓ 'inStock' is valid .where('invalid', 'eq', 'value') // ✗ Error: 'invalid' doesn't exist .execute(); ``` ### Strongly-typed filter helpers ```typescript type FilterValue = T[K]; interface FilterConfig { eq(attribute: K, value: FilterValue): void; find(attribute: K, value: string): void; gt(attribute: K, value: FilterValue): void; gte(attribute: K, value: FilterValue): void; lt(attribute: K, value: FilterValue): void; lte(attribute: K, value: FilterValue): void; isTruthy(attribute: K, value: boolean): void; } function createFilter(list: DcuplList): FilterConfig { // Conditions on different attributes combine with AND; a second condition // on the same attribute is ORed with the first const applyFilter = (operator: QueryOperator, attribute: string, value: any) => { list.catalog.query.addCondition({ operator, attribute, value }); }; return { eq: (attribute, value) => applyFilter('eq', attribute as string, value), find: (attribute, value) => applyFilter('find', attribute as string, value), gt: (attribute, value) => applyFilter('gt', attribute as string, value), gte: (attribute, value) => applyFilter('gte', attribute as string, value), lt: (attribute, value) => applyFilter('lt', attribute as string, value), lte: (attribute, value) => applyFilter('lte', attribute as string, value), isTruthy: (attribute, value) => applyFilter('isTruthy', attribute as string, value), }; } // Usage const filter = createFilter(productList); filter.eq('category', 'Electronics'); // ✓ Type-safe filter.gte('price', 100); // ✓ Type-safe filter.eq('inStock', true); // ✓ Type-safe filter.find('name', '/Laptop/'); // ✓ Type-safe text search: '/…/' matches a substring, a bare string the whole value filter.eq('invalid', 'value'); // ✗ Error: Property doesn't exist filter.eq('price', 'not-a-number'); // ✗ Error: Wrong type ``` ## Custom type guards Create type guards for runtime type checking: ```typescript function isProduct(item: ListItem): item is Product { return ( 'name' in item && 'price' in item && 'category' in item && typeof item.name === 'string' && typeof item.price === 'number' && typeof item.category === 'string' ); } function isCategory(item: ListItem): item is Category { return 'name' in item && 'slug' in item && typeof item.name === 'string' && typeof item.slug === 'string'; } // Usage const items = productList.catalog.query.items(); items.forEach((item) => { if (isProduct(item)) { // TypeScript knows item is Product console.log(item.price); } else if (isCategory(item)) { // TypeScript knows item is Category console.log(item.slug); } }); ``` ## Advanced typing patterns ### Mapped types for queries ```typescript type QueryableFields = { [K in keyof T]: T[K] extends string | number | boolean ? K : never; }[keyof T]; type QueryableProduct = QueryableFields; // Type: 'key' | 'name' | 'sku' | 'price' | 'category' | 'brand' | 'description' | 'inStock' | 'rating' function queryByField>(list: DcuplList, field: K, value: T[K]): T[] { list.catalog.query.clear(); list.catalog.query.addCondition({ operator: 'eq', attribute: field as string, value, }); return list.catalog.query.items(); } // Usage const byName = queryByField(productList, 'name', 'Laptop'); const byPrice = queryByField(productList, 'price', 999); ``` ### Conditional types for filters ```typescript type FilterOperatorForType = T extends string ? 'eq' | 'in' | 'find' | 'typeof' | 'isTruthy' | 'size' : T extends number ? 'eq' | 'in' | 'gt' | 'gte' | 'lt' | 'lte' | 'typeof' : T extends boolean ? 'eq' | 'isTruthy' : 'eq' | 'in' | 'typeof' | 'isTruthy'; interface TypedFilter { attribute: K; operator: FilterOperatorForType; value: T[K] | T[K][]; } function applyTypedFilter(list: DcuplList, filter: TypedFilter): T[] { // Cast needed: the generic attribute/operator pair isn't narrowed to one DcuplQuery variant list.catalog.query.addCondition({ operator: filter.operator, attribute: filter.attribute as string, value: filter.value, } as DcuplQuery); return list.catalog.query.items(); } // Usage - TypeScript enforces valid operators per type const stringFilter: TypedFilter = { attribute: 'name', operator: 'find', // ✓ Valid for string value: 'Laptop', }; const numberFilter: TypedFilter = { attribute: 'price', operator: 'gte', // ✓ Valid for number value: 100, }; const invalidFilter: TypedFilter = { attribute: 'price', operator: 'find', // ✗ Error: 'find' not valid for number value: 100, }; ``` ### Utility types for facets ```typescript type FacetableFields = { [K in keyof T]: T[K] extends string | number | boolean ? K : never; }[keyof T]; interface FacetResult { attribute: K; values: Array<{ value: T[K]; size: number; // number of items with this value enabled: boolean; }>; } function getFacets>( list: DcuplList, attribute: K, ): FacetResult { // DcuplFacet[]: { key, value, size, selected, enabled, ... } const facetValues = list.catalog.fn.facets({ attribute: attribute as string, }); return { attribute, values: facetValues.map((facet) => ({ value: facet.value as T[K], size: facet.size, enabled: facet.enabled, })), }; } // Usage const categoryFacets = getFacets(productList, 'category'); const brandFacets = getFacets(productList, 'brand'); const priceFacets = getFacets(productList, 'price'); ``` ## Type-safe aggregations ```typescript type NumericFields = { [K in keyof T]: T[K] extends number ? K : never; }[keyof T]; type AggregationType = 'sum' | 'avg' | 'min' | 'max' | 'count'; // a subset of AggregationType from @dcupl/common interface AggregationResult { attribute: K; aggregation: AggregationType; value: number; } function aggregate>( list: DcuplList, attribute: K, aggregation: AggregationType, ): AggregationResult { // Returns { attribute, types, avg?, sum?, ... } with one field per requested type const result = list.catalog.fn.aggregate({ attribute: attribute as string, types: [aggregation], }); return { attribute, aggregation, value: result[aggregation] ?? 0, }; } // Usage const avgPrice = aggregate(productList, 'price', 'avg'); const ratingSum = aggregate(productList, 'rating', 'sum'); // Error: Can't aggregate non-numeric field const invalidAgg = aggregate(productList, 'name', 'sum'); // ✗ Error ``` ## Strict null checks Handle nullable values properly. An attribute the source row has no value for is absent from the item (`undefined`); an explicit `null` in the source stays `null`: ```typescript interface ProductWithOptionals extends DcuplItem { name: string; price: number; description?: string; // Optional field discount?: number | null; // Missing (undefined) or explicitly null tags?: string[]; } function getProductDescription(product: ProductWithOptionals): string { // Nullish coalescing return product.description ?? 'No description available'; } function getDiscountedPrice(product: ProductWithOptionals): number { // == null catches both null and a missing value if (product.discount == null) { return product.price; } return product.price * (1 - product.discount); } function getFirstTag(product: ProductWithOptionals): string | undefined { // Array access with optional chaining return product.tags?.[0]; } ``` ## Typed sorting Sort orders are uppercase; `SortDirection` from `@dcupl/common` types them: ```typescript import type { SortDirection } from '@dcupl/common'; type SortAttribute = 'name' | 'price' | 'rating' | 'createdAt'; interface SortOptions { attribute: SortAttribute; order: SortDirection; // 'ASC' | 'DESC' } function sortProducts(list: DcuplList, options: SortOptions): Product[] { return list.catalog.query.items({ sort: { attributes: [options.attribute], order: [options.order], }, }); } // Usage const sorted = sortProducts(productList, { attribute: 'price', // ✓ Valid order: 'ASC', // ✓ Valid }); const invalid = sortProducts(productList, { attribute: 'invalid', // ✗ Error: Invalid attribute order: 'ascending', // ✗ Error: Invalid order }); ``` ## Type-safe event handling `dcupl.on()`, `list.on()` and `list.onResultChange()` pass a `DcuplUpdateMessage` and return an unsubscribe function. `message.type` is the `DcuplUpdateType` union, so TypeScript rejects event names that don't exist: ```typescript import type { DcuplUpdateMessage } from '@dcupl/common'; // Fires whenever the list's result changes (query change or dcupl.update()) const unsubscribe = productList.onResultChange((message: DcuplUpdateMessage) => { console.log(message.type, message.listKey); // 'result_updated', '' renderProducts(productList.catalog.query.items()); }); // Global events dcupl.on((message) => { if (message.type === 'dcupl_updated_manually') { console.log('dcupl.update() finished'); } if (message.type === 'data_updated') { // ✗ Error: 'data_updated' is not a DcuplUpdateType } }); // Later unsubscribe(); ``` ## Testing with types ```typescript import { describe, it, expect, beforeEach } from 'vitest'; import { Dcupl, DcuplList } from '@dcupl/core'; describe('Product Queries', () => { let dcupl: Dcupl; let productList: DcuplList; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set(productModel); dcupl.data.set( [ { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics', inStock: true, tags: ['work'] }, { key: 'p2', name: 'Desk', price: 299, category: 'Furniture', inStock: false, tags: [] }, ], { model: 'Product' }, ); await dcupl.init(); productList = dcupl.lists.create({ modelKey: 'Product' }); }); it('should filter products by category', () => { productList.catalog.query.addCondition({ operator: 'eq', attribute: 'category', value: 'Electronics', }); const results = productList.catalog.query.items(); expect(results).toBeInstanceOf(Array); expect(results.length).toBeGreaterThan(0); expect(results.every((p) => p.category === 'Electronics')).toBe(true); }); it('should type-check product properties', () => { const products = productList.catalog.query.items(); const firstProduct = products[0]; // TypeScript ensures these properties exist expect(typeof firstProduct.name).toBe('string'); expect(typeof firstProduct.price).toBe('number'); expect(typeof firstProduct.inStock).toBe('boolean'); }); }); ``` ## Best practices ### 1. Always define interfaces ```typescript // ✓ Good: Clear interface definition, passed as the generic interface Product extends DcuplItem { name: string; price: number; } const products = productList.catalog.query.items(); // ✗ Bad: Untyped results are any[] const untyped = productList.catalog.query.items(); ``` The generic, like a cast, is a promise about the shape that nothing checks at runtime. ### 2. Use type guards ```typescript // ✓ Good: Type guard for runtime safety function isValidProduct(item: any): item is Product { return ( typeof item === 'object' && 'name' in item && 'price' in item && typeof item.name === 'string' && typeof item.price === 'number' ); } const items = productList.catalog.query.items(); const products = items.filter(isValidProduct); // Type: Product[], checked at runtime // ✗ Bad: Unchecked type assertion const unchecked = items as Product[]; ``` ### 3. Leverage type inference ```typescript // ✓ Good: Let TypeScript infer when possible const products = getProducts(productList); // ✗ Bad: Redundant type annotations const annotated: Product[] = getProducts(productList); ``` ### 4. Use generics for reusability ```typescript // ✓ Good: Generic reusable function function findById(list: DcuplList, id: string): T | undefined { // one() returns null for a key that isn't in the list's result: ?? turns that into undefined return (list.catalog.query.one({ itemKey: id }) ?? undefined) as T | undefined; } // ✗ Bad: Specific implementations function findProductById(list: DcuplList, id: string): Product | undefined { // ... } function findCategoryById(list: DcuplList, id: string): Category | undefined { // ... } ``` ## Next steps - [React Integration](/learn/tutorials/react-integration) — Use typed dcupl with React - [Vue Integration](/learn/tutorials/vue-integration) — Use typed dcupl with Vue - [Getting started](/learn/getting-started) — Getting started with dcupl - [API reference](/sdk/api/types) — Complete type reference - [Faceted search](/sdk/advanced/faceted-search) — Facets, counts and filters for search UIs --- # How to test dcupl applications Source: https://docs.dcupl.com/sdk/guides/testing Description: Unit testing, mocking, and integration testing patterns for dcupl SDK # How to test dcupl applications Write reliable tests for applications that use the dcupl SDK, covering unit tests for queries, component tests with mocks, and integration tests. ## Prerequisites - Working dcupl application - Vitest installed. The examples import from `vitest` and use `vi`; with Jest, use `jest.fn()` for `vi.fn()` and Jest's globals instead of the imports - Basic understanding of [lists](/sdk/core-concepts/lists) and [queries](/sdk/queries/query-filter) ## Unit testing queries Test query logic in isolation using real dcupl instances with inline test data. ### Basic query test ```typescript filename="product-queries.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; describe('Product Queries', () => { let dcupl: Dcupl; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string', filter: true }, { key: 'inStock', type: 'boolean' }, ], data: [ { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics', inStock: true }, { key: 'p2', name: 'Mouse', price: 29, category: 'Electronics', inStock: true }, { key: 'p3', name: 'Desk', price: 299, category: 'Furniture', inStock: false }, { key: 'p4', name: 'Chair', price: 199, category: 'Furniture', inStock: true }, ], }; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set(productModel); await dcupl.init(); }); afterEach(() => { dcupl.destroy(); }); it('should return all products', () => { const list = dcupl.lists.create({ modelKey: 'Product' }); const items = list.catalog.query.items(); expect(items).toHaveLength(4); }); it('should filter by category', () => { const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.apply({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const items = list.catalog.query.items(); expect(items).toHaveLength(2); expect(items.every((item) => item.category === 'Electronics')).toBe(true); }); it('should filter by price range', () => { const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.apply({ groupKey: 'priceRange', groupType: 'and', queries: [ { attribute: 'price', operator: 'gte', value: 100 }, { attribute: 'price', operator: 'lte', value: 500 }, ], }); const items = list.catalog.query.items(); expect(items).toHaveLength(2); expect(items.map((i) => i.name)).toContain('Desk'); expect(items.map((i) => i.name)).toContain('Chair'); }); }); ``` ### Testing filter combinations ```typescript filename="filter-combinations.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; describe('Filter Combinations', () => { let dcupl: Dcupl; let list: ReturnType; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'category', type: 'string', filter: true }, { key: 'brand', type: 'string', filter: true }, { key: 'price', type: 'float' }, ], data: [ { key: 'p1', name: 'iPhone', category: 'Phones', brand: 'Apple', price: 999 }, { key: 'p2', name: 'Galaxy', category: 'Phones', brand: 'Samsung', price: 899 }, { key: 'p3', name: 'MacBook', category: 'Laptops', brand: 'Apple', price: 1299 }, { key: 'p4', name: 'ThinkPad', category: 'Laptops', brand: 'Lenovo', price: 999 }, ], }); await dcupl.init(); list = dcupl.lists.create({ modelKey: 'Product' }); }); afterEach(() => { dcupl.destroy(); }); it('should combine category and brand filters (AND)', () => { // Filter: Apple products in Phones category // Each condition goes into the group keyed by its attribute; different attributes combine with AND list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Phones', }); list.catalog.query.addCondition({ attribute: 'brand', operator: 'eq', value: 'Apple', }); const items = list.catalog.query.items(); expect(items).toHaveLength(1); expect(items[0].name).toBe('iPhone'); }); it('should support OR within a filter group', () => { // Filter: Apple OR Samsung products list.catalog.query.apply({ groupKey: 'brandFilter', groupType: 'or', queries: [ { attribute: 'brand', operator: 'eq', value: 'Apple' }, { attribute: 'brand', operator: 'eq', value: 'Samsung' }, ], }); const items = list.catalog.query.items(); expect(items).toHaveLength(3); }); it('should clear filters correctly', () => { list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Phones', }); expect(list.catalog.query.items()).toHaveLength(2); list.catalog.query.clear(); expect(list.catalog.query.items()).toHaveLength(4); }); it('should remove specific filter group', () => { list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Phones', }); list.catalog.query.addCondition({ attribute: 'brand', operator: 'eq', value: 'Apple', }); expect(list.catalog.query.items()).toHaveLength(1); // Remove only the category filter (its group is keyed by the attribute name) list.catalog.query.removeGroup('category'); // Should now show all Apple products const items = list.catalog.query.items(); expect(items).toHaveLength(2); expect(items.every((i) => i.brand === 'Apple')).toBe(true); }); }); ``` ### Testing sorting and pagination ```typescript filename="sorting-pagination.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; describe('Sorting and Pagination', () => { let dcupl: Dcupl; let list: ReturnType; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'rating', type: 'float' }, ], data: [ { key: 'p1', name: 'Product A', price: 100, rating: 4.5 }, { key: 'p2', name: 'Product B', price: 50, rating: 4.8 }, { key: 'p3', name: 'Product C', price: 150, rating: 4.2 }, { key: 'p4', name: 'Product D', price: 75, rating: 4.6 }, ], }); await dcupl.init(); list = dcupl.lists.create({ modelKey: 'Product' }); }); afterEach(() => { dcupl.destroy(); }); it('should sort by price ascending', () => { const items = list.catalog.query.items({ sort: { attributes: ['price'], order: ['ASC'] }, }); expect(items[0].price).toBe(50); expect(items[3].price).toBe(150); }); it('should sort by price descending', () => { const items = list.catalog.query.items({ sort: { attributes: ['price'], order: ['DESC'] }, }); expect(items[0].price).toBe(150); expect(items[3].price).toBe(50); }); it('should paginate results', () => { const page1 = list.catalog.query.items({ start: 0, count: 2 }); const page2 = list.catalog.query.items({ start: 2, count: 2 }); expect(page1).toHaveLength(2); expect(page2).toHaveLength(2); expect(page1[0].key).not.toBe(page2[0].key); }); it('should combine sorting and pagination', () => { const items = list.catalog.query.items({ sort: { attributes: ['price'], order: ['ASC'] }, start: 1, count: 2, }); expect(items).toHaveLength(2); expect(items[0].price).toBe(75); // Second cheapest expect(items[1].price).toBe(100); // Third cheapest }); }); ``` ## Mocking dcupl for component tests When testing UI components, mock dcupl to avoid initializing the full SDK. ### Creating a mock dcupl instance The mock needs every method your component calls. This one covers the common query, function and data methods; add the ones your components use. ```typescript filename="test-helpers/mock-dcupl.ts" import { vi, type Mock } from 'vitest'; export interface MockDcuplList { catalog: { query: { items: Mock; first: Mock; count: Mock; addCondition: Mock; setCondition: Mock; removeCondition: Mock; addGroup: Mock; removeGroup: Mock; apply: Mock; clear: Mock; remove: Mock; }; fn: { facets: Mock; aggregate: Mock; metadata: Mock; }; filters: { get: Mock; getAll: Mock; }; }; on: Mock; onResultChange: Mock; destroy: Mock; } export interface MockDcupl { models: { set: Mock; get: Mock; }; data: { set: Mock; update: Mock; upsert: Mock; remove: Mock; }; lists: { create: Mock; get: Mock; getAll: Mock; }; init: Mock; update: Mock; destroy: Mock; } export function createMockList(items: T[] = []): MockDcuplList { return { catalog: { query: { items: vi.fn().mockReturnValue(items), first: vi.fn().mockReturnValue(items[0]), count: vi.fn().mockReturnValue(items.length), addCondition: vi.fn(), setCondition: vi.fn(), removeCondition: vi.fn(), addGroup: vi.fn(), removeGroup: vi.fn(), apply: vi.fn(), clear: vi.fn(), remove: vi.fn(), }, fn: { facets: vi.fn().mockReturnValue([]), aggregate: vi.fn().mockReturnValue({ attribute: 'price', types: ['sum'], sum: 0 }), metadata: vi.fn().mockReturnValue({ key: 'TestModel', currentSize: items.length, initialSize: items.length, appliedQuery: { queries: [] }, }), }, filters: { get: vi.fn().mockReturnValue(undefined), getAll: vi.fn().mockReturnValue([]), }, }, on: vi.fn().mockReturnValue(() => {}), // returns an unsubscribe function, like the real one onResultChange: vi.fn().mockReturnValue(() => {}), destroy: vi.fn(), }; } export function createMockDcupl(lists: Map = new Map()): MockDcupl { return { models: { set: vi.fn(), get: vi.fn(), }, data: { set: vi.fn(), update: vi.fn(), upsert: vi.fn(), remove: vi.fn(), }, lists: { create: vi.fn().mockImplementation(({ modelKey }) => { return lists.get(modelKey) || createMockList(); }), get: vi.fn().mockImplementation((key) => lists.get(key)), getAll: vi.fn().mockReturnValue(Array.from(lists.values())), }, init: vi.fn().mockResolvedValue(true), // the real init() resolves true, or false on failure update: vi.fn().mockResolvedValue(true), destroy: vi.fn(), }; } ``` ### Testing a React component with mock The component under test gets the instance as a prop, creates its list, and filters by category: ```tsx filename="ProductList.tsx" import { useEffect, useState } from 'react'; import type { Dcupl, DcuplList } from '@dcupl/core'; type Product = { key: string; name: string; category: string }; export function ProductList({ dcupl }: { dcupl: Dcupl }) { const [list, setList] = useState(null); const [items, setItems] = useState([]); useEffect(() => { const productList = dcupl.lists.create({ modelKey: 'Product' }); setList(productList); setItems(productList.catalog.query.items()); return () => productList.destroy(); }, [dcupl]); const filterByCategory = (category: string) => { list?.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: category }); setItems(list?.catalog.query.items() ?? []); }; const clearFilters = () => { list?.catalog.query.clear(); setItems(list?.catalog.query.items() ?? []); }; return (
    {items.map((product) => (
  • {product.name}
  • ))}
); } ``` The test passes the mock in place of a `Dcupl` instance. The mock has only part of the `Dcupl` type, so cast it with `as unknown as Dcupl`. `toBeInTheDocument()` comes from `@testing-library/jest-dom`: import `@testing-library/jest-dom/vitest` in your Vitest setup file and run the tests with `environment: 'jsdom'`; for `tsc` to know the matcher, list the setup file in your `tsconfig.json` `include`. ```tsx filename="ProductList.spec.tsx" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { render, screen, fireEvent, cleanup } from '@testing-library/react'; import type { Dcupl } from '@dcupl/core'; import { ProductList } from './ProductList'; import { createMockDcupl, createMockList } from './test-helpers/mock-dcupl'; describe('ProductList Component', () => { const mockProducts = [ { key: 'p1', name: 'Laptop', price: 999, category: 'Electronics' }, { key: 'p2', name: 'Mouse', price: 29, category: 'Electronics' }, { key: 'p3', name: 'Desk', price: 299, category: 'Furniture' }, ]; let mockList: ReturnType; let mockDcupl: ReturnType; beforeEach(() => { mockList = createMockList(mockProducts); mockDcupl = createMockDcupl(new Map([['Product', mockList]])); }); afterEach(() => { cleanup(); // unmount; Testing Library does this on its own only with Vitest's globals enabled }); it('should render product list', () => { render(); expect(screen.getByText('Laptop')).toBeInTheDocument(); expect(screen.getByText('Mouse')).toBeInTheDocument(); expect(screen.getByText('Desk')).toBeInTheDocument(); }); it('should set the category condition when filtering', () => { render(); const filterButton = screen.getByRole('button', { name: /electronics/i }); fireEvent.click(filterButton); expect(mockList.catalog.query.setCondition).toHaveBeenCalledWith( expect.objectContaining({ attribute: 'category', operator: 'eq', value: 'Electronics', }), ); }); it('should call query.clear when clearing filters', () => { render(); const clearButton = screen.getByRole('button', { name: /clear/i }); fireEvent.click(clearButton); expect(mockList.catalog.query.clear).toHaveBeenCalled(); }); }); ``` ### Testing an Angular component with mock Here the instance comes from a service of your own, which the test replaces: ```typescript filename="services/dcupl.service.ts" import { Injectable } from '@angular/core'; import { Dcupl } from '@dcupl/core'; @Injectable({ providedIn: 'root' }) export class DcuplService { private readonly dcupl = new Dcupl(); getDcupl(): Dcupl { return this.dcupl; } } ``` ```typescript filename="product-list.component.ts" import { Component, inject } from '@angular/core'; import { DcuplService } from './services/dcupl.service'; type Product = { key: string; name: string }; @Component({ selector: 'app-product-list', template: ` @for (product of products; track product.key) {
{{ product.name }}
} `, }) export class ProductListComponent { private readonly list = inject(DcuplService).getDcupl().lists.create({ modelKey: 'Product' }); products = this.list.catalog.query.items(); filterByCategory(category: string) { this.list.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: category }); this.products = this.list.catalog.query.items(); } } ``` ```typescript filename="product-list.component.spec.ts" import { describe, it, expect, vi, beforeEach } from 'vitest'; import { ComponentFixture, TestBed } from '@angular/core/testing'; import { ProductListComponent } from './product-list.component'; import { DcuplService } from './services/dcupl.service'; import { createMockDcupl, createMockList, type MockDcuplList } from './test-helpers/mock-dcupl'; describe('ProductListComponent', () => { let component: ProductListComponent; let fixture: ComponentFixture; let mockList: MockDcuplList; const mockProducts = [ { key: 'p1', name: 'Laptop', price: 999 }, { key: 'p2', name: 'Mouse', price: 29 }, ]; beforeEach(async () => { mockList = createMockList(mockProducts); const mockDcupl = createMockDcupl(new Map([['Product', mockList]])); await TestBed.configureTestingModule({ imports: [ProductListComponent], providers: [{ provide: DcuplService, useValue: { getDcupl: vi.fn().mockReturnValue(mockDcupl) } }], }).compileComponents(); fixture = TestBed.createComponent(ProductListComponent); component = fixture.componentInstance; fixture.detectChanges(); }); it('should create', () => { expect(component).toBeTruthy(); }); it('should display products', () => { const productElements = fixture.nativeElement.querySelectorAll('.product-item'); expect(productElements.length).toBe(2); }); it('should filter products when category selected', () => { component.filterByCategory('Electronics'); expect(mockList.catalog.query.setCondition).toHaveBeenCalledWith( expect.objectContaining({ attribute: 'category', value: 'Electronics' }), ); }); }); ``` ## Integration testing Test complete workflows with real dcupl instances. ### Full workflow test ```typescript filename="product-catalog.integration.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; describe('Product Catalog Integration', () => { let dcupl: Dcupl; const categoryModel: ModelDefinition = { key: 'Category', properties: [{ key: 'name', type: 'string' }], data: [ { key: 'cat1', name: 'Electronics' }, { key: 'cat2', name: 'Furniture' }, { key: 'cat3', name: 'Clothing' }, ], }; const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'inStock', type: 'boolean' }, ], references: [{ key: 'category', type: 'singleValued', model: 'Category', filter: true }], data: [ { key: 'p1', name: 'Laptop', price: 999, category: 'cat1', inStock: true }, { key: 'p2', name: 'Mouse', price: 29, category: 'cat1', inStock: true }, { key: 'p3', name: 'Desk', price: 299, category: 'cat2', inStock: false }, { key: 'p4', name: 'Chair', price: 199, category: 'cat2', inStock: true }, { key: 'p5', name: 'T-Shirt', price: 25, category: 'cat3', inStock: true }, ], }; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set(categoryModel); dcupl.models.set(productModel); await dcupl.init(); }); afterEach(() => { dcupl.destroy(); }); it('should complete a full filter workflow', () => { const list = dcupl.lists.create({ modelKey: 'Product' }); // Step 1: Get initial facets const categoryFacets = list.catalog.fn.facets({ attribute: 'category' }); expect(categoryFacets).toHaveLength(3); // Step 2: Apply category filter (a reference matches by key) list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'cat1', }); const filteredItems = list.catalog.query.items(); expect(filteredItems).toHaveLength(2); // Step 3: Verify facets update // The facet leaves out the group of its own attribute, so all 3 categories keep their counts const updatedFacets = list.catalog.fn.facets({ attribute: 'category' }); const electronicsFacet = updatedFacets.find((f) => f.key === 'cat1'); expect(updatedFacets).toHaveLength(3); expect(electronicsFacet?.size).toBe(2); expect(electronicsFacet?.selected).toBe(true); // Step 4: Add price filter list.catalog.query.addCondition({ attribute: 'price', operator: 'lte', value: 100, }); const priceFilteredItems = list.catalog.query.items(); expect(priceFilteredItems).toHaveLength(1); expect(priceFilteredItems[0].name).toBe('Mouse'); // Step 5: Clear all filters list.catalog.query.clear(); expect(list.catalog.query.count()).toBe(5); }); it('should handle data updates correctly', async () => { const list = dcupl.lists.create({ modelKey: 'Product' }); // Initial count expect(list.catalog.query.count()).toBe(5); // Add new product (upsert adds unknown keys; set() would replace all Product items) dcupl.data.upsert([{ key: 'p6', name: 'Monitor', price: 399, category: 'cat1', inStock: true }], { model: 'Product', }); // Data calls are staged until update(); the list sees the new item on its next read await dcupl.update(); expect(list.catalog.query.count()).toBe(6); // Verify the new product is queryable list.catalog.query.apply({ attribute: 'name', operator: 'find', value: 'Monitor', }); const items = list.catalog.query.items(); expect(items).toHaveLength(1); expect(items[0].price).toBe(399); }); it('should create filtered lists', () => { // A list scoped to electronics: it keeps the keys the query selects now, with their current items const query = dcupl.query.generate('Product', { attribute: 'category.key', operator: 'eq', value: 'cat1', }); const electronicsList = dcupl.lists.create({ modelKey: 'Product', query }); const meta = electronicsList.catalog.fn.metadata(); expect(meta.initialSize).toBe(2); expect(meta.currentSize).toBe(2); // Additional filters work on the pre-filtered set electronicsList.catalog.query.apply({ attribute: 'price', operator: 'lt', value: 100, }); expect(electronicsList.catalog.query.count()).toBe(1); }); }); ``` ### Testing with aggregations ```typescript filename="aggregations.integration.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; describe('Aggregations', () => { let dcupl: Dcupl; beforeEach(async () => { dcupl = new Dcupl(); dcupl.models.set({ key: 'Order', properties: [ { key: 'product', type: 'string' }, { key: 'quantity', type: 'int' }, { key: 'price', type: 'float' }, { key: 'status', type: 'string', filter: true }, ], data: [ { key: 'o1', product: 'Laptop', quantity: 2, price: 999, status: 'completed' }, { key: 'o2', product: 'Mouse', quantity: 5, price: 29, status: 'completed' }, { key: 'o3', product: 'Keyboard', quantity: 3, price: 79, status: 'pending' }, { key: 'o4', product: 'Monitor', quantity: 1, price: 399, status: 'completed' }, ], }); await dcupl.init(); }); afterEach(() => { dcupl.destroy(); }); it('should calculate sum', () => { const list = dcupl.lists.create({ modelKey: 'Order' }); // aggregate() always needs `types` and returns one object with a field per type const result = list.catalog.fn.aggregate({ attribute: 'quantity', types: ['sum'], }); expect(result.sum).toBe(11); }); it('should calculate average', () => { const list = dcupl.lists.create({ modelKey: 'Order' }); const result = list.catalog.fn.aggregate({ attribute: 'price', types: ['avg'], }); expect(result.avg).toBe(376.5); }); it('should calculate min and max', () => { const list = dcupl.lists.create({ modelKey: 'Order' }); const result = list.catalog.fn.aggregate({ attribute: 'price', types: ['min', 'max'], }); expect(result.min).toBe(29); expect(result.max).toBe(999); }); it('should aggregate filtered results', () => { const list = dcupl.lists.create({ modelKey: 'Order' }); list.catalog.query.apply({ attribute: 'status', operator: 'eq', value: 'completed', }); const completed = list.catalog.fn.aggregate({ attribute: 'quantity', types: ['sum'], }); expect(completed.sum).toBe(8); // 2 + 5 + 1 }); }); ``` ## Test data strategies ### Using fixtures ```typescript filename="test-fixtures/products.ts" import type { ModelDefinition } from '@dcupl/common'; export const PRODUCT_MODEL: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string', filter: true }, { key: 'brand', type: 'string', filter: true }, { key: 'inStock', type: 'boolean' }, { key: 'rating', type: 'float' }, ], }; export const SAMPLE_PRODUCTS = [ { key: 'p1', name: 'Laptop Pro', price: 1299, category: 'Electronics', brand: 'TechCo', inStock: true, rating: 4.5, }, { key: 'p2', name: 'Wireless Mouse', price: 49, category: 'Electronics', brand: 'TechCo', inStock: true, rating: 4.2, }, { key: 'p3', name: 'Office Desk', price: 399, category: 'Furniture', brand: 'HomeStyle', inStock: false, rating: 4.0, }, { key: 'p4', name: 'Ergonomic Chair', price: 299, category: 'Furniture', brand: 'HomeStyle', inStock: true, rating: 4.8, }, { key: 'p5', name: 'USB-C Hub', price: 79, category: 'Electronics', brand: 'ConnectPro', inStock: true, rating: 4.3, }, ]; export function createProductModel(data = SAMPLE_PRODUCTS): ModelDefinition { return { ...PRODUCT_MODEL, data, }; } ``` ### Factory functions for test data ```typescript filename="test-helpers/factories.ts" export interface Product { key: string; name: string; price: number; category: string; brand: string; inStock: boolean; rating: number; } let idCounter = 0; export function createProduct(overrides: Partial = {}): Product { idCounter++; return { key: `product-${idCounter}`, name: `Test Product ${idCounter}`, price: 100 + idCounter * 10, category: 'General', brand: 'TestBrand', inStock: true, rating: 4.0, ...overrides, }; } export function createProducts(count: number, overrides: Partial = {}): Product[] { return Array.from({ length: count }, () => createProduct(overrides)); } export function createProductsInCategory(category: string, count: number): Product[] { return createProducts(count, { category }); } ``` Keep `describe` blocks in spec files: a helper module that contains one registers it again in every test file that imports the helper. ```typescript filename="product-volume.spec.ts" import { describe, it, expect } from 'vitest'; import { Dcupl } from '@dcupl/core'; import { createProducts } from './test-helpers/factories'; describe('Product Tests', () => { it('should handle many products', async () => { const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string', filter: true }, ], data: createProducts(1000), }); await dcupl.init(); const list = dcupl.lists.create({ modelKey: 'Product' }); expect(list.catalog.query.count()).toBe(1000); }); }); ``` ### Resetting state between tests ```typescript filename="test-helpers/test-dcupl.ts" import { Dcupl } from '@dcupl/core'; import type { ModelDefinition } from '@dcupl/common'; let dcuplInstance: Dcupl | null = null; export function getTestDcupl(): Dcupl { if (!dcuplInstance) { throw new Error('Dcupl not initialized. Call setupTestDcupl first.'); } return dcuplInstance; } export async function setupTestDcupl(models: ModelDefinition[]): Promise { dcuplInstance = new Dcupl(); models.forEach((model) => dcuplInstance!.models.set(model)); // init() resolves false instead of throwing, so fail the test setup explicitly if (!(await dcuplInstance.init())) { throw new Error('dcupl init failed'); } return dcuplInstance; } export function cleanupTestDcupl(): void { if (dcuplInstance) { dcuplInstance.destroy(); dcuplInstance = null; } } ``` Use the helpers in a spec file (here with the model from the fixtures above): ```typescript filename="my-tests.spec.ts" import { describe, it, expect, beforeEach, afterEach } from 'vitest'; import { setupTestDcupl, cleanupTestDcupl, getTestDcupl } from './test-helpers/test-dcupl'; import { createProductModel } from './test-fixtures/products'; describe('My Tests', () => { beforeEach(async () => { await setupTestDcupl([createProductModel()]); }); afterEach(() => { cleanupTestDcupl(); }); it('should work', () => { const dcupl = getTestDcupl(); const list = dcupl.lists.create({ modelKey: 'Product' }); expect(list.catalog.query.count()).toBe(5); }); }); ``` ## Testing data quality `init()` resolves `true` even when the data has quality errors: a value that doesn't fit its type (`WrongDataType`) or a reference to a key that doesn't exist (`RemoteReferenceKeyNotFound`) doesn't fail it. A test that only checks `init()` misses broken data, so assert on `dcupl.quality.errors.summary()` too: ```typescript filename="product-data-quality.spec.ts" import { describe, it, expect, afterEach } from 'vitest'; import { Dcupl } from '@dcupl/core'; import { createProductModel } from './test-fixtures/products'; describe('Product data quality', () => { let dcupl: Dcupl; afterEach(() => { dcupl.destroy(); }); it('loads the fixtures without quality errors', async () => { dcupl = new Dcupl(); // quality checks are on by default dcupl.models.set(createProductModel()); expect(await dcupl.init()).toBe(true); expect(dcupl.quality.errors.summary()).toEqual([]); }); it('reports a wrong type and a broken reference', async () => { dcupl = new Dcupl(); dcupl.models.set({ key: 'Category', properties: [{ key: 'name', type: 'string' }], data: [{ key: 'cat1', name: 'Electronics' }], }); dcupl.models.set({ key: 'Product', properties: [{ key: 'price', type: 'float' }], references: [{ key: 'category', type: 'singleValued', model: 'Category' }], data: [ { key: 'p1', price: 'abc', category: 'cat1' }, { key: 'p2', price: 10, category: 'cat9' }, ], }); // init() still resolves true expect(await dcupl.init()).toBe(true); const errorTypes = dcupl.quality.errors.summary().map((group) => group.errorType); expect(errorTypes).toEqual(expect.arrayContaining(['WrongDataType', 'RemoteReferenceKeyNotFound'])); }); }); ``` Each summary entry carries `model`, `attribute`, `errorType` and `totalCount`, so a test can also pin the exact errors it expects. See [Quality & validation](/sdk/advanced/quality-validation) for the error types. ## Best practices ### Test Structure **Arrange-Act-Assert pattern:** ```typescript it('should filter by category', () => { // Arrange const list = dcupl.lists.create({ modelKey: 'Product' }); // Act list.catalog.query.apply({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const items = list.catalog.query.items(); // Assert expect(items).toHaveLength(2); expect(items.every((i) => i.category === 'Electronics')).toBe(true); }); ``` ### Isolation **Each test should be independent:** ```typescript import { describe, beforeEach, afterEach } from 'vitest'; import { Dcupl, type DcuplList } from '@dcupl/core'; import { createProductModel } from './test-fixtures/products'; describe('Filter Tests', () => { let dcupl: Dcupl; let list: DcuplList; beforeEach(async () => { // Fresh instance for each test dcupl = new Dcupl(); dcupl.models.set(createProductModel()); await dcupl.init(); list = dcupl.lists.create({ modelKey: 'Product' }); }); afterEach(() => { // Clean up list.destroy(); dcupl.destroy(); }); // Tests are now isolated }); ``` ### Assertions **Be specific in assertions:** ```typescript // Bad - too vague expect(items.length).toBeGreaterThan(0); // Good - specific expectation expect(items).toHaveLength(3); // Good - verify actual content expect(items.map((i) => i.key)).toEqual(['p1', 'p2', 'p3']); // Good - verify properties expect(items[0]).toMatchObject({ name: 'Laptop', category: 'Electronics', }); ``` ## Next steps - [Debugging with Console](/sdk/guides/debugging) — Inspect queries at runtime - [TypeScript Guide](/sdk/guides/typescript) — Type-safe testing patterns - [React Integration](/learn/tutorials/react-integration) — Using dcupl in a React app --- # Debugging with Console Source: https://docs.dcupl.com/sdk/guides/debugging Description: Connect your running app to Console for live inspection # Debugging with Console Connect your running app to [dcupl Console](/console) for live inspection of lists, queries, and data flow. ## Prerequisites - A working dcupl application - Access to [dcupl Console](/console) ## Install @dcupl/connect ```bash group="install" tab="npm" npm install @dcupl/connect ``` ```bash group="install" tab="yarn" yarn add @dcupl/connect ``` ```bash group="install" tab="pnpm" pnpm add @dcupl/connect ``` `@dcupl/connect` depends on exactly its own version of `@dcupl/core` and `@dcupl/common`: install it in the same version as `@dcupl/core`. ## Connect your app Add the connect module to your dcupl instance: ```typescript filename="app.ts" import { Dcupl } from '@dcupl/core'; import { DcuplConnect } from '@dcupl/connect'; const dcupl = new Dcupl(); // ... your model and data setup await dcupl.init(); // Connect to Console. Don't await init() in your startup path (see below) const connect = new DcuplConnect({ dcuplInstance: dcupl }); connect.init().then((status) => console.log('dcupl Connect:', status.state)); // 'connected' | 'disconnected' ``` The connection is started from the Console: in your project, open **Applications** → **Connect**, enter your app's URL under **Application Url** and click **Connect Application**. The Console opens your app in a new browser window, and `connect.init()` connects to it (Connect only talks to the Console at `https://console.dcupl.com`). When the app is opened directly (it has no `window.opener`), `init()` resolves with `state: 'disconnected'` right away. When a page other than the Console's dcupl Connect page opened it (for example with `window.open()` or a link that keeps the opener), `init()` waits for an answer that never comes and its promise doesn't settle, so don't `await` it before your app starts. ## What you can inspect Once connected, the Console receives your app's models, data and events: - **Event stream** – every model added, data change and list created, updated or destroyed in your app, with its arguments and result. dcupl's internal `$Dcupl…` models are left out. - **Data Explorer, Models and Quality Analyzer** – work on the connected instance the way they do on an application you run in the Console. - **Show in Data Explorer** – opens any list your app created, with its current query. **Loader Insights** stays disabled: the Console didn't load the app from a `dcupl.lc.json`, so it has no loader run to show. ## Development only Disable the connect module in production. `import.meta.env.DEV` is Vite's flag; with other build tools, check `process.env.NODE_ENV !== 'production'`: ```typescript filename="app.ts" if (import.meta.env.DEV) { const connect = new DcuplConnect({ dcuplInstance: dcupl }); connect.init(); } ``` Call `connect.destroy()` when you tear the dcupl instance down, for example on hot module replacement, so Connect stops listening to your app and stops sending its events and URL changes to the Console. Calling it before a connection is safe. ## Next steps - [Console Overview](/console) — Learn more about Console features - [Setup options](/sdk/setup-options) — Loader setup basics - [Performance Tuning](/sdk/advanced/performance) — Optimize your queries --- # Error handling Source: https://docs.dcupl.com/sdk/guides/error-handling Description: Handle errors gracefully in dcupl applications - types, catching, recovery, and debugging # Error handling Learn how to handle errors gracefully in dcupl applications, from initialization failures to data quality issues. ## Error categories dcupl reports problems in three ways (`summary()` is `dcupl.quality.errors.summary()`): | Category | Description | Where you see it | | ----------------- | ------------------------------------------------- | --------------------------------------------------------------------------- | | **Model Errors** | Data quality and validation issues | `summary()` after `init()` or `update()` | | **Loader Errors** | Resources that failed to load or process | `summary()` with `type: 'loader'`, and `failedItems` in the loader progress | | **Thrown Errors** | Calls that can't be executed (e.g. unknown model) | An `Error` thrown by the call, see [Thrown Errors](#thrown-errors) | `init()` and `update()` don't throw: they resolve `false` when they fail, and the cause only goes to the console (see [When init() Resolves false](#when-init-resolves-false)). --- ## Model errors Model errors occur when data doesn't match the expected schema or validation rules. ### Error groups | Group | Description | | ---------------------- | ----------------------------------------------- | | `ReferenceDataError` | Reference points to missing data | | `PropertyDataError` | Property value doesn't match type or validation | | `DataContainerError` | Issues with data container processing | | `ModelDefinitionError` | Invalid model configuration | ### Error types Every error has an `errorType` within its group: `UndefinedValue`, `WrongDataType`, `RemoteReferenceKeyNotFound`, `MissingReference` and so on. [Quality & validation](/sdk/advanced/quality-validation#error-types) lists every model and loader error type and what triggers it. ### Handling model errors ```typescript import { Dcupl } from '@dcupl/core'; const dcupl = new Dcupl({ quality: { enabled: true }, // default }); // Setup models and data... await dcupl.init(); // Check for errors after init: one entry per (model, attribute, errorType) group const errors = dcupl.quality.errors.summary(); if (errors.length > 0) { // Count errors by type const countsByType = errors.reduce( (acc, group) => { acc[group.errorType] = (acc[group.errorType] ?? 0) + group.totalCount; return acc; }, {} as Record, ); console.error('Validation errors:', countsByType); // Handle specific error types if (countsByType.RemoteReferenceKeyNotFound) { console.warn('Some references are broken - data may be incomplete'); } if (countsByType.UndefinedValue) { // The individual records are in the built-in $DcuplErrorTrackingErrors model const missing = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'errorType', operator: 'eq', value: 'UndefinedValue' }], }); console.error( 'Required fields missing - blocking error:', missing.map((e) => `${e.itemKey}.${e.attribute}`), ); throw new Error('Data validation failed'); } } ``` ### Error structure `dcupl.quality.errors.summary()` returns one entry per group (type `QualityAnalyzer.ErrorGroupSummary` from `@dcupl/common`). `totalCount` keeps counting when `quality.maxErrorsPerGroup` (default `1000`) limits the stored records. Failed type coercions are the exception: they are consolidated into one `WrongDataType` record per attribute (affected rows in `meta.rowCount`), and when more values of one attribute fail than `maxErrorsPerGroup`, the counts are not exact (see [Limiting Stored Errors](/sdk/advanced/quality-validation#limiting-stored-errors)). ```typescript type ErrorGroupSummary = { key: string; // 'Model::attribute::ErrorType' model?: string; // Affected model key attribute?: string; // Affected attribute errorGroup: string; // e.g. 'PropertyDataError' errorType: string; // e.g. 'UndefinedValue', see Error Types type: 'model' | 'loader'; totalCount: number; // Number of occurrences, also beyond maxErrorsPerGroup storedCount: number; // Records kept in $DcuplErrorTrackingErrors truncated: boolean; // totalCount > storedCount }; ``` The `$DcuplErrorTrackingErrors` and `$DcuplErrorTrackingSummary` models are rebuilt by `init()` and, after a partial update, whenever an error record changed, so they and `dcupl.quality.errors.summary()` are current after `dcupl.update()`. A partial update leaves the same error records and `summary()` counts as a fresh `init()` on the same data: the changed rows are re-validated (`unique` included), the errors of removed rows are gone, and a value of the wrong type in `data.update()` is recorded as `WrongDataType` and dropped. The item then has no value for that attribute; its previous value is not kept. One exception: a keyless row in a `data.upsert()` that a partial update applies is currently dropped without a `MissingKey` record; `init()` and a full rebuild record it. Each record in the `$DcuplErrorTrackingErrors` model looks like this. `model`, `errorGroup` and `errorType` are references: filter them with a plain string (`value: 'UndefinedValue'`) and read them as `error.errorType.key`. ```typescript type ErrorRecord = { key: string; // Generated record key title: string; // Human-readable title description?: string; // Detailed description itemKey?: string; // Affected item key attribute?: string; // Affected attribute meta?: Record; groupTotal?: number; // Count of the record's group truncated?: boolean; // The group has more occurrences than stored records model?: { key: string }; // Affected model errorGroup: { key: string }; errorType: { key: string }; }; ``` See [Quality & validation](/sdk/advanced/quality-validation#getting-errors) for examples with real output. --- ## Loader errors Loader errors occur when fetching or processing configuration and data resources. ### Error groups | Group | Description | | ------------------- | --------------------------------------- | | `LoaderConfigError` | Group of `InvalidConfig` (not reported) | | `ResourceError` | Failed to fetch or process a resource | ### Error types The SDK records `InvalidResource`, `TransformerError` and `ScriptingDisabled`; `EndpointScriptMissing` and `EndpointDuplicate` come only from REST API instances. `InvalidConfig` is never recorded: a configuration that can't be fetched or parsed makes `loader.config.fetch()` throw. [Quality & validation](/sdk/advanced/quality-validation#error-types) explains each type. Loader errors show up in `dcupl.quality.errors.summary()` with `type: 'loader'`, also when quality checks are disabled. With quality enabled they are records in `$DcuplErrorTrackingErrors` too; the records have no `type` field, so filter them by `errorGroup` or `errorType`. ### Handling loader errors ```typescript import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; const dcupl = new Dcupl({ config: { projectId: 'my-project', apiKey: 'my-api-key' }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // Connect the loader before fetching its config // Track loading errors loader.on((progress) => { // Every update carries the whole run so far, so failedItems repeats earlier // failures. Report them once, when the run is finished. if (progress.remaining !== 0) return; for (const failure of progress.failedItems) { // status is the HTTP status; it is undefined for network errors (DNS, CORS, refused connection) const status = failure.status ? ` (HTTP ${failure.status})` : ''; console.error(`Failed to load: ${failure.key ?? failure.url}${status}`); } }); let configLoaded = false; try { await loader.config.fetch(); // Throws if the loader configuration can't be fetched configLoaded = true; } catch (error) { console.error('Failed to fetch config:', error); // Fallback to local config or show error UI } if (configLoaded) { // A resource that fails to load does not throw: it shows up in failedItems // and as an InvalidResource error with type 'loader' await loader.process(); // init() doesn't throw: it resolves false when initialization failed const initialized = await dcupl.init(); if (!initialized) { console.error('Initialization failed'); } const loaderErrors = dcupl.quality.errors.summary().filter((group) => group.type === 'loader'); if (loaderErrors.length > 0) { console.warn('Some resources could not be loaded:', loaderErrors); } } ``` ### Resource retry pattern ```typescript // The loader must already be added with dcupl.loaders.add(loader) async function loadWithRetry(loader: DcuplAppLoader, maxRetries = 3) { let lastError: Error | undefined; for (let attempt = 1; attempt <= maxRetries; attempt++) { try { await loader.config.fetch(); await loader.process(); return; // Success } catch (error) { lastError = error as Error; if (lastError.name === 'SharedModelKeyError') throw lastError; // fails the same way every time console.warn(`Attempt ${attempt}/${maxRetries} failed:`, error); if (attempt < maxRetries) { // Exponential backoff await new Promise((resolve) => setTimeout(resolve, 1000 * Math.pow(2, attempt))); } } } throw lastError; } ``` This retries when `config.fetch()` throws (the configuration couldn't be loaded) or `process()` throws (no configuration). It doesn't retry a `SharedModelKeyError` from `process()`, which fails the same way on every attempt ([Error Subclasses](#error-subclasses)). Single resources that fail to load don't throw; check `failedItems` in the progress callback or the `InvalidResource` errors in `dcupl.quality.errors.summary()`. --- ## Thrown errors Runtime calls such as queries, lists and functions return their results directly. When a call can't be executed, it throws an `Error`; read `error.message` (three are subclasses with their own `name`, see [Error Subclasses](#error-subclasses)). These are the messages to match on: | Message | Thrown by | | ------------------------------------------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ | | `Model with key X not found` | `dcupl.lists.create()` and `dcupl.query.execute()` for a model that isn't registered | | (empty message) | `dcupl.query.one()` and `dcupl.query.many()` for a model that isn't registered | | `invalid query - in operator needs an array value` | Any call that applies or runs an `in` condition whose value isn't an array. The list doesn't keep the condition: its query stays as it was | | `invalid query - unknown operator "X"; expected a built-in operator (…) or a registered custom operator (…)` | Any call that applies or runs a condition whose `operator` is neither built in nor registered (an [`UnknownOperator`](#error-subclasses) error). The list doesn't keep the condition | | `There is no config present. Make sure to call loader.config.set(...) or loader.config.fetch(...) first` | `loader.process()` before the loader has a configuration | | `dcupl not connected. Did you add your loader to dcupl? -> dcupl.loaders.add(loader)` | `loader.config.fetch()` before `dcupl.loaders.add(loader)` | | `Invalid Loader Configuration` | `loader.config.fetch()` when the loader configuration can't be fetched or isn't JSON | | `The applicationKey X does not exist` | `loader.process()` and `loader.resources.plan()` with an `applicationKey` the configuration doesn't define ([DcuplAppLoader](/sdk/api/loader)) | Query conditions are checked by `dcupl.query.execute()`, `dcupl.lists.create({ query })`, `dcupl.fn.*` with a `query`, a list's `addCondition()`, `setCondition()`, `apply()`, `addGroup()` and `setGroup()`, and a list read that gets conditions in its options (such as `items({ queries })`). The `dcupl.data.*` calls check their payload: | Message | Cause | | ------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | `invalid query - missing attribute or operator` | A condition with an `attribute` but no `operator` | | `invalid query - find operator cannot have array value` | A `find` condition whose value is an array | | `invalid query - missing query or attribute` | A condition without `attribute` inside a group (`apply()`, `addGroup()` or `setGroup()` with a group, `lists.create({ query })`, `dcupl.query.execute()`, `dcupl.fn.*` with a `query`, or a list read with `queries` in its options such as `items({ queries })`). Passed on its own to `addCondition()`, `setCondition()` or `apply()`, such a condition is skipped without an error | | `Invalid Data - Data must be an Array` | `dcupl.data.set()`, `update()`, `upsert()` and `remove()` with a payload that isn't an array, such as a single item | | `Invalid Data Container` | The same calls without a payload (`null` or `undefined`) or with options that have no `model` | ```typescript try { dcupl.lists.create({ modelKey: 'Nope' }); } catch (error) { console.error((error as Error).message); // Model with key Nope not found } ``` ### Error subclasses Three errors are subclasses of `Error` with their own `name` (a fourth, `DcuplSealedError`, is thrown by writes on a sealed instance, see [Performance](/sdk/advanced/performance)): - **`ScriptNotFound`** (`Script X not found`): `dcupl.scripts.execute()` throws it, and `executeAsync()` and `executeEndpoint()` reject with it, for a script key that isn't registered. Check `error.name === 'ScriptNotFound'`, not `instanceof`: `@dcupl/core` bundles its own copy of `@dcupl/common`, so the `ScriptNotFoundError` class you import is not the class core throws ([ScriptNotFoundError](/sdk/api/types#scriptnotfounderror)). With scripting disabled, the same calls throw (or reject with) a plain `Error`, `Scripting is disabled for this dcupl instance`. - **`UnknownOperator`** (`invalid query - unknown operator "X"; expected a built-in operator (…) or a registered custom operator (…)`): every call that checks query conditions (see above) throws it for an `operator` that is neither [built in](/sdk/api/query-operators) nor registered as an [operator script](/sdk/guides/scripts) or with `registerCustomOperator()`. The list doesn't keep the condition. Check `error.name === 'UnknownOperator'`, not `instanceof`, for the same reason as `ScriptNotFound`. - **`SharedModelKeyError`** (`Model "X" is already loaded by app loader "a"; app loader "b" cannot load model files or data for it. …`, or `View "X" …` for a view file): `loader.process()` rejects with it when another app loader of the same dcupl instance already loads that key, and the run changes nothing. It is exported from `@dcupl/loader`, so `error instanceof SharedModelKeyError` works, and it carries `modelKey`, `owner` and `loader`. Retrying doesn't help: give each model key to one loader ([process()](/sdk/api/loader#process)). ### Handling thrown errors ```typescript import type { DcuplList } from '@dcupl/core'; import type { DcuplListQueryOptions } from '@dcupl/common'; // dcupl methods return results directly, but you can wrap in try-catch function safeQuery(list: DcuplList, options: DcuplListQueryOptions): T[] | null { try { return list.catalog.query.items(options) as T[]; } catch (error) { console.error('Query failed:', error); return null; } } // Usage const products = safeQuery(productList, { start: 0, count: 10 }); if (products === null) { // Show error state } ``` --- ## When init() resolves false `init()` and `update()` catch their own errors. They resolve `false` and log the cause to the console as `[Dcupl] Initialization failed` or `[Dcupl] Update failed`, followed by the error. No API returns that error, and the core SDK's logger can't be redirected (see [Logging Configuration](#logging-configuration)), so in production the cause is only in the browser console. To get it, capture console errors, for example with Sentry's `captureConsoleIntegration`. ```typescript if (!(await dcupl.init())) { // The cause is in the console line '[Dcupl] Initialization failed' throw new Error('dcupl.init() failed, see the console for the cause'); } ``` Most problems with model definitions don't fail `init()`. A reference to a model that isn't registered (`MissingModel`) or an invalid definition such as an unknown property type (`InvalidModelDefinition`) resolve `true` and show up in `summary()` in the `ModelDefinitionError` group. Check that group during development: ```typescript const definitionErrors = dcupl.quality.errors.summary().filter((group) => group.errorGroup === 'ModelDefinitionError'); if (definitionErrors.length > 0) { throw new Error(`Invalid models: ${definitionErrors.map((group) => group.key).join(', ')}`); } ``` ### When data calls throw `dcupl.data.*` calls only stage changes. They throw only for a payload that isn't an array or options without `model` ([Thrown Errors](#thrown-errors)), and problems with the data don't throw later either: - `data.update()` ignores keys that don't exist, also on a model that has no data yet (use `data.upsert()` to add items). `data.remove()` ignores unknown keys, also on a model without data. - Values that don't fit the model become [model errors](#model-errors), not exceptions. - Data for a model key that isn't registered loads into no model (the rows stay staged under that key until such a model is registered). `init()` and a full rebuild log only a console line ([Data for an Unknown Model](/sdk/guides/common-mistakes#data-for-an-unknown-model)); a `data.upsert()`, `update()` or `remove()` applied by a partial update after `init()` logs nothing. - `dcupl.query.one()` returns `null` for an item key that doesn't exist. So validate input before you stage it, and check what `await dcupl.update()` resolves to. --- ## Event-based error handling dcupl emits no error events: a failed `init()` or `update()` resolves `false`, and data and loader problems are collected in `dcupl.quality.errors.summary()`. Events help you spot slow operations and follow what happened before a failure. `dcupl.on()` receives the instance's own events (models, data, `init()`/`update()`, list creation); list events such as `query_execute` only reach `list.on()`. ```typescript const dcupl = new Dcupl({ quality: { enabled: true }, }); // Global event listener dcupl.on((message) => { // Track slow operations if (message.duration && message.duration > 100) { console.warn(`Slow operation: ${message.type} took ${message.duration}ms`); } // Track staged data changes for debugging (the model is in the staged container, args[0]) if (message.type === 'data_change') { const { model, type } = message.args[0]; console.log(`Data staged (${type}):`, model); } }); // List-specific error tracking const list = dcupl.lists.create({ modelKey: 'Product' }); list.on((message) => { if (message.type === 'query_execute') { // Track query execution console.log(`Query executed in ${message.duration}ms`); } }); ``` --- ## Logging configuration In SDK 2.0, the core SDK's logging can't be configured. `@dcupl/core` writes its errors and warnings (for example `[Catalog] Invalid pivot options`) to the console at a fixed level. Neither `logger.configure()` from `@dcupl/common` nor the `logging` option of `new Dcupl({ ... })` changes that. > **Watch out:** `logger.configure()`, `Logger.setGlobalConfig()` and `Logger.setModuleLevel()` from `@dcupl/common` don't reach the core SDK's logs (prefixes such as `Dcupl`, `Catalog`, `ModelParser`, `QualityController`). The `logging` option of `new Dcupl({ ... })` is typed but not wired; its only effect is that `level: 'debug'` prints a `[Filter] Caches invalidated` line whenever a list's caches are dropped. What you can configure is the loader: `@dcupl/loader` logs through the shared `logger` from `@dcupl/common`. Configure it **before** you create the loader, because each loader copies the settings when it's created: ```typescript import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; import { logger, Logger } from '@dcupl/common'; declare const errorTracker: { captureError(entry: unknown): void }; // your error tracking service logger.configure({ // Log levels: 'silent' | 'error' | 'warn' | 'info' | 'debug' level: process.env.NODE_ENV === 'development' ? 'debug' : 'warn', // Output format format: process.env.NODE_ENV === 'development' ? 'pretty' : 'json', // Custom log handler for external services (replaces the console output) onLog: (entry) => { if (entry.level === 'error') { // e.g. '[AppLoader] Failed to load data resource' with { url, model } in entry.context errorTracker.captureError(entry); } }, }); // Module-specific levels by log prefix ('AppLoader', 'CsvParser', 'DcuplCsvParser'); apply immediately Logger.setModuleLevel('CsvParser', 'error'); const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); // created after configure(), so it uses these settings dcupl.loaders.add(loader); ``` A log entry has `level`, `message`, `timestamp`, and optionally `prefix` (the module), `context` and `error`. For the core SDK, rely on what it returns instead of its logs: `init()` and `update()` resolve `false` when they fail, failed calls throw (wrap your own calls in `try`/`catch`), and data and loader problems are listed by `dcupl.quality.errors.summary()`. The cause of a failed `init()` or `update()` is only in the console ([When init() Resolves false](#when-init-resolves-false)). --- ## Error recovery strategies ### Graceful degradation ```typescript async function initializeDcupl() { const dcupl = new Dcupl({ config: { projectId: process.env.DCUPL_PROJECT_ID!, apiKey: process.env.DCUPL_API_KEY!, }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); try { // Try to load from remote await loader.config.fetch(); } catch (error) { console.warn('Remote config failed, using local fallback'); // Fallback to local configuration loader.config.set({ resources: [ { type: 'model', url: '/models/Product.dcupl.json' }, { key: 'products', type: 'data', url: '/data/products.json', model: 'Product' }, ], }); } await loader.process(); if (!(await dcupl.init())) { throw new Error('dcupl.init() failed'); // init() resolves false instead of throwing } // Check quality even after successful init const errors = dcupl.quality.errors.summary(); if (errors.length > 0) { const total = errors.reduce((sum, group) => sum + group.totalCount, 0); console.warn(`Initialized with ${total} data quality issues`); } return dcupl; } ``` ### Partial data loading Load the critical resources first and the optional ones afterwards. A tag filter skips untagged resources, so tag every resource, including the model resources. Data resources default to `updateType: 'set'`, which replaces all rows of their model. When an optional resource feeds the same model as a critical one, give it `updateType: 'upsert'`; otherwise the optional run replaces the rows the critical run loaded. ```typescript loader.config.set({ resources: [ { type: 'model', url: '/models/Product.dcupl.json', tags: ['critical'] }, { key: 'top-sellers', type: 'data', url: '/data/top-sellers.json', model: 'Product', tags: ['critical'] }, { key: 'long-tail', type: 'data', url: '/data/long-tail.json', model: 'Product', tags: ['optional'], options: { updateType: 'upsert' }, // add to the critical rows instead of replacing them }, ], }); ``` ```typescript // Processes the resources with the given tags and returns the ones that failed async function processTags(loader: DcuplAppLoader, resourceTags: string[]): Promise { let failed: string[] = []; const unsubscribe = loader.on((progress) => { // Each update lists every failure of the run so far: keep the latest list failed = progress.failedItems.map((item) => item.key ?? item.url); }); try { await loader.process({ resourceTags }); } finally { unsubscribe(); } return failed; } // The loader is already added (dcupl.loaders.add) and has the config above async function loadCriticalDataFirst(dcupl: Dcupl, loader: DcuplAppLoader) { // Load critical resources first const failedCritical = await processTags(loader, ['critical']); if (failedCritical.length > 0) { // Can't continue without critical data throw new Error(`Critical data failed to load: ${failedCritical.join(', ')}`); } if (!(await dcupl.init())) { throw new Error('dcupl.init() failed'); } // Load optional resources afterwards const failedOptional = await processTags(loader, ['optional']); if (!(await dcupl.update())) { console.warn('dcupl.update() failed: the optional resources are not applied'); } if (failedOptional.length > 0) { console.warn(`Optional resources failed: ${failedOptional.join(', ')}`); // Continue without them } } ``` ### Validation error handling ```typescript function handleValidationErrors(dcupl: Dcupl, options: { strict: boolean }) { // One entry per (model, attribute, errorType) group const errors = dcupl.quality.errors.summary(); // Categorize errors by severity const critical = errors.filter((e) => e.errorType === 'MissingModel' || e.errorType === 'InvalidModelDefinition'); const warnings = errors.filter( (e) => e.errorType === 'RemoteReferenceKeyNotFound' || e.errorType === 'UndefinedValue' || e.errorType === 'WrongDataType', ); const info = errors.filter((e) => e.errorType === 'UndefinedAttribute' || e.errorType === 'UnknownColumn'); // Report errors if (critical.length > 0) { console.error('Critical errors - cannot proceed:', critical); if (options.strict) { throw new Error('Critical validation errors'); } } if (warnings.length > 0) { console.warn('Data warnings:', warnings); } if (info.length > 0 && process.env.NODE_ENV === 'development') { console.info('Data info:', info); } return { critical, warnings, info }; } ``` --- ## Debugging tips ### Enable debug logging There is no debug output to switch on in SDK 2.0: the core SDK's log level can't be changed, and the loader logs only errors and one warning (a `config.fetch()` option that a variable overrides), which its default level already shows. To follow what the SDK does, log its events ([Track Event Flow](#track-event-flow)) and inspect the quality summary ([Inspect Quality Metrics](#inspect-quality-metrics)). ### Inspect quality metrics ```typescript await dcupl.init(); // Error totals per (model, attribute, errorType) group console.table( dcupl.quality.errors.summary().map(({ model, attribute, errorType, type, totalCount }) => ({ model, attribute, errorType, type, totalCount, })), ); // Columns in loaded CSV files that the model doesn't declare (reported for CSV resources only) const unknownColumns = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'errorType', operator: 'eq', value: 'UnknownColumn' }], }); console.log( 'Unused columns:', unknownColumns.map((e) => `${e.model?.key}.${e.attribute}`), ); ``` ### Track event flow ```typescript // Development: log the instance's events (list events go to list.on()) if (process.env.NODE_ENV === 'development') { dcupl.on((msg) => { console.log(`[dcupl] ${msg.type}`, { args: msg.args, // e.g. the staged container ({ model, type, data }) for data_change duration: msg.duration ? `${msg.duration}ms` : undefined, }); }); } ``` --- ## Integration with error tracking services ### Sentry integration ```typescript import * as Sentry from '@sentry/browser'; import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; import { logger } from '@dcupl/common'; // The core SDK logs to the console only: captureConsoleIntegration reports those errors, // including the cause of a failed init() ('[Dcupl] Initialization failed') Sentry.init({ dsn: 'your-dsn', integrations: [Sentry.captureConsoleIntegration({ levels: ['error'] })] }); // Loader logs: configure before creating the loader. onLog replaces the console output. logger.configure({ onLog: (entry) => { if (entry.level === 'error') { Sentry.captureException(entry.error ?? new Error(entry.message), { tags: { module: entry.prefix }, extra: entry.context, }); } }, }); const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // ... loader.config.fetch(), loader.process() // After init, report quality issues (including loader errors) await dcupl.init(); const errors = dcupl.quality.errors.summary(); if (errors.length > 0) { Sentry.captureMessage('dcupl data quality issues', { level: 'warning', extra: { errorGroups: errors.length, errors: errors.slice(0, 10) }, }); } ``` --- ## Best practices ### 1. Always enable quality checking ```typescript // Quality checks are on by default - keep them on, at least in development const dcupl = new Dcupl({ quality: { enabled: true }, }); ``` ### 2. Check the result and the errors after init ```typescript if (!(await dcupl.init())) { throw new Error('dcupl.init() failed'); // init() resolves false instead of throwing } const errors = dcupl.quality.errors.summary(); // Don't ignore errors! ``` ### 3. Use event listeners for real-time monitoring ```typescript import type { DcuplUpdateMessage } from '@dcupl/common'; declare function reportSlowOperation(message: DcuplUpdateMessage): void; // yours dcupl.on((msg) => { if (msg.duration && msg.duration > 100) { reportSlowOperation(msg); } }); ``` ### 4. Implement graceful fallbacks ```typescript import type { AppLoaderConfiguration } from '@dcupl/common'; declare const localFallback: AppLoaderConfiguration.ProjectConfiguration; // a configuration you ship with the app try { await loader.config.fetch(); // throws when the configuration can't be loaded } catch { loader.config.set(localFallback); } ``` ### 5. Log appropriately per environment ```typescript import { logger } from '@dcupl/common'; // Applies to the loader's logs; call it before creating the loader. // The core SDK's logging can't be configured. logger.configure({ level: process.env.NODE_ENV === 'production' ? 'warn' : 'debug', format: process.env.NODE_ENV === 'production' ? 'json' : 'pretty', }); ``` --- ## Next steps - [Quality & validation](/sdk/advanced/quality-validation) — Data validation rules - [Events reference](/sdk/api/events) — All event types - [Production Debugging](/sdk/guides/production-debugging) — Error tracking and monitoring in production - [Debugging with Console](/sdk/guides/debugging) — Inspect your running app in dcupl Console - [DcuplAppLoader API](/sdk/api/loader) — Loader configuration --- # Common mistakes Source: https://docs.dcupl.com/sdk/guides/common-mistakes Description: Avoid these beginner pitfalls when working with dcupl # Common mistakes This guide helps you identify and fix the most common mistakes beginners make with dcupl. Each section shows the wrong approach, explains why it fails, and provides the correct solution. ## Forgetting to await init() The most common mistake is querying data before dcupl has finished initializing. ### The problem ```typescript title="Wrong: Querying before init completes" const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [...] }); dcupl.data.set([...], { model: 'Product' }); dcupl.init(); // Missing await! const list = dcupl.lists.create({ modelKey: 'Product' }); const results = list.catalog.query.items(); // Not guaranteed to see the data: init() may not have finished ``` **Why it fails:** `init()` is async and returns a `Promise`. Without `await`, your code doesn't wait for initialization to finish, so results can be empty or incomplete — and you never learn whether it worked: `init()` does not throw. It catches its own errors, logs "Initialization failed" and resolves `false`. ### The fix ```typescript title="Correct: Always await init()" const dcupl = new Dcupl(); dcupl.models.set({ key: 'Product', properties: [...] }); dcupl.data.set([...], { model: 'Product' }); const ready = await dcupl.init(); // Wait for initialization if (!ready) { throw new Error('dcupl initialization failed'); // init() resolves false instead of throwing } const list = dcupl.lists.create({ modelKey: 'Product' }); const results = list.catalog.query.items(); // Results: [{ key: 'p1', ... }, { key: 'p2', ... }] ``` `init()` does not run or wait for loaders. With a loader, await `loader.process()` first, then `dcupl.init()`: ```typescript title="Correct: Load, then initialize" await loader.config.fetch(); await loader.process(); // fetches models and data await dcupl.init(); // processes what the loader added ``` > **Tip:** The same applies to `dcupl.update()` after modifying data. Always await it — like `init()`, it resolves `false` instead of throwing when it fails. ## Error: Model with key X not found A list or query for a model key that isn't registered throws. It does not return `[]`. ```typescript title="Wrong: Typo in model key" dcupl.models.set({ key: 'Product', properties: [...] }); await dcupl.init(); const list = dcupl.lists.create({ modelKey: 'Products' }); // Typo: 'Products' vs 'Product' // Throws: Error: Model with key Products not found ``` ```typescript title="Correct: Use a registered model key" console.log('Available models:', dcupl.models.keys()); // ['Product'] const list = dcupl.lists.create({ modelKey: 'Product' }); ``` With a loader, `loader.process()` only stages the models; `dcupl.init()` registers them. Create lists after `await dcupl.init()`. ## Error: Data must be an Array The `dcupl.data.*` calls take an array of items, also for a single item. ```typescript title="Wrong: A single item" dcupl.data.update({ key: 'p1', price: 199 }, { model: 'Product' }); // Throws: Error: Invalid Data - Data must be an Array ``` ```typescript title="Correct: Wrap it in an array" dcupl.data.update([{ key: 'p1', price: 199 }], { model: 'Product' }); ``` Options without `model`, or a `null` or `undefined` payload, throw `Invalid Data Container`. In TypeScript, a single item and options without `model` are compile errors, and so is a `null` or `undefined` payload with the `strict` option (`strictNullChecks`). ## Query returns no results When your query returns an empty array unexpectedly, check these common causes. ### Data for an unknown model Data staged for a model key that doesn't exist loads into no model and raises no error: the rows stay staged under that key and load once a model with that key is registered. `init()` still resolves `true`, `dcupl.quality.errors.summary()` stays empty, and the model you meant has no items. The only sign is a console line during `init()` or a full rebuild, `[ModelParser] Model not found during data processing`. Data staged after `init()` with `data.upsert()`, `update()` or `remove()` and applied as a partial update doesn't even log it; the `dcupl_partial_update` event lists the unknown key in `changedModels`. ```typescript title="Wrong: Typo in the data's model key" dcupl.models.set({ key: 'Product', properties: [{ key: 'name', type: 'string' }] }); dcupl.data.set([{ key: 'p1', name: 'Laptop' }], { model: 'Prodcut' }); // Typo await dcupl.init(); // true dcupl.lists.create({ modelKey: 'Product' }).catalog.query.count(); // 0 ``` ```typescript title="Correct: Stage the data for a registered model" console.log('Available models:', dcupl.models.keys()); // ['Product'] dcupl.data.set([{ key: 'p1', name: 'Laptop' }], { model: 'Product' }); ``` In a loader configuration, a typo in a data resource's `model` logs nothing by default. Missing-model handling is on unless the configuration's `options.missingModelHandling`, or the `options` you pass to `loader.process()` (they replace the configuration's), turns it off (see [ProcessOptions](/sdk/api/loader#processoptions)). The loader creates a second model under the misspelled key, infers its properties from the rows and loads them there, so the model you meant stays empty. Check `dcupl.models.keys()` for a key you didn't expect. With `missingModelHandling.autoGenerateProperties: false`, or `missingModelHandling: {}`, which also turns it off, the rows load into no model and the console shows the same `Model not found` line. A CSV resource's rows are then not parsed at all, so they don't load later either. ### Wrong attribute name ```typescript title="Wrong: Attribute name doesn't match model" dcupl.models.set({ key: 'Product', properties: [ { key: 'productName', type: 'string' }, // Property is 'productName' ], }); list.catalog.query.addCondition({ attribute: 'name', // Wrong: should be 'productName' operator: 'eq', value: 'Laptop', }); const results = list.catalog.query.items(); // Results: [] (no match because attribute doesn't exist) ``` **Debug tip:** Inspect the model definition: ```typescript const model = dcupl.models.get('Product'); console.log( 'Properties:', model?.properties?.map((p) => p.key), ); ``` ### Data type mismatch ```typescript title="Wrong: String vs number comparison" dcupl.data.set( [ { key: 'p1', price: 100 }, // price is a number ], { model: 'Product' }, ); list.catalog.query.addCondition({ attribute: 'price', operator: 'eq', value: '100', // Wrong: string instead of number }); // Results: [] (no match - types don't match) ``` ```typescript title="Correct: Match the data type" list.catalog.query.addCondition({ attribute: 'price', operator: 'eq', value: 100, // Correct: number type }); ``` ## Forgetting to call items() The query builder is fluent - it returns the query API, not results. ### The problem ```typescript title="Wrong: Expecting results from addCondition" list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const results = list.catalog.query; // Wrong: This is the query API object, not results console.log(results.length); // undefined ``` ### The fix ```typescript title="Correct: Call items() to get results" list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics', }); const results = list.catalog.query.items(); // Call items() to execute console.log(results.length); // 2 ``` **Related methods that return data:** ```typescript list.catalog.query.items(); // All matching items list.catalog.query.first(); // First matching item list.catalog.query.count(); // Number of matches (the page length once start/count are applied) ``` ## Not cleaning up lists Lists consume memory and event listeners. Forgetting to destroy them causes memory leaks. ### The problem ```typescript title="Wrong: Lists accumulate without cleanup" function showProducts(category: string) { const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category }); return list.catalog.query.items(); // List is never destroyed - memory leak! } // Each call creates a new list that's never cleaned up showProducts('Electronics'); showProducts('Furniture'); showProducts('Clothing'); ``` ### The fix **Option 1: Destroy after use** ```typescript title="Correct: Destroy list when done" function showProducts(category: string) { const list = dcupl.lists.create({ modelKey: 'Product' }); list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category }); const results = list.catalog.query.items(); list.destroy(); // Clean up return results; } ``` **Option 2: Reuse a single list** ```typescript title="Better: Reuse a single list" const productList = dcupl.lists.create({ modelKey: 'Product' }); function showProducts(category: string) { productList.catalog.query.clear(); // Reset filters productList.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: category, }); return productList.catalog.query.items(); } ``` **Option 3: Framework cleanup hooks** ```typescript title="React: Cleanup in useEffect" useEffect(() => { const list = dcupl.lists.create({ modelKey: 'Product' }); // ... use list return () => list.destroy(); // Cleanup on unmount }, []); ``` ## Mutating items directly Query results are copies by default, so direct mutations don't persist. ### The problem ```typescript title="Wrong: Mutating items directly" const products = list.catalog.query.items(); products[0].price = 199; // Direct mutation // The change is lost - original data unchanged const refreshed = list.catalog.query.items(); console.log(refreshed[0].price); // Still the original price ``` **Why it fails:** `items()` returns cloned data (deep clones are on by default, `performance: { clone: { enabled: true } }`). Mutations don't affect the source. With `performance: { clone: { enabled: false } }` you get dcupl's stored objects instead, and a mutation changes dcupl's data directly, bypassing `dcupl.data.update()` and `dcupl.update()`. Don't mutate results in either mode. ### The fix ```typescript title="Correct: Use dcupl.data.update()" // Update through the data API (changes existing items; use upsert() to add new ones) dcupl.data.update([{ key: 'p1', price: 199 }], { model: 'Product' }); await dcupl.update(); // Process changes const results = list.catalog.query.items(); console.log(results[0].price); // 199 - updated! ``` ## Properties disappear after models.set() In SDK 2.0, `dcupl.models.set()` registers the whole definition for its key. Called again with only the property you want to add, it replaces the model: the properties you left out are removed, and their values are gone after the next `update()`. ```typescript title="Wrong: set() with only the new property" dcupl.models.set({ key: 'Product', properties: [{ key: 'color', type: 'string' }] }); await dcupl.update(); // Product now has only 'color': 'name' and 'price' are gone ``` ```typescript title="Correct: update() merges into the existing model" dcupl.models.update({ key: 'Product', properties: [{ key: 'color', type: 'string' }] }); await dcupl.update(); // Product has 'name', 'price' and 'color' ``` To change a property the model already has, get the definition, change it and `set()` the whole definition ([Models API](/sdk/api/models#set)). ## Case sensitivity in filters String comparisons are case-sensitive by default. ### The problem ```typescript title="Wrong: Case mismatch" // Data has: { status: 'Active' } list.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', // Won't match 'Active' }); // Results: [] ``` ### The fix **Option 1: Use transform for case-insensitive matching** ```typescript title="Correct: Use lowercase transform" list.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'active', options: { transform: ['lowercase'] }, }); ``` **Option 2: Match the exact case in your data** ```typescript title="Alternative: Match exact case" list.catalog.query.addCondition({ attribute: 'status', operator: 'eq', value: 'Active', // Matches 'Active' exactly }); ``` > **Best practice:** Normalize data during import (e.g., lowercase all status values) for consistent filtering. ## Search finds only exact matches The `find` operator with a plain string matches the whole value, not a part of it. ### The problem ```typescript title="Wrong: Expecting a substring search" // Data has: 'Laptop', 'Gaming Laptop', 'laptop stand' list.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: 'Laptop', }); // Results: only 'Laptop' ``` ### The fix Wrap the search term in slashes: `find` then treats it as a regular expression and matches it anywhere in the value. Add `regexFlags: 'i'` (or `transform: ['lowercase']`) to ignore case: ```typescript title="Correct: Regex for a substring search" list.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: '/laptop/', options: { regexFlags: 'i' }, }); // Results: 'Laptop', 'Gaming Laptop', 'laptop stand' ``` Escape regex characters in user input before you put it between the slashes. See [find](/sdk/api/query-operators#find) for the options. ## Import and installation issues ### Missing package ```typescript title="Wrong: Incorrect import" import { Dcupl } from 'dcupl'; // Wrong package name // Error: Cannot find module 'dcupl' ``` ```typescript title="Correct: Import from @dcupl/core" import { Dcupl } from '@dcupl/core'; ``` ### Loader not installed ```typescript title="Wrong: Using loader without installing" import { DcuplAppLoader } from '@dcupl/loader'; // Error: Cannot find module '@dcupl/loader' ``` ```bash title="Fix: Install the loader package" npm install @dcupl/loader ``` ### Version mismatch Ensure all dcupl packages are the same version. `@dcupl/core` depends on exactly its own version of `@dcupl/common`, and `@dcupl/loader` on exactly its own versions of both, so pin the same exact version for every package: ```json filename="package.json" { "dependencies": { "@dcupl/common": "2.0.0", "@dcupl/core": "2.0.0", "@dcupl/loader": "2.0.0" } } ``` ## Filters accumulating unexpectedly Filters persist until explicitly cleared. `addCondition()` combines conditions on different attributes with AND, and conditions on the same attribute with OR (each attribute gets its own `or` group). Use `setCondition()` to replace the conditions of an attribute. ### The problem ```typescript title="Wrong: Filters accumulate" // First filter list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); const electronics = list.catalog.query.items(); // Works: 2 items // Add second filter - accumulates with first! list.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 500 }); const expensive = list.catalog.query.items(); // Only expensive Electronics, not all expensive items ``` ### The fix ```typescript title="Correct: Clear before new query" // First query list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: 'Electronics' }); const electronics = list.catalog.query.items(); // Clear and start fresh list.catalog.query.clear(); list.catalog.query.addCondition({ attribute: 'price', operator: 'gte', value: 500 }); const expensive = list.catalog.query.items(); // All expensive items ``` ## Quick debugging checklist When things don't work as expected: 1. **Check init:** Did you `await dcupl.init()` (after `await loader.process()` when you use a loader), and did it resolve `true`? 2. **Check model keys:** Is the model name spelled correctly, in the list and in the data you staged? 3. **Check attribute names:** Do they match your model properties? 4. **Check data types:** Are filter values the correct type (number vs string)? 5. **Check case:** Are string comparisons matching case? 6. **Check search:** Does a `find` for part of a value use the `/…/` form? 7. **Check filters:** Do you need to `clear()` previous filters? 8. **Check execution:** Did you call `items()` to get results? ```typescript title="Debug helper" // Log current state console.log('Models:', dcupl.models.keys()); console.log('Query:', JSON.stringify(list.catalog.query.current(), null, 2)); console.log('Results:', list.catalog.query.items().length); ``` ## Next steps - [Error handling](/sdk/guides/error-handling) — What fails, what throws, and where the errors go - [Debugging with Console](/sdk/guides/debugging) — Inspect your running app in dcupl Console - [Quick start](/sdk/quick-start) — Build your first query correctly --- # Production deployment Source: https://docs.dcupl.com/sdk/guides/deployment Description: Deploy dcupl applications to production with confidence # Production deployment Deploy your dcupl application to production with this checklist and the dcupl-specific parts of a deployment: the production setup, API keys, data loading and refresh, long-running servers, and the Content Security Policy. For error tracking and monitoring, see [Production Debugging](/sdk/guides/production-debugging). ## Pre-deployment checklist ### Code quality - [ ] All quality validation errors resolved - [ ] TypeScript compilation passes without errors - [ ] Tests passing, including the data quality checks ([Testing](/sdk/guides/testing)) - [ ] The results of `init()` and `update()` are checked (they resolve `false` instead of throwing) ### Configuration - [ ] `config.version` names the version you published to (the default is `draft`) - [ ] Quality validation disabled, or kept on on purpose to report data errors - [ ] CDN URLs verified ### Performance - [ ] `@dcupl/core` is bundled once, in one version (each entry bundles its dependencies, so tree shaking can't trim the SDK) - [ ] Data loading strategy defined - [ ] Caching strategy implemented - [ ] Large dataset handling tested ### Security - [ ] The browser bundle carries only a scoped read-only key - [ ] The CLI key and other keys that can write are only in server environments and CI secrets - [ ] The project is private: a request for one of its files without one of the project's keys answers `403`. New projects are private by default; for an older project, private file serving is part of a paid plan that dcupl turns on for your project; ask your dcupl contact ([What the API Key Protects](#what-the-api-key-protects)) - [ ] Content Security Policy allows the dcupl hosts ## Production configuration ### Optimized dcupl setup ```typescript filename="src/config/production.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; export function createProductionDcupl() { const dcupl = new Dcupl({ config: { projectId: process.env.DCUPL_PROJECT_ID!, apiKey: process.env.DCUPL_API_KEY!, version: 'production', // The version you published to; the default is 'draft' }, quality: { enabled: false, // Disable for performance }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); return { dcupl, loader }; } export async function initializeProduction() { const { dcupl, loader } = createProductionDcupl(); try { // Fetch published configuration await loader.config.fetch(); await loader.process(); // init() doesn't throw: it logs the error and resolves false if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); return dcupl; } catch (error) { console.error('Failed to initialize dcupl:', error); throw error; } } ``` `loader.config.fetch()` throws when the loader configuration can't be fetched. A model or data file that fails to download does **not** make `loader.process()` or `init()` return `false`: the failure is logged and recorded as a loader error (`type: 'loader'`, e.g. `InvalidResource`) in `dcupl.quality.errors.summary()`, even with quality disabled. Check for those (or for `failedItems` in the loader progress, see [Capture Loader Errors](/sdk/guides/production-debugging#capture-loader-errors)) before you treat the instance as ready. [Load from the console](/sdk/guides/load-from-console) covers the project ID, the key, the version and what each fetch error means. ### Environment variables ```bash filename=".env.production" # Production Configuration DCUPL_PROJECT_ID=prod-project-id DCUPL_API_KEY=prod-api-key-xxx NODE_ENV=production ``` In a browser app, every variable the bundle reads ends up in the bundle: see [What the API Key Protects](#what-the-api-key-protects) for which key goes there. ## Performance optimization ### 1. Disable quality validation Quality validation adds overhead. Disable in production: ```typescript const dcupl = new Dcupl({ quality: { enabled: false, // Production: disabled }, }); ``` **Trade-off:** with quality disabled, `dcupl.quality.errors.summary()` returns only loader errors (resources that failed to load) and the `$DcuplErrorTrackingErrors` model does not exist. Keep it enabled if you report data-quality errors from production (see [Production Debugging](/sdk/guides/production-debugging)), and measure the difference on your own data. ### 2. Use partial updates Avoid full data reloads when possible: ```typescript // Bad: Full reload dcupl.data.set(allData, { model: 'Product' }); await dcupl.init(); // Good: Partial update — only the changed attributes and what depends on them are reprocessed dcupl.data.update([{ key: 'p1', price: 999 }], { model: 'Product' }); await dcupl.update(); ``` `data.update()` only changes items that already exist (unknown keys are ignored); use `data.upsert()` to add new items in the same call. `update()` falls back to a full rebuild when a `data.set()` or `data.reset()` is part of the batch, when a model definition changed after `init()` (`models.set()`, `models.update()` or `models.remove()`), when the changes can alter the result of a [query reference](/sdk/api/models#reference), or when more than half of the models **and** more than half of their items changed. An `update()` with nothing staged also re-runs the full build. Both thresholds are configurable, see [Partial Updates](/sdk/advanced/performance#partial-updates). ### 3. Implement list pooling Reuse lists instead of creating new ones: ```typescript class ListPool { private pool: Map = new Map(); get(dcupl: Dcupl, modelKey: string): DcuplList { const pool = this.pool.get(modelKey) || []; if (pool.length > 0) { return pool.pop()!; } return dcupl.lists.create({ modelKey }); } release(list: DcuplList) { list.catalog.query.clear(); const modelKey = list.modelKey; const pool = this.pool.get(modelKey) || []; pool.push(list); this.pool.set(modelKey, pool); } } ``` ### 4. Use pagination For large result sets: ```typescript const productList = dcupl.lists.create({ modelKey: 'Product' }); // Page 1, 50 items (reasonable page size) productList.catalog.query.applyOptions({ start: 0, count: 50 }); const page = productList.catalog.query.items(); // Total number of matches (count() returns the page length) const total = productList.catalog.fn.metadata().currentSize; ``` ### 5. Lazy load non-critical data Load the resources the first screen needs, render, then load the rest in the background. Tag the resources in your loader configuration and process them by tag. A tag filter skips untagged resources, so tag the model resources too: ```typescript // Resources tagged 'critical' (the models and the first data), then the app can render await loader.process({ resourceTags: ['critical'] }); if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); renderApp(dcupl); // The rest in the background, applied with update() void loader.process({ resourceTags: ['optional'] }).then(() => dcupl.update()); ``` A data resource replaces all rows of its model by default (`options.updateType` is `'set'`). When an optional resource feeds the same model as a critical one, give it `options: { updateType: 'upsert' }`. See [Partial Data Loading](/sdk/guides/error-handling#partial-data-loading) for a version that reports failed resources. ## Error handling ### Graceful initialization ```typescript export async function initWithFallback() { const { dcupl, loader } = createProductionDcupl(); try { await loader.config.fetch(); await loader.process(); if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); return { dcupl, status: 'ready' }; } catch (error) { console.error('dcupl initialization failed:', error); // Option 1: Return degraded state return { dcupl: null, status: 'error', error }; // Option 2: Retry with exponential backoff // return retryInit(); } } ``` ### Retry logic ```typescript async function retryInit(maxRetries = 3): Promise { for (let attempt = 1; attempt <= maxRetries; attempt++) { // A fresh instance per attempt: the failed attempt's loader errors would // otherwise stay in dcupl.quality.errors.summary() const { dcupl, loader } = createProductionDcupl(); try { await loader.config.fetch(); await loader.process(); if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); return dcupl; } catch (error) { console.error(`Init attempt ${attempt} failed:`, error); dcupl.destroy(); if (attempt === maxRetries) { throw new Error(`Failed after ${maxRetries} attempts`); } // Exponential backoff await new Promise((resolve) => setTimeout(resolve, 1000 * Math.pow(2, attempt - 1))); } } throw new Error('Unreachable'); } ``` ### Query error handling ```typescript function safeQuery(list: DcuplList) { try { return list.catalog.query.items(); } catch (error) { console.error('Query failed:', error); return []; // Return empty array as fallback } } ``` ## Monitoring [Production Debugging](/sdk/guides/production-debugging) covers error tracking with Sentry, startup and query timings, loader failures and data-quality alerts. For a server, add a health check: ### Health check endpoint ```typescript // Express/Node.js example // Liveness: the process is up app.get('/health', (req, res) => { res.json({ status: 'ok' }); }); // Readiness: dcupl is initialized and has data app.get('/health/dcupl', (req, res) => { try { const list = dcupl.lists.create({ modelKey: 'Product' }); // throws when the model is missing const count = list.catalog.query.count(); list.destroy(); if (count === 0) throw new Error('No products loaded'); res.json({ status: 'healthy', productCount: count, timestamp: new Date().toISOString(), }); } catch (error) { res.status(503).json({ status: 'unhealthy', error: (error as Error).message, timestamp: new Date().toISOString(), }); } }); ``` `/health/dcupl` answers `503` when the model is missing or has no items, for example after a failed load. Use it to take the instance out of rotation (a readiness check), not to restart it: a restart doesn't fix a data problem. ## Deployment strategies ### Static site (Vercel, Netlify) ```typescript filename="src/lib/dcupl.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; let dcuplInstance: Dcupl | null = null; export async function getDcupl(): Promise { if (dcuplInstance) { return dcuplInstance; } const dcupl = new Dcupl({ config: { projectId: process.env.NEXT_PUBLIC_DCUPL_PROJECT_ID!, // NEXT_PUBLIC_ variables are inlined into the browser bundle: a scoped read-only key only apiKey: process.env.NEXT_PUBLIC_DCUPL_API_KEY!, version: 'production', }, quality: { enabled: false }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); await loader.config.fetch(); await loader.process(); // Don't cache a failed instance: init() resolves false instead of throwing if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); dcuplInstance = dcupl; return dcupl; } ``` ### Node.js server and SSR On a server, build one instance per process and share it between requests. `init()` parses every model and data file, so an instance per request is slow: ```typescript filename="src/server/dcupl.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; let instance: Promise<{ dcupl: Dcupl; loader: DcuplAppLoader }> | null = null; // One instance per process, shared by every request export function getDcupl() { instance ??= (async () => { const dcupl = new Dcupl({ config: { projectId: process.env.DCUPL_PROJECT_ID!, apiKey: process.env.DCUPL_API_KEY!, // stays on the server version: 'production', }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); await loader.config.fetch(); await loader.process(); if (!(await dcupl.init())) throw new Error('dcupl.init() failed'); return { dcupl, loader }; })().catch((error) => { instance = null; // let the next request try again throw error; }); return instance; } // Call it after you publish new data (from a webhook or a timer). // Reloading only the resources tagged 'data' skips the model files; // a plain process() also reloads the model files; update() applies their changes. export async function refreshData(): Promise { const { dcupl, loader } = await getDcupl(); await loader.config.fetch(); // also renews the ?v= version parameter of CDN urls await loader.process({ resourceTags: ['data'] }); return dcupl.update(); } ``` ```typescript title="Per request" app.get('/products', async (req, res) => { const { dcupl } = await getDcupl(); const list = dcupl.lists.create({ modelKey: 'Product' }); try { list.catalog.query.addCondition({ attribute: 'category', operator: 'eq', value: String(req.query.category) }); res.json(list.catalog.query.items({ count: 50 })); } finally { list.destroy(); // lists stay registered in dcupl.lists until destroyed } }); ``` - **New data:** a running instance never picks up data you publish later. Reload the data resources as above ([Problem: Stale Data](/sdk/guides/production-debugging#problem-stale-data)). If the model files changed too, call `loader.process()` without `resourceTags` in `refreshData()`: it reloads the model files as well, and the `dcupl.update()` after it applies the changed models to the running instance, so you need no new instance. - **Lists:** a list created per request stays in `dcupl.lists` until `list.destroy()`. For a one-off query, `dcupl.query.execute({ modelKey, queries })` needs no list. - **Shared results:** results are deep clones by default (`performance.clone`), so a handler that changes a result doesn't change the shared instance. Keep cloning on for a shared instance. - **Runtime:** the loader uses the global `fetch` (see [Compatibility](/sdk/compatibility#nodejs) for the Node.js versions). The packages work with both `import` and `require()`. ### Docker container ```dockerfile filename="Dockerfile" FROM node:24-alpine WORKDIR /app # Copy package files COPY package*.json ./ RUN npm ci # Copy app files COPY . . # Build, then drop the dev dependencies the build needed RUN npm run build && npm prune --omit=dev # Set environment ENV NODE_ENV=production # Run CMD ["npm", "start"] ``` Pass `DCUPL_PROJECT_ID` and `DCUPL_API_KEY` as environment variables when you start the container, not as `ENV` lines in the image. ### Kubernetes The dcupl-specific parts of a deployment are the key from a secret and the two health checks: ```yaml filename="k8s/deployment.yaml" # spec.template.spec of the Deployment containers: - name: app image: your-registry/dcupl-app:latest env: - name: DCUPL_PROJECT_ID valueFrom: secretKeyRef: name: dcupl-secrets key: project-id - name: DCUPL_API_KEY valueFrom: secretKeyRef: name: dcupl-secrets key: api-key livenessProbe: httpGet: path: /health port: 3000 readinessProbe: httpGet: path: /health/dcupl port: 3000 initialDelaySeconds: 10 periodSeconds: 10 ``` ## Security considerations ### What the API key protects The loader sends the API key with every request to the dcupl CDN, as an `api-key` query parameter next to the `?v=` version parameter. In a browser app, anyone who opens the network tab can read it, and environment variables don't change that: the build inlines them into the bundle. So treat a key that ships to the browser as public. It identifies the project and, on a private project, decides which files can be read. It doesn't hide the data the app loads from the people who use the app. - **Browser apps** use a scoped read-only key whose file read patterns (`files.read`) cover only what the app loads. The key new projects get for browser apps reads `dcupl.lc.json`, `models/**`, `data/**`, `transformers/**` and `operators/**`. It leaves out `scripts/**` on purpose: the scripts of your REST APIs and workflows live there. Don't add `scripts/**` to a key that ships in a browser: an app that loads `script` resources needs its own scoped key, with read patterns for just the scripts it loads. The SDK readonly key works too, but it reads every file of the project. See [API keys](/console/api-keys). - **Keys that can write**, the CLI key and scoped keys with file write patterns (`files.write`), stay on the server and in CI secrets. Never put them in a browser bundle or a `NEXT_PUBLIC_` / `VITE_` variable. - **A private project** serves its files only to requests that carry one of the project's keys. On a public project, anyone who knows the URLs can read the files, with or without a key. New projects are private by default and get a scoped read-only key for browser apps. A project created before projects became private by default stays public: private file serving for it is part of a paid plan that dcupl turns on for your project; ask your dcupl contact. See [API keys](/console/api-keys). On a server (Node.js, SSR, Docker), the key stays in the server's environment: ```typescript // Never hardcode API keys const dcupl = new Dcupl({ config: { projectId: process.env.DCUPL_PROJECT_ID!, // From environment apiKey: process.env.DCUPL_API_KEY!, // From environment }, }); ``` ### Secrets storage The CLI key belongs in your CI provider's secret store. The CLI reads it from `DCUPL_API_KEY`: ```yaml filename=".github/workflows/deploy.yml" # GitHub Actions example - name: Deploy env: DCUPL_PROJECT_ID: ${{ secrets.DCUPL_PROJECT_ID }} DCUPL_API_KEY: ${{ secrets.DCUPL_API_KEY }} run: npm run deploy ``` See [CI/CD integration](/cli/ci-cd) for pushing files from CI. ### Content Security Policy The dcupl part of a Content Security Policy: ```typescript // Next.js example const cspHeader = ` default-src 'self'; script-src 'self' 'unsafe-eval'; connect-src 'self' https://cdn.dcupl.com https://api.dcupl.com; `; ``` - `connect-src`: `cdn.dcupl.com` serves your files. `api.dcupl.com` answers the version check the loader makes before loading from the dcupl CDN; it sets the `?v=` parameter that keeps browsers and caches from serving old files. Add the hosts of any other resource URLs in your loader configuration. - `'unsafe-eval'` is only needed when the app runs scripts, custom operators or loader transformers: dcupl compiles them with `new Function`. Leave it out when the app has none, when you turn scripting off with `scripting: { enabled: false }` (the loader then skips those resources with a `ScriptingDisabled` error), or when you pass your own compiler as `scripting.compile` ([Dcupl API](/sdk/api/dcupl)). ## Post-deployment checklist ### Immediate checks - [ ] Application loads without errors - [ ] dcupl initialization succeeds - [ ] `dcupl.quality.errors.summary()` has no loader errors (`type: 'loader'`) - [ ] Queries return expected results - [ ] Health check endpoint responds ### Ongoing monitoring - [ ] Error tracking configured, including console errors ([Production Debugging](/sdk/guides/production-debugging)) - [ ] Alerting for initialization failures - [ ] A data refresh after each publish, for long-running servers ### Rollback plan - [ ] Previous version tagged and deployable - [ ] Rollback procedure documented ## Troubleshooting production issues [Common Production Problems](/sdk/guides/production-debugging#common-production-problems) covers the debug steps and messages for data that doesn't load, slow initialization, memory and stale data. The dcupl-specific checks first: - **Data doesn't load:** check `failedItems` in the loader progress and the `InvalidResource` errors in `summary()`. A `403` on a private project means the key is missing, isn't a key of the project, or doesn't cover that path. - **Old data after a publish:** check that `config.version` names the version you published to (the default is `draft`), and reload the data in a long-running process ([Problem: Stale Data](/sdk/guides/production-debugging#problem-stale-data)). - **Memory grows:** check the number of lists (`dcupl.lists.getAll().length`) and destroy lists you no longer need. ## Next steps - [Production Debugging](/sdk/guides/production-debugging) — Error tracking and monitoring - [Performance Guide](/sdk/advanced/performance) — Optimization strategies - [API keys](/console/api-keys) — Key types and scoped read patterns - [Common mistakes](/sdk/guides/common-mistakes) — Avoid common pitfalls --- # Production debugging and monitoring Source: https://docs.dcupl.com/sdk/guides/production-debugging Description: Set up error tracking, performance monitoring, and alerting for dcupl applications in production # Production debugging and monitoring Monitor dcupl applications in production with error tracking, performance metrics, and data integrity alerts. ## Prerequisites - Working dcupl application deployed to production - Basic understanding of [lists](/sdk/core-concepts/lists) and [data loading](/sdk/core-concepts/data-management) - Error tracking service (Sentry, DataDog, or similar) ## Error tracking with Sentry Integrate Sentry to capture dcupl errors and provide context for debugging. ### Install Sentry ```bash group="sentry-install" tab="npm" npm install @sentry/browser ``` ```bash group="sentry-install" tab="yarn" yarn add @sentry/browser ``` ```bash group="sentry-install" tab="pnpm" pnpm add @sentry/browser ``` ### Initialize Sentry with dcupl context ```typescript filename="monitoring/sentry.ts" import * as Sentry from '@sentry/browser'; import type { Dcupl } from '@dcupl/core'; export function initSentry(dsn: string) { Sentry.init({ dsn, environment: import.meta.env.MODE, // Vite; use process.env.NODE_ENV with other build tools // The core SDK logs only to the console: this reports its errors, including // the cause of a failed init() ('[Dcupl] Initialization failed') integrations: [Sentry.captureConsoleIntegration({ levels: ['error'] })], }); } export function setDcuplContext(dcupl: Dcupl) { // Add dcupl state as context for all errors Sentry.setContext('dcupl', { models: dcupl.models.keys(), listsCount: dcupl.lists.getAll().length, }); } ``` dcupl does not throw for invalid data. Data-quality problems (wrong types, missing required values, unknown references) are collected per model and attribute and read from `dcupl.quality` — see [Track Data Quality Errors](#track-data-quality-errors). ### Capture loader errors Track data loading failures with full context. ```typescript filename="monitoring/loader-tracking.ts" import * as Sentry from '@sentry/browser'; import type { DcuplAppLoader } from '@dcupl/loader'; import type { AppLoaderProgress } from '@dcupl/loader'; export function trackLoaderErrors(loader: DcuplAppLoader) { loader.on((progress: AppLoaderProgress.ProgressItem) => { // Every update carries the whole run so far, so failedItems repeats earlier // failures. Report once, when the run is finished. if (progress.remaining !== 0) return; // Resources that failed to download, already collected for you const failedResources = progress.failedItems; for (const resource of failedResources) { Sentry.captureException(new Error(`Failed to load resource: ${resource.url}`), { tags: { resourceType: resource.type, model: resource.type === 'data' ? resource.model : 'unknown', }, extra: { url: resource.url, rawUrl: resource.rawUrl, httpStatus: resource.status, resourceKey: resource.key, resourceId: resource.id, }, }); } // Alert on high failure rate const failureRate = failedResources.length / progress.totalItems.length; if (failureRate > 0.1) { Sentry.captureMessage('High resource failure rate detected', { level: 'error', extra: { failureRate: `${(failureRate * 100).toFixed(1)}%`, failedCount: failedResources.length, totalCount: progress.totalItems.length, }, }); } }); } ``` ### Detect skipped resources A failed resource is loud — it shows up in `failedItems`. A resource that was never _scheduled_ is silent: if the run filters by `resourceTags` and a resource carries no matching tag, it is simply not part of the run. `totalItems` covers only what was scheduled, so on its own it cannot tell "loaded everything" apart from "loaded everything this application was allowed to see". `skippedItems` closes that gap — every declared resource that dropped out, with the reason — and `run` records what the run filtered by. ```typescript filename="monitoring/skipped-tracking.ts" import * as Sentry from '@sentry/browser'; import type { DcuplAppLoader, AppLoaderProgress } from '@dcupl/loader'; export function trackSkippedResources(loader: DcuplAppLoader, expectedModels: string[]) { loader.on((progress: AppLoaderProgress.ProgressItem) => { if (progress.remaining !== 0) return; const skipped = progress.skippedItems ?? []; // A model you expected in production that no resource in this run provides const scheduledModels = new Set(progress.totalItems.flatMap((r) => (r.type === 'data' ? [r.model] : []))); const missing = expectedModels.filter((m) => !scheduledModels.has(m)); if (missing.length) { Sentry.captureMessage('Expected model not scheduled for this run', { level: 'warning', extra: { missing, run: progress.run, // applicationKey / resourceTags / environmentKeys skipped: skipped.map((s) => ({ url: s.resource.url, model: s.resource.type === 'data' ? s.resource.model : undefined, reason: s.reason, // 'no-tags' | 'tag-mismatch' })), }, }); } }); } ``` `reason` tells you which mistake you made: | Reason | Meaning | Usual cause | | -------------- | -------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------ | | `no-tags` | The run filters by `resourceTags`, and the resource carries no tags at all | A resource added to the config without tagging it — it silently disappears from every tag-filtered run | | `tag-mismatch` | The run filters by `resourceTags`, and none matched the resource's tags | A typo in the tag, or a deploy passing the wrong `resourceTags` | To catch this **before** a deploy rather than from production telemetry, resolve the same plan without loading anything: ```typescript const plan = loader.resources.plan({ resourceTags: ['critical'] }); console.log(plan.skipped); // same SkippedResource[], no I/O ``` See the [DcuplAppLoader API reference](/sdk/api/loader) for the full `ResourcePlan` shape. ### Track data quality errors Monitor data validation errors that occur during initialization and updates. Quality checks are on by default (`new Dcupl({ quality: { enabled: true } })`). - `dcupl.quality.errors.summary()` returns one entry per model, attribute and error type with its count: `{ key, model, attribute, errorGroup, errorType, type, totalCount, storedCount, truncated }`. Loader errors (`type: 'loader'`) are included. - The individual error records live in the built-in `$DcuplErrorTrackingErrors` model; query it like any other model. At most `maxErrorsPerGroup` records (default 1000) are stored per group — `truncated: true` in the summary means the records are a sample, while `totalCount` keeps counting. Failed type coercions are consolidated into one `WrongDataType` record per attribute; when more values of one attribute fail than `maxErrorsPerGroup`, those counts are not exact (see [Limiting Stored Errors](/sdk/advanced/quality-validation#limiting-stored-errors)). - The `$DcuplErrorTrackingErrors` model is rebuilt by `init()` and, after a partial update, whenever an error record changed, so it and `summary()` are current after `dcupl.update()`. - A partial update leaves the same error records and `summary()` counts as a fresh `init()` on the same data: the changed rows are re-validated (`unique` included), the errors of removed rows are gone, and a value of the wrong type in `data.update()` is recorded as `WrongDataType` and dropped. The item then has no value for that attribute; its previous value is not kept. One exception: a keyless row in a `data.upsert()` that a partial update applies is currently dropped without a `MissingKey` record; `init()` and a full rebuild record it. ```typescript filename="monitoring/quality-tracking.ts" import * as Sentry from '@sentry/browser'; import type { Dcupl } from '@dcupl/core'; export function trackQualityErrors(dcupl: Dcupl) { // After init() / update(): one entry per model, attribute and error type const summary = dcupl.quality.errors.summary(); if (summary.length > 0) { const totalErrors = summary.reduce((sum, group) => sum + group.totalCount, 0); // Count errors by type const errorsByType = summary.reduce( (acc, group) => { acc[group.errorType] = (acc[group.errorType] || 0) + group.totalCount; return acc; }, {} as Record, ); // A few individual records; model, errorGroup and errorType are references ({ key }) const sampleErrors = dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [], count: 5 }); Sentry.captureMessage('dcupl quality errors detected', { level: 'warning', extra: { totalErrors, errorsByType, sampleErrors: sampleErrors.map((e) => ({ type: e.errorType?.key, model: e.model?.key, attribute: e.attribute, description: e.description, })), }, }); } } ``` To list the records of one model, filter the error model: `dcupl.query.execute({ modelKey: '$DcuplErrorTrackingErrors', queries: [{ attribute: 'model', operator: 'eq', value: 'Product' }] })`. With `quality: { enabled: false }`, the `$DcuplErrorTrackingErrors` model does not exist and `summary()` lists only loader errors (resources that failed to load, `type: 'loader'`), so it is not empty when a resource failed. ## Performance monitoring Track initialization time, query performance, and resource loading. ### Measure initialization time ```typescript filename="monitoring/performance.ts" import type { Dcupl } from '@dcupl/core'; import type { DcuplAppLoader } from '@dcupl/loader'; interface PerformanceMetrics { configFetch: number; resourceLoading: number; initialization: number; total: number; } export async function measureStartupPerformance(dcupl: Dcupl, loader: DcuplAppLoader): Promise { const metrics: PerformanceMetrics = { configFetch: 0, resourceLoading: 0, initialization: 0, total: 0, }; const totalStart = performance.now(); // Measure config fetch const configStart = performance.now(); await loader.config.fetch(); metrics.configFetch = performance.now() - configStart; // Measure resource loading const loadStart = performance.now(); await loader.process(); metrics.resourceLoading = performance.now() - loadStart; // Measure initialization (init() resolves false instead of throwing) const initStart = performance.now(); if (!(await dcupl.init())) { throw new Error('dcupl initialization failed'); } metrics.initialization = performance.now() - initStart; metrics.total = performance.now() - totalStart; return metrics; } ``` ### Track query performance ```typescript filename="monitoring/query-performance.ts" import type { DcuplList } from '@dcupl/core'; interface QueryMetric { operation: string; duration: number; resultCount: number; modelKey: string; } const queryMetrics: QueryMetric[] = []; export function trackQueryPerformance(list: DcuplList, operation: string, fn: () => T): T { const start = performance.now(); const result = fn(); const duration = performance.now() - start; const metric: QueryMetric = { operation, duration, resultCount: Array.isArray(result) ? result.length : 1, modelKey: list.catalog.fn.metadata().key, }; queryMetrics.push(metric); // Alert on slow queries if (duration > 100) { console.warn(`Slow query detected: ${operation} took ${duration.toFixed(2)}ms`); } return result; } // Usage example function getFilteredProducts(list: DcuplList, category: string) { return trackQueryPerformance(list, 'filterByCategory', () => { // setCondition replaces the previous category condition list.catalog.query.setCondition({ attribute: 'category', operator: 'eq', value: category, }); return list.catalog.query.items(); }); } export function getQueryMetrics(): QueryMetric[] { return [...queryMetrics]; } export function getSlowQueries(thresholdMs = 50): QueryMetric[] { return queryMetrics.filter((m) => m.duration > thresholdMs); } ``` ### Resource loading metrics ```typescript filename="monitoring/resource-metrics.ts" import type { DcuplAppLoader, AppLoaderProgress } from '@dcupl/loader'; interface ResourceMetrics { totalResources: number; loadedResources: number; failedResources: number; loadTime: number; byType: Record; // resources per type } // Call it before loader.process(): it resolves when the next run has finished export function collectResourceMetrics(loader: DcuplAppLoader): Promise { return new Promise((resolve) => { const metrics: ResourceMetrics = { totalResources: 0, loadedResources: 0, failedResources: 0, loadTime: 0, byType: {}, }; const startTime = performance.now(); const unsubscribe = loader.on((progress: AppLoaderProgress.ProgressItem) => { // Every scheduled resource has been processed if (progress.remaining === 0) { metrics.totalResources = progress.totalItems.length; metrics.loadedResources = progress.downloadedItems.length; metrics.failedResources = progress.failedItems.length; metrics.loadTime = performance.now() - startTime; // Group by type for (const resource of progress.totalItems) { const type = resource.type || 'unknown'; metrics.byType[type] = (metrics.byType[type] ?? 0) + 1; } unsubscribe(); resolve(metrics); } }); }); } ``` Start it before the run and await it afterwards: `const metrics = collectResourceMetrics(loader); await loader.process(); console.log(await metrics);`. The progress items carry no byte sizes. For transfer sizes, read the browser's `performance.getEntriesByType('resource')` entries for the resource URLs. ## Logging best practices ### Route dcupl's logs dcupl writes two kinds of logs, and only one of them can be redirected: - **Loader logs** (`[AppLoader] …`, `[CsvParser] …`) go through the `logger` from `@dcupl/common`. Call `logger.configure({ onLog })` **before** you create the loader to send them to your monitoring service ([Logging Configuration](/sdk/guides/error-handling#logging-configuration)). - **Core SDK logs** (`[Dcupl] …`, `[ModelParser] …`, `[Catalog] …`) always go to the console. That includes the only copy of the cause of a failed `init()` or `update()`. Capture console errors to keep them, as the `captureConsoleIntegration` in [Initialize Sentry with dcupl Context](#initialize-sentry-with-dcupl-context) does. ### Structured logger A logger of your own for your app's messages (it doesn't receive dcupl's logs). It's named `appLogger` so it doesn't clash with the `logger` from `@dcupl/common`: ```typescript filename="monitoring/logger.ts" type LogLevel = 'debug' | 'info' | 'warn' | 'error'; interface LogEntry { timestamp: string; level: LogLevel; message: string; context?: Record; error?: Error; } class AppLogger { private logs: LogEntry[] = []; private maxLogs = 1000; private log(level: LogLevel, message: string, context?: Record, error?: Error) { const entry: LogEntry = { timestamp: new Date().toISOString(), level, message, context, error, }; this.logs.push(entry); // Keep log buffer bounded if (this.logs.length > this.maxLogs) { this.logs.shift(); } // Console output in development (import.meta.env is Vite's; use process.env.NODE_ENV elsewhere) if (import.meta.env.DEV) { const logFn = level === 'error' ? console.error : level === 'warn' ? console.warn : console.log; logFn(`[dcupl:${level}]`, message, context || '', error || ''); } // Send critical logs to monitoring service in production if (level === 'error' && import.meta.env.PROD) { this.sendToMonitoring(entry); } } debug(message: string, context?: Record) { this.log('debug', message, context); } info(message: string, context?: Record) { this.log('info', message, context); } warn(message: string, context?: Record) { this.log('warn', message, context); } error(message: string, error?: Error, context?: Record) { this.log('error', message, context, error); } getLogs(level?: LogLevel): LogEntry[] { if (level) { return this.logs.filter((l) => l.level === level); } return [...this.logs]; } private sendToMonitoring(entry: LogEntry) { // Send to your monitoring service // Example: Sentry, DataDog, CloudWatch, etc. } } export const appLogger = new AppLogger(); ``` ### Log dcupl lifecycle events ```typescript filename="app.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; import { appLogger } from './monitoring/logger'; const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); // Track loader progress loader.on((progress) => { appLogger.info('Loader progress', { context: progress.context, processed: progress.processed, total: progress.total, percentage: `${progress.progress}%`, }); // failedItems lists every failure of the run so far: log them once, when the run is finished if (progress.remaining !== 0) return; for (const failure of progress.failedItems) { appLogger.error('Resource load failed', undefined, { url: failure.url, type: failure.type, httpStatus: failure.status, }); } }); async function initialize() { try { appLogger.info('Starting dcupl initialization'); await loader.config.fetch(); appLogger.info('Config fetched', { resourceCount: loader.resources.getAll().length, }); await loader.process(); appLogger.info('Resources processed'); // init() doesn't throw: it resolves false when initialization failed if (!(await dcupl.init())) { throw new Error('dcupl initialization failed'); } appLogger.info('dcupl initialized', { models: dcupl.models.keys(), }); } catch (error) { appLogger.error('Initialization failed', error as Error); throw error; } } ``` ## Data integrity alerts Monitor data quality and alert on integrity issues. ### Data validation service ```typescript filename="monitoring/data-validation.ts" import type { Dcupl } from '@dcupl/core'; import { appLogger } from './logger'; interface ValidationRule { modelKey: string; check: (dcupl: Dcupl) => ValidationResult; } interface ValidationResult { passed: boolean; message: string; details?: Record; } const validationRules: ValidationRule[] = []; export function addValidationRule(rule: ValidationRule) { validationRules.push(rule); } export function runValidations(dcupl: Dcupl): ValidationResult[] { const results: ValidationResult[] = []; for (const rule of validationRules) { try { const result = rule.check(dcupl); results.push(result); if (!result.passed) { appLogger.warn('Validation failed', { model: rule.modelKey, message: result.message, details: result.details, }); } } catch (error) { results.push({ passed: false, message: `Validation error: ${(error as Error).message}`, }); } } return results; } // Example validation rules addValidationRule({ modelKey: 'Product', check: (dcupl) => { const list = dcupl.lists.create({ modelKey: 'Product' }); const count = list.catalog.query.count(); list.destroy(); if (count === 0) { return { passed: false, message: 'No products loaded', details: { expectedMinimum: 1, actual: 0 }, }; } return { passed: true, message: 'Products loaded successfully' }; }, }); addValidationRule({ modelKey: 'Product', check: (dcupl) => { const list = dcupl.lists.create({ modelKey: 'Product' }); // Check for products without prices (price undefined or null) list.catalog.query.addCondition({ attribute: 'price', operator: 'typeof', value: 'number', options: { invert: true }, }); const missingPrices = list.catalog.query.count(); list.destroy(); if (missingPrices > 0) { return { passed: false, message: 'Products with missing prices detected', details: { count: missingPrices }, }; } return { passed: true, message: 'All products have prices' }; }, }); ``` ### Scheduled health checks ```typescript filename="monitoring/health-check.ts" import type { Dcupl } from '@dcupl/core'; import { runValidations } from './data-validation'; import { appLogger } from './logger'; interface HealthStatus { status: 'healthy' | 'degraded' | 'unhealthy'; checks: { name: string; passed: boolean; message: string; }[]; timestamp: string; } export function checkHealth(dcupl: Dcupl): HealthStatus { const checks: HealthStatus['checks'] = []; // Check if dcupl is initialized try { const modelCount = dcupl.models.keys().length; checks.push({ name: 'initialization', passed: modelCount > 0, message: modelCount > 0 ? `${modelCount} models loaded` : 'No models loaded', }); } catch (error) { checks.push({ name: 'initialization', passed: false, message: (error as Error).message, }); } // Run data validations const validationResults = runValidations(dcupl); for (const result of validationResults) { checks.push({ name: 'data-validation', passed: result.passed, message: result.message, }); } // Check quality errors (with quality disabled, summary() still lists loader errors such as InvalidResource) const criticalErrors = dcupl.quality.errors .summary() .filter((group) => group.errorType === 'MissingModel' || group.errorType === 'InvalidResource') .reduce((sum, group) => sum + group.totalCount, 0); checks.push({ name: 'quality', passed: criticalErrors === 0, message: criticalErrors === 0 ? 'No critical quality errors' : `${criticalErrors} critical errors`, }); // Determine overall status const failedChecks = checks.filter((c) => !c.passed); let status: HealthStatus['status'] = 'healthy'; if (failedChecks.length > 0) { status = failedChecks.some((c) => c.name === 'initialization') ? 'unhealthy' : 'degraded'; } const healthStatus: HealthStatus = { status, checks, timestamp: new Date().toISOString(), }; // Log health status if (status !== 'healthy') { appLogger.warn('Health check failed', { status: healthStatus }); } return healthStatus; } ``` ## Debugging production issues Techniques for diagnosing problems in production. ### Export debug information ```typescript filename="monitoring/debug-export.ts" import type { Dcupl } from '@dcupl/core'; import { appLogger } from './logger'; interface DebugSnapshot { timestamp: string; models: { key: string; itemCount: number; properties: string[]; }[]; lists: { modelKey: string; currentSize: number; initialSize: number; }[]; qualityErrors: { type: string; model: string; count: number; }[]; recentLogs: unknown[]; } export function createDebugSnapshot(dcupl: Dcupl): DebugSnapshot { const snapshot: DebugSnapshot = { timestamp: new Date().toISOString(), models: [], lists: [], qualityErrors: [], recentLogs: appLogger.getLogs().slice(-100), }; // Collect model info for (const modelKey of dcupl.models.keys()) { const model = dcupl.models.get(modelKey); if (model) { const list = dcupl.lists.create({ modelKey }); snapshot.models.push({ key: modelKey, itemCount: list.catalog.query.count(), properties: model.properties?.map((p) => p.key) || [], }); list.destroy(); } } // Collect list info for (const list of dcupl.lists.getAll()) { const meta = list.catalog.fn.metadata(); snapshot.lists.push({ modelKey: meta.key, currentSize: meta.currentSize, initialSize: meta.initialSize, }); } // Collect quality errors grouped by type and model const errorMap = new Map(); for (const group of dcupl.quality.errors.summary()) { const key = `${group.errorType}:${group.model ?? 'unknown'}`; errorMap.set(key, (errorMap.get(key) || 0) + group.totalCount); } for (const [key, count] of errorMap) { const [type, model] = key.split(':'); snapshot.qualityErrors.push({ type, model, count }); } return snapshot; } // Expose for browser console debugging if (typeof window !== 'undefined') { (window as any).__dcuplDebug = { createSnapshot: createDebugSnapshot, getLogs: () => appLogger.getLogs(), }; } ``` ### Remote debugging with Console Enable dcupl Console connection only when needed. ```typescript filename="debug/remote-debug.ts" import type { Dcupl } from '@dcupl/core'; import { DcuplConnect } from '@dcupl/connect'; let connectInstance: DcuplConnect | null = null; export function enableRemoteDebug(dcupl: Dcupl) { if (connectInstance) { console.warn('Remote debug already enabled'); return; } connectInstance = new DcuplConnect({ dcuplInstance: dcupl }); // Not awaited: when a page other than the Console opened the app, init() never settles void connectInstance.init(); // Connects only when the app window was opened from dcupl Connect in the Console console.log('Remote debug enabled - connect via dcupl Console'); } export function disableRemoteDebug() { if (connectInstance) { connectInstance.destroy(); // a no-op when init() never connected connectInstance = null; console.log('Remote debug disabled'); } } // Expose to browser console for on-demand debugging if (typeof window !== 'undefined') { (window as any).__dcuplEnableDebug = enableRemoteDebug; (window as any).__dcuplDisableDebug = disableRemoteDebug; } ``` See [Debugging with Console](/sdk/guides/debugging#connect-your-app) for how the Console opens and connects your app. ## Common production problems ### Problem: data not loading **Symptoms:** Empty results, missing models **Debug steps:** ```typescript // 1. Check loader progress loader.on((progress) => { console.log('Scheduled:', progress.totalItems); console.log('Failed:', progress.failedItems); // Declared in the config but never scheduled for this run console.log('Skipped:', progress.skippedItems); console.log('Ran as:', progress.run); }); // 2. Check registered models console.log('Models:', dcupl.models.keys()); // 3. Check quality errors console.log('Errors:', dcupl.quality.errors.summary()); ``` **What the messages mean:** - **`loader.config.fetch()` throws `Invalid Loader Configuration`**, logged as `[AppLoader] Could not fetch loader configuration. Are you sure your projectId + apiKey are correct?`: a wrong `projectId` or `apiKey`, a `version` that doesn't exist, or a network error. With a non-dcupl `baseUrl` the log line ends after "configuration". - **A model or data file is missing:** `[AppLoader] Failed to load data resource` with `{ url, model }`, or `[AppLoader] Failed to load model resource` with `{ url }`, and an `InvalidResource` error in `summary()` with `type: 'loader'`. The file doesn't exist at that URL, or it was refused (`status` in `failedItems`, e.g. `403` on a private project when the key is missing or doesn't cover the path). - **A model has no items although `init()` was `true`:** if the log shows `[ModelParser] Model not found during data processing`, data was staged with `data.set()` for a model key that isn't registered, or a data resource's `model` isn't declared while missing-model handling is off (`missingModelHandling.autoGenerateProperties` is `false` or not set inside a `missingModelHandling` object). With the default options there is no log line: the loader creates a model under the data resource's `model` key and loads the rows there, so check `dcupl.models.keys()` for a key you didn't expect. **Common causes:** - Network errors (CORS, DNS, SSL) - Invalid API key or project ID - CDN version mismatch - Malformed model/data files - The resource was never scheduled — check `skippedItems` for a `no-tags` / `tag-mismatch` reason before assuming a network failure (see [Detect Skipped Resources](#detect-skipped-resources)) - An unresolved `${variable}` in the url — an unknown variable is substituted with an empty string, so a broken config looks like a missing file. Check `unresolvedVariables` on the scheduled item. ### Problem: slow initialization **Symptoms:** Long startup time, timeout errors **Debug steps:** ```typescript // Measure each phase const metrics = await measureStartupPerformance(dcupl, loader); console.log('Config fetch:', metrics.configFetch, 'ms'); console.log('Resource loading:', metrics.resourceLoading, 'ms'); console.log('Initialization:', metrics.initialization, 'ms'); ``` **Common causes:** - Too many resources loaded at once - Large data files without compression - Slow CDN response times - Complex model relationships **Solutions:** - Use resource tags to load only needed data - Enable gzip/brotli compression - Use a CDN closer to users - Consider lazy loading for large datasets ### Problem: memory issues **Symptoms:** Browser crashes, slow performance over time **Debug steps:** ```typescript // Check list count console.log('Active lists:', dcupl.lists.getAll().length); // Check data size per model for (const modelKey of dcupl.models.keys()) { const list = dcupl.lists.create({ modelKey }); console.log(`${modelKey}: ${list.catalog.query.count()} items`); list.destroy(); } ``` **Common causes:** - Lists not destroyed after use - Loading more data than needed - Unused models consuming memory **Solutions:** - Always call `list.destroy()` when done - Use resource tags to limit loaded data - Implement pagination for large result sets ### Problem: stale data **Symptoms:** Old data displayed, updates not reflected **Debug steps:** ```typescript // Check CDN version console.log('Variables:', loader.variables.getAll()); ``` **Refresh the data:** fetch the configuration and run `loader.process()` again, then apply the result with `dcupl.update()`. A plain `process()` reloads every resource; to skip the model files, reload only the data resources, for example by tagging them (here `data`): ```typescript await loader.config.fetch(); // Picks up a new configuration and renews the ?v= version parameter of CDN urls await loader.process({ resourceTags: ['data'] }); // Only the resources tagged 'data' await dcupl.update(); ``` Data resources replace their model's rows by default (`updateType: 'set'`), so the refreshed rows take the place of the old ones. A re-process also rebuilds the models it loads from their current files and drops the resources that left the configuration, so changed models need no new `Dcupl` instance, and the lists your app already holds serve the new rows ([Processing](/sdk/api/loader#processing)). Re-process the loader you have: a second loader for the same models rejects with `SharedModelKeyError`. Without a loader, fetch the data yourself and stage it with `dcupl.data.set(items, { model })` (or `upsert`) before `await dcupl.update()`. **Common causes:** - An app or process that stays open and doesn't reload the data after a publish - `config.version` names a different version than the one you published to (the default is `draft`) - Files on your own host served with long cache headers **Solutions:** - Reload the data after each publish as shown above: `loader.config.fetch()` renews the `?v=` version parameter of dcupl CDN URLs, so browsers and caches fetch the new files - Set `config.version` to the version you publish to - For files on your own host, set short cache headers or put a version in their URLs ## Complete monitoring setup ```typescript filename="monitoring/index.ts" import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; import { initSentry, setDcuplContext } from './sentry'; import { trackLoaderErrors } from './loader-tracking'; import { trackQualityErrors } from './quality-tracking'; import { measureStartupPerformance } from './performance'; import { checkHealth } from './health-check'; import { appLogger } from './logger'; export async function initializeWithMonitoring(config: { projectId: string; apiKey: string; sentryDsn?: string }) { // Initialize Sentry if (config.sentryDsn) { initSentry(config.sentryDsn); } const dcupl = new Dcupl(); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader, 'main'); // Set up loader variables loader.variables.global.set('projectId', config.projectId); loader.variables.global.set('apiKey', config.apiKey); // Track loader errors trackLoaderErrors(loader); try { // Measure and log performance const metrics = await measureStartupPerformance(dcupl, loader); appLogger.info('Startup complete', { configFetch: `${metrics.configFetch.toFixed(0)}ms`, resourceLoading: `${metrics.resourceLoading.toFixed(0)}ms`, initialization: `${metrics.initialization.toFixed(0)}ms`, total: `${metrics.total.toFixed(0)}ms`, }); // Set Sentry context setDcuplContext(dcupl); // Track quality errors trackQualityErrors(dcupl); // Run health check const health = checkHealth(dcupl); if (health.status !== 'healthy') { appLogger.warn('Application started in degraded state', { health }); } return { dcupl, loader, health }; } catch (error) { appLogger.error('Initialization failed', error as Error); throw error; } } ``` ## Next steps - [Debugging with Console](/sdk/guides/debugging) — Live inspection with dcupl Console - [Testing dcupl Applications](/sdk/guides/testing) — Write tests for your dcupl code - [Common mistakes](/sdk/guides/common-mistakes) — Avoid common pitfalls --- # Migration guide Source: https://docs.dcupl.com/sdk/guides/migration Description: Upgrade from SDK 1.x to 2.0 — the behaviour changes, and the query API method names introduced with 2.0 # Migration guide This guide helps you upgrade from dcupl SDK 1.x (1.11.x) to 2.0. 2.0 keeps every 1.x method, but some behaviour changed: check [Behaviour Changes from 1.x](#behaviour-changes-from-1x) first. The new query API method names are optional; the older names still work in 2.0. ## Behaviour changes from 1.x Upgrade `@dcupl/core`, `@dcupl/common` and `@dcupl/loader` together, to the same 2.0 version. If your app uses dcupl Connect ([Debugging](/sdk/guides/debugging)), upgrade `@dcupl/connect` with them: its 1.x release depends on `@dcupl/core` 1.11. ```bash npm install @dcupl/core @dcupl/common @dcupl/loader npm install @dcupl/connect # only with dcupl Connect ``` Then check your code for these changes (compared with 1.11.11): **Data** - `data.update()` and `data.upsert()` replace every attribute in the payload as a whole. 1.x deep-merged arrays and `json` objects: `tags: ['x', 'y']` updated with `['z']` became `['z', 'y']`, now it is `['z']`. To change one key of a `json` attribute, send the whole object ([`update()`](/sdk/api/data#update)). - `data.reset({ model })` is staged like the other data methods: the rows are removed by the next `await dcupl.update()`. 1.x cleared the model at once, and the next `update()` brought the rows back. - `autoGenerateProperties` infers `int`, `float`, `boolean` and `date` (strict ISO 8601) from string values, for example CSV columns. 1.x typed every string column as `string`. Declare the property's type in the model to keep it a `string`. - `boolean` properties accept `'true'`, `'false'`, `'1'` and `'0'` (any case) and the numbers `1` and `0`, so `'false'` and `'0'` now load as `false` (1.x read every non-empty string as `true`). Any other string, number, object or array is a `WrongDataType` error and stays undefined. CSV columns follow the same rule, except that an empty cell counts as no value, not as `WrongDataType`. - Rows with the same key in one `data.set()` payload or data file: the last row wins as a whole, with one `NonUniqueKey` error per replaced row. 1.x merged the later rows into the first. - Keys compare as strings: `data.update([{ key: '1', … }])` changes a row loaded with `key: 1` (1.x ignored the update). A row without a key value (no `key`, or no value for a custom `keyProperty`) is dropped and reported as `MissingKey`; 1.x stored it under the key `'undefined'` (`'null'` for an explicit `null`). See [Rows without a key value](#rows-without-a-key-value). - An explicit `null` on a reference stays `null` (an `eq null` condition matches it) and is reported as `NullValue` unless the reference sets `quality: { nullable: true }`. 1.x turned it into `undefined`. See [Error Types](/sdk/advanced/quality-validation#error-types). - A `dcupl.update()` after `init()` that takes the partial path gives the same result as a full rebuild: derived properties and references (also those on resolved references) are recomputed instead of keeping their old values, an update that can change a query reference's result runs as a full update, and the data quality records equal those of an `init()` on the same data ([Partial updates](/sdk/api/data#partial-updates)). - A list created with `listItemKeys` or `query` stays in sync with the data: after every `dcupl.update()`, partial or full, it serves the current values of its items and drops the removed ones. Items that start matching a seed `query` still join only with `list.update()` or auto-update ([create()](/sdk/api/lists#create)). **Queries** - `invert` applies to every operator. 1.x ignored it on `gt`, `gte`, `lt`, `lte`, `size` and `isTruthy`. - Conditions on `Array<…>` properties match without an index. 1.x returned `[]` until a facet had built the attribute's index. - Dotted paths through references (`customer.country`) are new in 2.0 ([Deep queries](/sdk/queries/query-deep)); in 1.x such a condition or facet matched nothing. - A group without `groupKey` added in `add` mode (`addGroup()`, `apply(group)`) is added as its own group. 1.x ignored such a group (one with `groupKey: undefined` written out was merged into the first such group, overwriting its `groupType`). - A nested group without `groupType` is evaluated as `'and'`. 1.x could return the matches of a later condition when an earlier one matched nothing. A group you pass to `addGroup()`, `setGroup()` or `apply(group)` without `groupType` stays an `'or'` group, as in 1.x. - The `in` operator is new; in 1.x an `in` condition matched nothing ([Query Operators](/sdk/api/query-operators)). - A condition whose `operator` is neither built in nor registered as an [operator script](/sdk/guides/scripts) throws an `UnknownOperator` error ([Custom operators](/sdk/api/query-operators#custom-operators)). 1.x matched nothing, silently. - A sort the SDK can't apply is ignored with a warning: an `order` value other than `'ASC'` or `'DESC'`, an `order` without exactly one entry per attribute, or a sort script key with no registered script. 1.x ignored such a sort silently ([sort scripts](/sdk/guides/scripts#sort--reorder-query-results)). - `aggregate()` `count` is the number of values the attribute holds (0 when no item has one): each value of an `Array<…>` property or of a multi-valued dotted path counts once, and on a number attribute it is the count `avg` divides by. 1.x returned the number of items in the list, or `undefined` when no item had a numeric value, so `count` on a string or date attribute was always `undefined`. `min` and `max` skip items without a value (1.x could return `null` as the minimum). `sum` and `avg` are unchanged. **Models** - `models.set()` replaces the whole definition registered under the key, as `views.set()` already did in 1.x; `models.update()` merges into it ([`set()`](/sdk/api/models#set), [`update()`](/sdk/api/models#update)). In 1.x a second `models.set()` for the same key added its properties to the first one; now the first part is gone. The app loader still merges model files with the same key. - A `models.set()` or `models.update()` after `init()` takes effect on the next `dcupl.update()`, which rebuilds all data like `init()` (a full update, also when only one row changed). Lists created before the change keep working and serve the new definition. - `models.get()` and `views.get()` return a copy, and `models.set()` and `views.set()` store one, so changing either object no longer changes the registered definition (1.x handed out the registered definition). - One app loader per model key: a loader whose `process()` would load a model key that another loader of the instance already loads rejects with `SharedModelKeyError` (exported from `@dcupl/loader`). `dcupl.loaders.remove(key)` removes the models, views and rows that loader loaded on the next `update()` ([Processing](/sdk/api/loader#processing)). **Events and data quality** - `dcupl.update()` emits `result_updated` on every list whose result can have changed, and the global `dcupl_partial_update` event when it applies staged data changes or a model definition change (`models.set()`, `models.update()` or `models.remove()` after `init()`, also with no data staged; see [Events](/sdk/api/events)). 1.x emitted neither on a data update. `dcupl_handle_datacontainer` now fires when `update()` rebuilds everything; a partial update emits it, with just the `$DcuplErrorTrackingErrors` and `$DcuplErrorTrackingSummary` containers, whenever the model has recorded data quality errors, whether or not they changed. While no list over those two models exists, they are built at the first read instead of during `init()`/`update()`, and the event for that build fires then ([Quality reports](/sdk/advanced/quality-validation#quality-reports)). - `dcupl.quality.errors.summary()` is new: error totals per model, attribute and error type. - A CSV column that the model doesn't declare is reported as an `UnknownColumn` error (1.x dropped it without an error). A column whose values all fail type conversion is reported as `WrongDataType` (1.x: `UndefinedAttribute`). - A resource that fails to load is recorded as a loader error even with `quality: { enabled: false }`. **Packages** - The packages are native ES modules (`"type": "module"`), so `import { Dcupl } from '@dcupl/core'` works in Node ESM. 1.x failed there with `ERR_UNSUPPORTED_DIR_IMPORT`; `require()` still works. - The browser script builds are gone: `@dcupl/core` and `@dcupl/loader` no longer ship `dist/browser/*.umd.js`. For a ` ``` Create the filter sidebar component: ```vue filename="src/components/FilterSidebar.vue" ``` `FilterSidebar` doesn't change its `filters` prop: its inputs show the current filters and emit a `change` event with the fields that changed. `ProductBrowser` passes those changes to `updateFilters()` of `useProductList`, whose watcher then applies them to the query. A cleared price input becomes `null`, so no bound is applied. ## Step 6: build the main app component Update the main App component: ```vue filename="src/App.vue" ``` Create the ProductBrowser component: ```vue filename="src/components/ProductBrowser.vue" ``` ## Step 7: add real-time updates Create an admin panel component to simulate real-time changes: ```vue filename="src/components/AdminPanel.vue" ``` Render the panel in `src/components/ProductBrowser.vue`: import it next to the other components, and add it at the end of the `product-grid-section`: ```vue title="src/components/ProductBrowser.vue: add the AdminPanel" ``` You don't need to wire anything else: `await dcupl.update()` emits `result_updated` on the product list, the `onResultChange` subscription in `useProductList` re-reads products, facets and price statistics, and the grid re-renders. ## Step 8: run the application Start the development server: ```bash filename="terminal" npm run dev ``` Open your browser to `http://localhost:5173` to see the product browser. ## What you learned In this tutorial, you learned how to: 1. Set up dcupl in a Vue 3 application with Vite 2. Create a singleton dcupl instance for shared state 3. Build Vue composables for dcupl initialization (`useDcupl`) 4. Build reactive composables for filtered data (`useProductList`) 5. Use Vue's `watch` to react to filter changes 6. Re-read when a list's result changes (`onResultChange`) for real-time updates, with proper cleanup 7. Create filter components that work with facets, with props down and events up 8. Handle loading and error states properly 9. Implement real-time data modifications with visual feedback ## Key patterns **Singleton dcupl Instance** ```typescript // instance.ts export const dcupl = new Dcupl(); // Use the same instance everywhere ``` **Initialization Composable** ```typescript const { dcupl, isInitialized, isLoading, error } = useDcupl(); ``` **Reactive Filters with watch** ```typescript const filters = reactive({ category: '', ... }); // A watcher on a reactive object is deep watch(filters, applyFilters); ``` **Result Subscription with Cleanup** ```typescript onMounted(() => { productList = dcupl.lists.create({ modelKey: 'Product' }); // Fires after query changes and after `await dcupl.update()`; returns the unsubscribe function unsubscribe = productList.onResultChange(readResults); }); onUnmounted(() => { unsubscribe?.(); productList?.destroy(); }); ``` **Props Down, Events Up** ```vue ``` ## Query operators used The filters above use `eq`, `gte`, `lte` and `find` (a `/regex/` value with `regexFlags`); every operator and its options is in the [query operators reference](/sdk/api/query-operators). ## Next steps - [Building with React](/learn/tutorials/react-integration) — Compare with React patterns - [Real-time data updates](/learn/tutorials/real-time-updates) — Deep dive into update patterns - [Build a Product Catalog](/learn/tutorials/product-catalog) — Faceted navigation in depth - [Lists API](/sdk/api/lists) — Typed lists, onResultChange() and destroy() - [Query operators reference](/sdk/api/query-operators) — Every operator, including find with regexFlags - [Real-time updates](/sdk/advanced/realtime-updates) — Data changes and list events in the SDK ## Common issues **dcupl not initialized when component renders** Always check `isInitialized` before rendering components that use dcupl: ```vue ``` **Stale data after updates** Data calls (`data.set`, `upsert`, `update`, `remove`) are staged until `await dcupl.update()`. Re-read the list when its result changes: ```typescript const unsubscribe = list.onResultChange(() => readResults()); // call unsubscribe() in onUnmounted ``` **Memory leaks from subscriptions** Always clean up subscriptions and lists in `onUnmounted`: ```typescript let unsubscribe: (() => void) | null = null; onMounted(() => { unsubscribe = productList.onResultChange(readResults); }); onUnmounted(() => { unsubscribe?.(); productList?.destroy(); }); ``` **Filters not reactive** Keep the filters in one `reactive` object and watch the object itself. A watcher on a reactive object is deep, so it fires when any field changes: ```typescript const filters = reactive({ ... }); watch(filters, applyFilters); ``` --- # Best practices Source: https://docs.dcupl.com/learn/best-practices Description: Patterns and recommendations for building production-ready dcupl applications. These guides cover data modeling, performance optimization, and error handling… # Best practices Patterns and recommendations for building production-ready dcupl applications. These guides cover data modeling, performance optimization, and error handling strategies. - [Data modeling](/learn/best-practices/data-modeling) — Structure your data models effectively with proper types, references, and indexing strategies. - [Performance](/learn/best-practices/performance) — Optimize queries, updates, and memory usage for fast, efficient applications. - [Error handling](/learn/best-practices/error-handling) — Handle errors gracefully with retry logic, fallbacks, and recovery strategies. ## Quick tips > **Note:** > > **Initialize Once.** Call `init()` once at startup, then stage changes with `data.update()` and apply them with `dcupl.update()`. Re-initialization is expensive. > **Note:** > > **Use Specific Types.** Define properties with specific types like `int`, `float`, `boolean` instead of storing everything as `string`. > **Note:** > > **Reuse Lists.** Create lists once and reuse them with `query.clear()`. Creating new lists has overhead. > **Note:** > > **Limit Reference Depth.** Keep reference chains to 2-3 levels. Use derived properties for frequently-accessed deep data. ## Related - [Performance Guide](/sdk/advanced/performance) — Advanced performance optimization - [Caching](/sdk/advanced/caching) — Caching strategies - [Quality Validation](/sdk/advanced/quality-validation) — Data validation and quality checks - [Common mistakes](/sdk/guides/common-mistakes) — Avoid typical pitfalls --- # Data modeling best practices Source: https://docs.dcupl.com/learn/best-practices/data-modeling Description: How to structure your data models effectively for performance and maintainability # Data modeling best practices Effective data modeling is the foundation of a performant dcupl application. This guide covers patterns and recommendations for structuring your models, properties, and relationships. ## Core principles Good data models share these characteristics: - **Explicit structure** - Every property has a defined type - **Minimal duplication** - Use references instead of copying data - **Query-optimized** - Properties used in filters are indexed - **Right-sized** - Only include properties you actually need ## Property design ### Choose specific types Use the most specific type for each property. This enables better validation and optimization. ```typescript group="types" tab="Good" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'quantity', type: 'int' }, { key: 'isActive', type: 'boolean' }, { key: 'tags', type: 'Array' }, { key: 'createdAt', type: 'date' }, ], }; ``` ```typescript group="types" tab="Avoid" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'string' }, // Should be 'float' { key: 'quantity', type: 'string' }, // Should be 'int' { key: 'isActive', type: 'string' }, // Should be 'boolean' { key: 'tags', type: 'string' }, // Should be 'Array' { key: 'createdAt', type: 'string' }, // Should be 'date' ], }; ``` **Available types:** `string`, `int`, `float`, `boolean`, `date`, `Array`, `Array`, `Array`, `Array`, `json`, `any` ### Index properties for filtering Properties you filter on should be indexed for fast queries. dcupl always indexes references and builds a property's index on demand (for example the first time you request facets for it); until then, filters on that property scan the items. Set `index: true` to build the index while the data is processed, so `eq` and `in` filters on frequently looked-up keys use it from the first query: ```typescript filename="indexed-properties.ts" const productModel: ModelDefinition = { key: 'Product', properties: [ // Standard properties (indexed on demand, e.g. for facets) { key: 'name', type: 'string' }, { key: 'category', type: 'string' }, { key: 'price', type: 'float' }, // Explicit index for fast key lookups { key: 'sku', type: 'string', index: true }, // Display-only (no filtering needed) { key: 'description', type: 'string' }, ], }; ``` `filter: true` on a property doesn't index anything: it only selects the attributes that [`catalog.filters`](/sdk/api/catalog#catalogfilters) returns for building filter UIs. ### Only include what you need Every property increases memory usage and processing time. Remove properties you do not query or display. ```typescript group="lean" tab="Good" // Lean model - only what the UI needs const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string' }, { key: 'imageUrl', type: 'string' }, ], }; ``` ```typescript group="lean" tab="Avoid" // Bloated model - includes unused fields const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'category', type: 'string' }, { key: 'imageUrl', type: 'string' }, { key: 'legacyId', type: 'string' }, // Never used { key: 'internalNotes', type: 'string' }, // Never used { key: 'importTimestamp', type: 'string' }, // Never used { key: 'rawData', type: 'json' }, // Never used ], }; ``` ### Mark optional properties Every declared property and reference is required by default. An item without a value for one of them records an `UndefinedValue` quality error (a single `UndefinedAttribute` when no item has the attribute at all). `init()` still resolves `true`, the item loads without the value and nothing is logged, so read `dcupl.quality.errors.summary()` during development. Set `quality: { required: false }` on attributes that are missing on some items (a value sent as `null` is a separate `NullValue` error, which `quality: { nullable: true }` allows): ```typescript filename="optional-properties.ts" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, // Only products on sale have a sale price { key: 'salePrice', type: 'float', quality: { required: false } }, ], }; ``` To make most attributes of a model optional, set `quality: { attributes: { required: false } }` on the model and mark the must-have ones `quality: { required: true }`. References take `required` too: the root of a category tree has no parent. See [Model-wide Defaults](/sdk/advanced/quality-validation#model-wide-defaults). ## Reference design ### Normalize related data Use references instead of duplicating data across models. This keeps data consistent and reduces memory usage. ```typescript group="normalize" tab="Good" // Normalized: Customer data lives in one place const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderNumber', type: 'string' }, { key: 'total', type: 'float' }, { key: 'status', type: 'string' }, ], references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }], }; const customerModel: ModelDefinition = { key: 'Customer', properties: [ { key: 'name', type: 'string' }, { key: 'email', type: 'string' }, { key: 'phone', type: 'string' }, ], }; ``` ```typescript group="normalize" tab="Avoid" // Denormalized: Customer data duplicated in every order const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderNumber', type: 'string' }, { key: 'total', type: 'float' }, { key: 'status', type: 'string' }, { key: 'customerName', type: 'string' }, // Duplicated { key: 'customerEmail', type: 'string' }, // Duplicated { key: 'customerPhone', type: 'string' }, // Duplicated ], }; ``` ### Use correct reference types Choose the right reference type based on the relationship: | Relationship | Reference Type | Example | | ------------ | ------------------------------------------------------------------------------------------------- | ------------------------- | | Many-to-one | `singleValued` | Order has one Customer | | One-to-many | `multiValued`, or a `singleValued` reference on the child plus a resolved reference on the parent | Order has many OrderItems | | Many-to-many | `multiValued` | Product has many Tags | ```typescript filename="reference-types.ts" const productModel: ModelDefinition = { key: 'Product', properties: [{ key: 'name', type: 'string' }], references: [ // Many products belong to one category { key: 'category', model: 'Category', type: 'singleValued' }, // One product has many tags { key: 'tags', model: 'Tag', type: 'multiValued' }, ], }; ``` ### Limit reference depth Keep reference chains to 2-3 levels. Deep chains slow down queries significantly. ```typescript group="depth" tab="Good" // 2-level depth - fast queries orderList.catalog.query.addCondition({ attribute: 'customer.country', operator: 'eq', value: 'USA', }); ``` ```typescript group="depth" tab="Avoid" // 4-level depth - very slow orderList.catalog.query.addCondition({ attribute: 'customer.address.country.region', operator: 'eq', value: 'West Coast', }); ``` Each level resolves one more reference for every item the condition evaluates, so the cost grows with the depth and with the number of items. How much depends on your data and on whether the list has already resolved the path, so measure with your own data (see [Profiling and Debugging](/sdk/advanced/performance#profiling-and-debugging)). ### Use derived properties for deep queries For frequently-accessed deep data, use derived properties to flatten the access path: ```typescript filename="derived-property.ts" const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderNumber', type: 'string' }, { key: 'total', type: 'float' }, // Derived property for fast queries { key: 'customerCountry', type: 'string', derive: { localReference: 'customer', remoteProperty: 'country', }, }, ], references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }], }; // Fast: Query derived property directly orderList.catalog.query.addCondition({ attribute: 'customerCountry', operator: 'eq', value: 'USA', }); ``` ### Avoid circular references Circular references between models are detected and rejected. Design one-way relationships and query in reverse when needed. A reference from a model to itself is not a cycle: a Category with a `parent` reference to Category is fine. dcupl resolves such references against all rows of the model, whatever the row order, so hierarchies work with deep queries (`parent.name`) and facets. ```typescript group="circular" tab="Good" // One-way reference: Order -> Customer const orderModel: ModelDefinition = { key: 'Order', properties: [{ key: 'orderNumber', type: 'string' }], references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }], }; const customerModel: ModelDefinition = { key: 'Customer', properties: [{ key: 'name', type: 'string' }], // No reference back to Order }; // Query orders for a customer (reverse direction) orderList.catalog.query.addCondition({ attribute: 'customer', operator: 'eq', value: 'customer-123', }); ``` ```typescript group="circular" tab="Avoid" // Circular reference: Order -> Customer -> Order const orderModel: ModelDefinition = { key: 'Order', references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }], }; const customerModel: ModelDefinition = { key: 'Customer', references: [ { key: 'orders', model: 'Order', type: 'multiValued' }, // Circular! ], }; // await dcupl.init() logs "Circular dependency detected: Order → Customer → Order" // and resolves false ``` If you need the orders on each customer, declare a resolved reference instead. It collects the items that point at this one and is not part of the dependency check, so it doesn't create a cycle: ```typescript filename="resolved-reference.ts" const customerModel: ModelDefinition = { key: 'Customer', properties: [{ key: 'name', type: 'string' }], references: [ // Every Order whose `customer` reference points at this customer { key: 'orders', model: 'Order', type: 'multiValued', resolve: { model: 'Order', reference: 'customer' } }, ], }; ``` See [Models API](/sdk/api/models) for resolved and derived references. ## Model organization ### One model per entity Each distinct entity should have its own model. Do not combine unrelated data. ```typescript group="entities" tab="Good" // Separate models for distinct entities const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, ], }; const categoryModel: ModelDefinition = { key: 'Category', properties: [ { key: 'name', type: 'string' }, { key: 'slug', type: 'string' }, ], }; ``` ```typescript group="entities" tab="Avoid" // Generic model for everything const entityModel: ModelDefinition = { key: 'Entity', properties: [ { key: 'type', type: 'string' }, // 'product' or 'category' { key: 'data', type: 'json' }, // Unstructured blob ], }; ``` ### Consistent key naming Use consistent naming conventions across all models: ```typescript filename="naming.ts" // Recommended: camelCase for properties, PascalCase for models const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'productName', type: 'string' }, { key: 'listPrice', type: 'float' }, { key: 'isAvailable', type: 'boolean' }, { key: 'createdAt', type: 'date' }, ], references: [{ key: 'parentCategory', model: 'Category', type: 'singleValued' }], }; ``` ## Data key strategy Every item needs a unique `key`. Choose a strategy that fits your data. If the identifier is in another column, set `keyProperty` on the model instead of copying it into `key`: `keyProperty: 'sku'`, or an array such as `['sku', 'currency']` for a composite key (stored as `'SKU-1::EUR'`; see [Types](/sdk/api/types)). Duplicate or missing keys don't stop loading. When rows loaded together (one `data.set()` call, a model's `data` or one resource) share a key, the last row wins as a whole and dcupl records a `NonUniqueKey` quality error for each row it replaced. A row without a key (no value for `keyProperty`, or for every part of a composite key) is dropped as `MissingKey`; a composite key with only some parts missing keeps each missing part as `'undefined'` (`'null'` for a `null` value), e.g. `'undefined::EUR'`. On a model with derived, grouped, resolved or query references, or derived or expression properties, a keyless row currently makes `init()` (or an `update()` that rebuilds) log a failure and resolve `false`, so remove such rows before loading. After `init()`, a keyless row in a `data.upsert()` that a partial update applies is currently dropped without a `MissingKey` record; the next full rebuild records it, and on those models resolves `false`. See [Error Types](/sdk/advanced/quality-validation#error-types). ### Use natural keys when available ```typescript filename="natural-keys.ts" // Products with SKU dcupl.data.set( [ { key: 'SKU-12345', name: 'Laptop', price: 999 }, { key: 'SKU-67890', name: 'Mouse', price: 29 }, ], { model: 'Product' }, ); // Users with email dcupl.data.set( [ { key: 'alice@example.com', name: 'Alice' }, { key: 'bob@example.com', name: 'Bob' }, ], { model: 'User' }, ); ``` ### Generate keys for new data ```typescript filename="generated-keys.ts" function generateKey(): string { return `${Date.now()}-${Math.random().toString(36).slice(2, 9)}`; } // Or use a library like nanoid import { nanoid } from 'nanoid'; const newProduct = { key: nanoid(), name: 'New Product', price: 49.99, }; ``` ## Common patterns ### E-commerce product catalog ```typescript filename="ecommerce-model.ts" const productModel: ModelDefinition = { key: 'Product', properties: [ { key: 'sku', type: 'string', index: true }, { key: 'name', type: 'string' }, { key: 'price', type: 'float' }, { key: 'salePrice', type: 'float', quality: { required: false } }, { key: 'inStock', type: 'boolean' }, { key: 'rating', type: 'float' }, { key: 'imageUrl', type: 'string' }, ], references: [ { key: 'category', model: 'Category', type: 'singleValued' }, { key: 'brand', model: 'Brand', type: 'singleValued' }, { key: 'tags', model: 'Tag', type: 'multiValued' }, ], }; const categoryModel: ModelDefinition = { key: 'Category', properties: [ { key: 'name', type: 'string' }, { key: 'slug', type: 'string', index: true }, ], references: [ // Root categories have no parent { key: 'parent', model: 'Category', type: 'singleValued', quality: { required: false } }, ], }; const brandModel: ModelDefinition = { key: 'Brand', properties: [ { key: 'name', type: 'string' }, { key: 'logoUrl', type: 'string' }, ], }; ``` ### Content management ```typescript filename="cms-model.ts" const articleModel: ModelDefinition = { key: 'Article', properties: [ { key: 'slug', type: 'string', index: true }, { key: 'title', type: 'string' }, { key: 'excerpt', type: 'string' }, { key: 'content', type: 'string' }, { key: 'status', type: 'string' }, { key: 'publishedAt', type: 'date' }, ], references: [ { key: 'author', model: 'Author', type: 'singleValued' }, { key: 'categories', model: 'Category', type: 'multiValued' }, { key: 'tags', model: 'Tag', type: 'multiValued' }, ], }; ``` ### Order management ```typescript filename="order-model.ts" const orderModel: ModelDefinition = { key: 'Order', properties: [ { key: 'orderNumber', type: 'string', index: true }, { key: 'status', type: 'string' }, { key: 'total', type: 'float' }, { key: 'itemCount', type: 'int' }, { key: 'createdAt', type: 'date' }, // Derived for fast queries { key: 'customerEmail', type: 'string', derive: { localReference: 'customer', remoteProperty: 'email', }, }, ], references: [ { key: 'customer', model: 'Customer', type: 'singleValued' }, { key: 'items', model: 'OrderItem', type: 'multiValued' }, ], }; const orderItemModel: ModelDefinition = { key: 'OrderItem', properties: [ { key: 'quantity', type: 'int' }, { key: 'unitPrice', type: 'float' }, { key: 'lineTotal', type: 'float' }, ], references: [{ key: 'product', model: 'Product', type: 'singleValued' }], }; ``` ## Modeling checklist Before finalizing your models, verify: - [ ] Every property has a specific type (not just `string` for everything) - [ ] Exact-match lookups use references or `index: true` properties - [ ] Relationships use references instead of duplicated data - [ ] Reference depth is 3 levels or less - [ ] Deep query paths use derived properties - [ ] No circular references between models exist (a model may reference itself) - [ ] Unused properties are removed - [ ] Optional properties and references have `quality: { required: false }` - [ ] Keys are unique and meaningful ## Next steps - [Performance](/learn/best-practices/performance) — Optimize queries and updates - [Error handling](/learn/best-practices/error-handling) — Handle errors gracefully - [Models](/sdk/core-concepts/models) — Property types, references, and options --- # Performance best practices Source: https://docs.dcupl.com/learn/best-practices/performance Description: The principles behind fast dcupl applications, with links to the SDK performance guide # Performance best practices This page sums up the principles behind fast dcupl applications. The options, thresholds and code for each of them are in the SDK guide [Performance optimization](/sdk/advanced/performance). ## Performance principles ### Initialize once, update incrementally `init()` processes all models and data. Call it once at startup. Afterwards, stage changes with `data.update()`, `data.upsert()` or `data.remove()` and apply them with `await dcupl.update()`; never call `init()` again to apply a change. See [Partial Updates](/sdk/advanced/performance#partial-updates). ### Prefer partial updates, and batch them A partial update reprocesses only the changed items and the attributes that depend on them. `data.set()` and `data.reset()` always trigger a full rebuild, which re-runs the processing of `init()` over all models, and dcupl also rebuilds fully when more than half of the models and more than half of their items changed, when a model definition changed after `init()` (`models.set()`, `models.update()` or `models.remove()`), or when the data a query reference depends on changed. See [When dcupl Falls Back to a Full Update](/sdk/advanced/performance#when-dcupl-falls-back-to-a-full-update). Each `dcupl.update()` call has a fixed cost, so stage a batch of changes and apply it with one call. ### Filter on indexed properties References are indexed, and `index: true` adds an index to a property. `eq` and `in` conditions on them are index lookups; other operators evaluate the items. In an `'and'` group, conditions run in order on the items that are left, so put the most selective scanning conditions first. Index-backed `eq` and `in` conditions are cheap in any position. See [Query Optimization](/sdk/advanced/performance#query-optimization). ### Flatten deep queries Each level of a deep query such as `customer.country` resolves one more reference for every evaluated item. Use a [derived property](/sdk/advanced/derived-properties) for paths you filter on often, keep paths to 2–3 levels, and see [Data modeling](/learn/best-practices/data-modeling) for how to structure references. ### Read pages, not everything Pass `start` and `count` to a read: `items({ start: 50, count: 50 })`. Paging stored with `applyOptions({ start, count })` works too; `count()` then returns the page length, and `fn.metadata().currentSize` the number of all matches. See [Use Pagination](/sdk/advanced/performance#use-pagination). ### Reuse and destroy lists Create a list once and change its query instead of creating a list per search. Every `dcupl.lists.create()` registers a list that stays alive until you call `destroy()`. See [List Management](/sdk/advanced/performance#list-management). ### Load only what you need Select resources with tags, leave out properties the app never reads, and render long result lists virtualized. See [Memory Optimization](/sdk/advanced/performance#memory-optimization) and [Large Dataset Handling](/sdk/advanced/performance#large-dataset-handling). ### Measure with your own data Timings depend on your models, your data and the device. Time `init()`, `dcupl.update()` and your reads with `console.time()`, or read the `duration` of the `dcupl_initialized` and `dcupl_updated_manually` events. See [Profiling and Debugging](/sdk/advanced/performance#profiling-and-debugging). ## What holds in general - **One `dcupl.update()` per batch** is far faster than one `dcupl.update()` per change. - **Partial updates** (`data.update()`, `upsert()`, `remove()`) are faster than replacing a model with `data.set()`. - **Repeated reads of an unchanged query** are served from the list's cache until the next update that affects the list. ## Performance checklist - [ ] `init()` runs once, at startup - [ ] Changes are staged and applied with one `await dcupl.update()` per batch - [ ] Update payloads hold only the changed attributes - [ ] Exact-match filters use references or `index: true` properties - [ ] Frequent deep query paths are derived properties - [ ] Large result sets are read page by page - [ ] Lists are reused, and destroyed when no longer needed - [ ] Only the data and properties the app needs are loaded ## Next steps - [Performance optimization](/sdk/advanced/performance) — The SDK guide with options, thresholds and examples - [Data modeling](/learn/best-practices/data-modeling) — Structure models effectively - [Error handling](/learn/best-practices/error-handling) — Handle errors gracefully - [Caching](/sdk/advanced/caching) — Caching strategies --- # Error handling best practices Source: https://docs.dcupl.com/learn/best-practices/error-handling Description: Error handling principles and app-level recovery patterns for robust dcupl applications # Error handling best practices Robust error handling keeps your application stable and gives users a good experience even when things go wrong. This page covers the principles and the patterns that belong in your app: retries, fallbacks and empty states. How the SDK reports errors (error groups and types, loader failures, quality errors) is described in the SDK guide [Error handling](/sdk/guides/error-handling). ## Error handling principles - **Anticipate failures** - Assume network requests and data operations can fail - **Check results** - The SDK reports many problems through return values and quality errors, not exceptions - **Fail gracefully** - Provide fallbacks and meaningful error messages - **Log for debugging** - Capture enough context (operation, model, key) to diagnose issues - **Recover when possible** - Retry transient failures automatically ## Know what the SDK does instead of throwing Most dcupl calls don't throw when something is wrong. Check for each case: - **`init()` or `dcupl.update()` fails:** it logs the error and resolves `false`. Check the boolean it resolves to. - **Invalid model definition:** dcupl records an `InvalidModelDefinition` quality error, and `init()` still resolves `true`. - **Invalid or missing values:** dcupl records quality errors such as `WrongDataType` or `UndefinedValue` (attributes are required by default). - **Unknown key in `data.update()` or `remove()`:** ignored. Check that the item exists first, or use `data.upsert()`. - **Unknown key in `query.one()`:** returns `null`. Handle `null` in the UI. - **Unknown attribute or operator in a condition:** an empty result, with no error or log. Check the names against the model. - **Malformed condition** (`in` with a value that isn't an array, `find` with an array, no `operator`): throws `invalid query - …` and isn't stored, so the list's query stays as it was. - **Unknown model key:** `lists.create()` and `query.execute()` throw "Model with key X not found", and `query.one()` throws an Error with an empty message. Create lists after `await dcupl.init()`, and check the key. - **`data.set()` for a model that isn't registered:** the rows don't load into any model. `init()` only logs `Model not found during data processing` and still resolves `true`. See [Data for an Unknown Model](/sdk/guides/common-mistakes#data-for-an-unknown-model). Quality errors don't stop anything, so read them after loading, at least during development: ```typescript const qualityErrors = dcupl.quality.errors.summary(); if (qualityErrors.length > 0) console.warn('Data quality issues:', qualityErrors); ``` For an empty result you didn't expect, see [Query Returns No Results](/sdk/guides/common-mistakes#query-returns-no-results). For checking data quality after loading, see [Quality Validation](/sdk/advanced/quality-validation). ## App-level patterns The snippets use placeholder helpers from your app: `showErrorMessage()`, `showMessage()` and `renderProducts()`. ### Retry failed data loading Network requests can fail. Retry with exponential backoff before you give up: ```typescript filename="retry-loading.ts" async function loadDataWithRetry( dcupl: Dcupl, fetchFn: () => Promise, model: string, maxRetries = 3, ): Promise { for (let attempt = 1; attempt <= maxRetries; attempt++) { try { const data = await fetchFn(); dcupl.data.set(data, { model }); // staged until init() or update() return true; } catch (error) { console.error(`Load attempt ${attempt} failed for ${model}:`, error); if (attempt === maxRetries) { return false; } // Exponential backoff: 1s, 2s, 4s const delay = 1000 * Math.pow(2, attempt - 1); await new Promise((resolve) => setTimeout(resolve, delay)); } } return false; } // Usage const loaded = await loadDataWithRetry(dcupl, () => fetch('/api/products').then((r) => r.json()), 'Product'); if (!loaded) { showErrorMessage('Unable to load products. Please check your connection.'); } ``` When you load through `DcuplAppLoader`, the loader reports failed resources instead; see [Resource Retry Pattern](/sdk/guides/error-handling#resource-retry-pattern). ### Fall back to cached data If fresh data fails to load, use the last copy you cached: ```typescript filename="cache-fallback.ts" async function loadProductsWithFallback(dcupl: Dcupl): Promise { try { // Try to load fresh data const products = await fetch('/api/products').then((r) => r.json()); dcupl.data.set(products, { model: 'Product' }); // Cache for future fallback localStorage.setItem('products_cache', JSON.stringify(products)); localStorage.setItem('products_cache_time', Date.now().toString()); return true; } catch (error) { console.error('Failed to load fresh data:', error); // Try cached data const cached = localStorage.getItem('products_cache'); if (cached) { const cacheTime = localStorage.getItem('products_cache_time'); const cacheAge = Date.now() - Number(cacheTime); const maxAge = 24 * 60 * 60 * 1000; // 24 hours if (cacheAge < maxAge) { console.log('Using cached data'); dcupl.data.set(JSON.parse(cached), { model: 'Product' }); return true; } else { console.warn('Cache too old, not using'); } } return false; } } ``` ### Validate user input before querying Check filter values from the UI (a negative price, a minimum above the maximum) before they reach the query, and show a message instead of an empty result. ### Handle empty results Empty results are not errors, but the UI should say why it is empty: ```typescript filename="empty-results.ts" function searchProducts(list: DcuplList, searchTerm: string) { list.catalog.query.clear(); if (searchTerm.trim()) { // Wrap the term in slashes for substring search; a bare string must match the whole value. // Escape regex characters in user input first. const escaped = searchTerm.trim().replace(/[.*+?^${}()|[\]\\/]/g, '\\$&'); list.catalog.query.addCondition({ attribute: 'name', operator: 'find', value: `/${escaped}/`, options: { regexFlags: 'i' }, }); } const results = list.catalog.query.items(); const count = list.catalog.query.count(); return { items: results, count, isEmpty: count === 0, hasSearchTerm: searchTerm.trim().length > 0, }; } // Usage in UI const search = searchProducts(productList, userInput); if (search.isEmpty && search.hasSearchTerm) { showMessage(`No products found matching "${userInput}"`); } else if (search.isEmpty) { showMessage('No products available'); } else { renderProducts(search.items); } ``` ### Degrade gracefully If `init()` resolves `false`, keep the rest of the page usable: show the error and fallback content instead of the data-driven parts. The SDK guide shows [Graceful Degradation](/sdk/guides/error-handling#graceful-degradation) with a local fallback configuration and [Partial Data Loading](/sdk/guides/error-handling#partial-data-loading) with critical and optional resources. ### Log with context Log the operation, the model and the key with every error, and send errors to your monitoring service in production. See [Integration with Error Tracking Services](/sdk/guides/error-handling#integration-with-error-tracking-services). ## Error handling checklist - [ ] Check the boolean `init()` and `dcupl.update()` resolve to (they don't throw) - [ ] Check `dcupl.quality.errors.summary()` after loading, at least during development - [ ] Retry transient load failures, and fall back to cached data - [ ] Check that an item exists before you update it - [ ] Handle `null` from `query.one()` and empty results in the UI - [ ] Validate user-provided filter values before querying - [ ] Log failed operations with context ## Next steps - [Error handling](/sdk/guides/error-handling) — How the SDK reports errors - [Common mistakes](/sdk/guides/common-mistakes) — Avoid typical pitfalls - [Quality Validation](/sdk/advanced/quality-validation) — Validate data quality - [Data modeling](/learn/best-practices/data-modeling) — Structure models effectively --- # Playground Source: https://docs.dcupl.com/playground Description: Try dcupl in the browser: load a demo catalog, then filter, facet and search it step by step. # Playground ### 1. Initialize dcupl Connect to dcupl and load data One-time load: Data is fetched once and cached in browser memory. All subsequent queries are instant. ```text import { Dcupl } from '@dcupl/core'; import { DcuplAppLoader } from '@dcupl/loader'; const dcupl = new Dcupl({ config: { projectId: 'YOUR_PROJECT_ID', apiKey: 'YOUR_API_KEY' }, }); const loader = new DcuplAppLoader(); dcupl.loaders.add(loader); await loader.config.fetch(); // loader config from the dcupl CDN await loader.process(); await dcupl.init(); ``` ### 2. Create list Create a list to display Style data Create a queryable list from your Style model for filtering and display. ```text const list = dcupl.lists.create({ modelKey: 'Style' }); ``` ### 3. Get facets Retrieve available filter options Get filter options (facets) with item counts for each value. ```text // Facets: each value of an attribute with its item count const genderFacets = list.catalog.fn.facets({ attribute: 'gender' }); // Returns: [{ value: 'Men', size: 22165, ... }, ...] ``` ### 4. Apply filters Filter data instantly - all processing happens client-side. ```text list.catalog.query.addCondition({ attribute: 'gender', operator: 'eq', value: 'Men' }); ``` Advanced features (optional) ### 5. Search by name Search products by name instantly ```text list.catalog.query.addCondition({ attribute: 'productDisplayName', operator: 'find', value: '/shirt/', // '/…/' = contains options: { regexFlags: 'i' } }); ``` ### 6. Sort results Order by any attribute ```text list.catalog.query.items({ sort: { attributes: ['articleType'], order: ['ASC'] } }); ``` ### 7. Count items & values Count matching items and distinct values ```text // Count all items matching current filters // (query.count() would only count the current page) const totalItems = list.catalog.fn.metadata().currentSize; // Get unique values per attribute const categories = list.catalog.fn.facets({ attribute: 'masterCategory' }); ``` ## Query 44,000+ items instantly No backend setup. All processing in your browser. Instant queries No backend 44,000+ items **dcupl Way** You ask your client for a simple data export. You create a data model, upload it to the dcupl console, and start coding immediately. **Within hours, you're getting feedback on the actual application — not mockups or wireframes.** When the proof of concept is validated, you decide: keep working with static exports, or connect directly to their API. Either way, your frontend code stays exactly the same. All data loading, queries, and aggregations happen in the browser — fast and performant. ```mermaid sequenceDiagram participant UI participant SDK participant Server Note over UI,Server: One-time load UI->>Server: Load models + data Server-->>UI: Models & Data UI->>SDK: Initialize Note over UI,Server: All interactions = instant UI->>SDK: filter(category) SDK-->>UI: Instant results UI->>SDK: sort(price) SDK-->>UI: Instant results UI->>SDK: search(term) SDK-->>UI: Instant results Note over UI,Server: No more server calls! ``` **Traditional Way** First, you need to get access to the client's PIM system. Then you set up servers and databases to handle the query and facet logic. You build out the backend infrastructure. Only then can you start on the actual frontend. Weeks pass before you can even validate if the idea works — and you haven't written a single line of frontend code yet. ```mermaid sequenceDiagram participant UI participant Server participant DB Note over UI,DB: Every interaction = server calls UI->>Server: Query products Server->>DB: SELECT... DB-->>Server: Results Server-->>UI: Products UI->>Server: Get facets Server->>DB: COUNT BY... Server-->>UI: Facets Note over UI,DB: User filters... UI->>Server: Query + filter Server->>DB: SELECT WHERE... Server-->>UI: Products UI->>Server: Update facets Server-->>UI: Facets ``` ```mermaid flowchart LR subgraph Source PIM[PIM System] end subgraph Cloud["dcupl Console"] Export[Data Export] Model[Auto-gen Model] end subgraph Browser SDK[dcupl SDK] Memory[(Browser Memory)] UI[Your App] end PIM -->|export| Export Export --> Model Model -->|one-time load| SDK SDK --> Memory UI <-->|instant queries| Memory ``` - **In-browser processing:** 44,000+ products loaded into browser memory - **No network round trips:** Queries, filters, search, sorting & aggregations all run client-side - **Larger datasets:** Query time grows with data size. Measured with 1 million items: indexed queries took 25–65 ms, facets about 2.7 s