Data modeling best practices

Effective data modeling is the foundation of a performant dcupl application. This guide covers patterns and recommendations for structuring your models, properties, and relationships.

Core principles

Good data models share these characteristics:

Explicit structure - Every property has a defined type
Minimal duplication - Use references instead of copying data
Query-optimized - Properties used in filters are indexed
Right-sized - Only include properties you actually need

Property design

Choose specific types

Use the most specific type for each property. This enables better validation and optimization.

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'quantity', type: 'int' },
    { key: 'isActive', type: 'boolean' },
    { key: 'tags', type: 'Array<string>' },
    { key: 'createdAt', type: 'date' },
  ],
};

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'string' }, // Should be 'float'
    { key: 'quantity', type: 'string' }, // Should be 'int'
    { key: 'isActive', type: 'string' }, // Should be 'boolean'
    { key: 'tags', type: 'string' }, // Should be 'Array<string>'
    { key: 'createdAt', type: 'string' }, // Should be 'date'
  ],
};

Available types: string, int, float, boolean, date, Array<string>, Array<int>, Array<float>, Array<date>, json, any

Index properties for filtering

Properties you filter on should be indexed for fast queries. dcupl always indexes references and builds a property's index on demand (for example the first time you request facets for it); until then, filters on that property scan the items. Set index: true to build the index while the data is processed, so eq and in filters on frequently looked-up keys use it from the first query:

indexed-properties.ts

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    // Standard properties (indexed on demand, e.g. for facets)
    { key: 'name', type: 'string' },
    { key: 'category', type: 'string' },
    { key: 'price', type: 'float' },

    // Explicit index for fast key lookups
    { key: 'sku', type: 'string', index: true },

    // Display-only (no filtering needed)
    { key: 'description', type: 'string' },
  ],
};

filter: true on a property doesn't index anything: it only selects the attributes that catalog.filters returns for building filter UIs.

Only include what you need

Every property increases memory usage and processing time. Remove properties you do not query or display.

// Lean model - only what the UI needs
const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'category', type: 'string' },
    { key: 'imageUrl', type: 'string' },
  ],
};

// Bloated model - includes unused fields
const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'category', type: 'string' },
    { key: 'imageUrl', type: 'string' },
    { key: 'legacyId', type: 'string' }, // Never used
    { key: 'internalNotes', type: 'string' }, // Never used
    { key: 'importTimestamp', type: 'string' }, // Never used
    { key: 'rawData', type: 'json' }, // Never used
  ],
};

Mark optional properties

Every declared property and reference is required by default. An item without a value for one of them records an UndefinedValue quality error (a single UndefinedAttribute when no item has the attribute at all). init() still resolves true, the item loads without the value and nothing is logged, so read dcupl.quality.errors.summary() during development. Set quality: { required: false } on attributes that are missing on some items (a value sent as null is a separate NullValue error, which quality: { nullable: true } allows):

optional-properties.ts

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    // Only products on sale have a sale price
    { key: 'salePrice', type: 'float', quality: { required: false } },
  ],
};

To make most attributes of a model optional, set quality: { attributes: { required: false } } on the model and mark the must-have ones quality: { required: true }. References take required too: the root of a category tree has no parent. See Model-wide Defaults.

Reference design

Use references instead of duplicating data across models. This keeps data consistent and reduces memory usage.

// Normalized: Customer data lives in one place
const orderModel: ModelDefinition = {
  key: 'Order',
  properties: [
    { key: 'orderNumber', type: 'string' },
    { key: 'total', type: 'float' },
    { key: 'status', type: 'string' },
  ],
  references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }],
};

const customerModel: ModelDefinition = {
  key: 'Customer',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'email', type: 'string' },
    { key: 'phone', type: 'string' },
  ],
};

// Denormalized: Customer data duplicated in every order
const orderModel: ModelDefinition = {
  key: 'Order',
  properties: [
    { key: 'orderNumber', type: 'string' },
    { key: 'total', type: 'float' },
    { key: 'status', type: 'string' },
    { key: 'customerName', type: 'string' }, // Duplicated
    { key: 'customerEmail', type: 'string' }, // Duplicated
    { key: 'customerPhone', type: 'string' }, // Duplicated
  ],
};

Use correct reference types

Choose the right reference type based on the relationship:

Relationship	Reference Type	Example
Many-to-one	`singleValued`	Order has one Customer
One-to-many	`multiValued`, or a `singleValued` reference on the child plus a resolved reference on the parent	Order has many OrderItems
Many-to-many	`multiValued`	Product has many Tags

reference-types.ts

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [{ key: 'name', type: 'string' }],
  references: [
    // Many products belong to one category
    { key: 'category', model: 'Category', type: 'singleValued' },

    // One product has many tags
    { key: 'tags', model: 'Tag', type: 'multiValued' },
  ],
};

Limit reference depth

Keep reference chains to 2-3 levels. Deep chains slow down queries significantly.

// 2-level depth - fast queries
orderList.catalog.query.addCondition({
  attribute: 'customer.country',
  operator: 'eq',
  value: 'USA',
});

// 4-level depth - very slow
orderList.catalog.query.addCondition({
  attribute: 'customer.address.country.region',
  operator: 'eq',
  value: 'West Coast',
});

Each level resolves one more reference for every item the condition evaluates, so the cost grows with the depth and with the number of items. How much depends on your data and on whether the list has already resolved the path, so measure with your own data (see Profiling and Debugging).

Use derived properties for deep queries

For frequently-accessed deep data, use derived properties to flatten the access path:

derived-property.ts

const orderModel: ModelDefinition = {
  key: 'Order',
  properties: [
    { key: 'orderNumber', type: 'string' },
    { key: 'total', type: 'float' },

    // Derived property for fast queries
    {
      key: 'customerCountry',
      type: 'string',
      derive: {
        localReference: 'customer',
        remoteProperty: 'country',
      },
    },
  ],
  references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }],
};

// Fast: Query derived property directly
orderList.catalog.query.addCondition({
  attribute: 'customerCountry',
  operator: 'eq',
  value: 'USA',
});

Avoid circular references

Circular references between models are detected and rejected. Design one-way relationships and query in reverse when needed.

A reference from a model to itself is not a cycle: a Category with a parent reference to Category is fine. dcupl resolves such references against all rows of the model, whatever the row order, so hierarchies work with deep queries (parent.name) and facets.

// One-way reference: Order -> Customer
const orderModel: ModelDefinition = {
  key: 'Order',
  properties: [{ key: 'orderNumber', type: 'string' }],
  references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }],
};

const customerModel: ModelDefinition = {
  key: 'Customer',
  properties: [{ key: 'name', type: 'string' }],
  // No reference back to Order
};

// Query orders for a customer (reverse direction)
orderList.catalog.query.addCondition({
  attribute: 'customer',
  operator: 'eq',
  value: 'customer-123',
});

// Circular reference: Order -> Customer -> Order
const orderModel: ModelDefinition = {
  key: 'Order',
  references: [{ key: 'customer', model: 'Customer', type: 'singleValued' }],
};

const customerModel: ModelDefinition = {
  key: 'Customer',
  references: [
    { key: 'orders', model: 'Order', type: 'multiValued' }, // Circular!
  ],
};
// await dcupl.init() logs "Circular dependency detected: Order → Customer → Order"
// and resolves false

If you need the orders on each customer, declare a resolved reference instead. It collects the items that point at this one and is not part of the dependency check, so it doesn't create a cycle:

resolved-reference.ts

const customerModel: ModelDefinition = {
  key: 'Customer',
  properties: [{ key: 'name', type: 'string' }],
  references: [
    // Every Order whose `customer` reference points at this customer
    { key: 'orders', model: 'Order', type: 'multiValued', resolve: { model: 'Order', reference: 'customer' } },
  ],
};

See Models API for resolved and derived references.

Model organization

One model per entity

Each distinct entity should have its own model. Do not combine unrelated data.

// Separate models for distinct entities
const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
  ],
};

const categoryModel: ModelDefinition = {
  key: 'Category',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'slug', type: 'string' },
  ],
};

// Generic model for everything
const entityModel: ModelDefinition = {
  key: 'Entity',
  properties: [
    { key: 'type', type: 'string' }, // 'product' or 'category'
    { key: 'data', type: 'json' }, // Unstructured blob
  ],
};

Consistent key naming

Use consistent naming conventions across all models:

naming.ts

// Recommended: camelCase for properties, PascalCase for models
const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'productName', type: 'string' },
    { key: 'listPrice', type: 'float' },
    { key: 'isAvailable', type: 'boolean' },
    { key: 'createdAt', type: 'date' },
  ],
  references: [{ key: 'parentCategory', model: 'Category', type: 'singleValued' }],
};

Data key strategy

Every item needs a unique key. Choose a strategy that fits your data. If the identifier is in another column, set keyProperty on the model instead of copying it into key: keyProperty: 'sku', or an array such as ['sku', 'currency'] for a composite key (stored as 'SKU-1::EUR'; see Types).

Duplicate or missing keys don't stop loading. When rows loaded together (one data.set() call, a model's data or one resource) share a key, the last row wins as a whole and dcupl records a NonUniqueKey quality error for each row it replaced. A row without a key (no value for keyProperty, or for every part of a composite key) is dropped as MissingKey; a composite key with only some parts missing keeps each missing part as 'undefined' ('null' for a null value), e.g. 'undefined::EUR'. On a model with derived, grouped, resolved or query references, or derived or expression properties, a keyless row currently makes init() (or an update() that rebuilds) log a failure and resolve false, so remove such rows before loading. After init(), a keyless row in a data.upsert() that a partial update applies is currently dropped without a MissingKey record; the next full rebuild records it, and on those models resolves false. See Error Types.

Use natural keys when available

natural-keys.ts

// Products with SKU
dcupl.data.set(
  [
    { key: 'SKU-12345', name: 'Laptop', price: 999 },
    { key: 'SKU-67890', name: 'Mouse', price: 29 },
  ],
  { model: 'Product' },
);

// Users with email
dcupl.data.set(
  [
    { key: 'alice@example.com', name: 'Alice' },
    { key: 'bob@example.com', name: 'Bob' },
  ],
  { model: 'User' },
);

Generate keys for new data

generated-keys.ts

function generateKey(): string {
  return `${Date.now()}-${Math.random().toString(36).slice(2, 9)}`;
}

// Or use a library like nanoid
import { nanoid } from 'nanoid';

const newProduct = {
  key: nanoid(),
  name: 'New Product',
  price: 49.99,
};

Common patterns

E-commerce product catalog

ecommerce-model.ts

const productModel: ModelDefinition = {
  key: 'Product',
  properties: [
    { key: 'sku', type: 'string', index: true },
    { key: 'name', type: 'string' },
    { key: 'price', type: 'float' },
    { key: 'salePrice', type: 'float', quality: { required: false } },
    { key: 'inStock', type: 'boolean' },
    { key: 'rating', type: 'float' },
    { key: 'imageUrl', type: 'string' },
  ],
  references: [
    { key: 'category', model: 'Category', type: 'singleValued' },
    { key: 'brand', model: 'Brand', type: 'singleValued' },
    { key: 'tags', model: 'Tag', type: 'multiValued' },
  ],
};

const categoryModel: ModelDefinition = {
  key: 'Category',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'slug', type: 'string', index: true },
  ],
  references: [
    // Root categories have no parent
    { key: 'parent', model: 'Category', type: 'singleValued', quality: { required: false } },
  ],
};

const brandModel: ModelDefinition = {
  key: 'Brand',
  properties: [
    { key: 'name', type: 'string' },
    { key: 'logoUrl', type: 'string' },
  ],
};

Content management

cms-model.ts

const articleModel: ModelDefinition = {
  key: 'Article',
  properties: [
    { key: 'slug', type: 'string', index: true },
    { key: 'title', type: 'string' },
    { key: 'excerpt', type: 'string' },
    { key: 'content', type: 'string' },
    { key: 'status', type: 'string' },
    { key: 'publishedAt', type: 'date' },
  ],
  references: [
    { key: 'author', model: 'Author', type: 'singleValued' },
    { key: 'categories', model: 'Category', type: 'multiValued' },
    { key: 'tags', model: 'Tag', type: 'multiValued' },
  ],
};

Order management

order-model.ts

const orderModel: ModelDefinition = {
  key: 'Order',
  properties: [
    { key: 'orderNumber', type: 'string', index: true },
    { key: 'status', type: 'string' },
    { key: 'total', type: 'float' },
    { key: 'itemCount', type: 'int' },
    { key: 'createdAt', type: 'date' },

    // Derived for fast queries
    {
      key: 'customerEmail',
      type: 'string',
      derive: {
        localReference: 'customer',
        remoteProperty: 'email',
      },
    },
  ],
  references: [
    { key: 'customer', model: 'Customer', type: 'singleValued' },
    { key: 'items', model: 'OrderItem', type: 'multiValued' },
  ],
};

const orderItemModel: ModelDefinition = {
  key: 'OrderItem',
  properties: [
    { key: 'quantity', type: 'int' },
    { key: 'unitPrice', type: 'float' },
    { key: 'lineTotal', type: 'float' },
  ],
  references: [{ key: 'product', model: 'Product', type: 'singleValued' }],
};

Modeling checklist

Before finalizing your models, verify:

Every property has a specific type (not just string for everything)
Exact-match lookups use references or index: true properties
Relationships use references instead of duplicated data
Reference depth is 3 levels or less
Deep query paths use derived properties
No circular references between models exist (a model may reference itself)
Unused properties are removed
Optional properties and references have quality: { required: false }
Keys are unique and meaningful