.md

Importing data

The import form quietly does three things, and the last one is a checkbox most people scroll past. Here is all three.

flowchart LR
  I["Upload · paste · URL"] --> F["Project file<br/>data/products.csv"]
  F --> M["Model<br/>existing · inferred · none"]
  F --> R["Resource entry<br/>in dcupl.lc.json"]

1. Your rows become a file

An upload or a paste is written to the path shown under Upload Path — the field starts at data/, and picking products.csv makes it data/products.csv — and is a project file from then on. You can open it in the file editor, replace it, or push and pull it with the CLI like any other file.

An external URL is different: it is not copied. The import points at the data where it already lives, and every load re-fetches it from there. Use it for a feed you do not own; use an upload when you want the rows versioned with the project.

2. The rows are loaded into a model

Model handling decides which model the file feeds:

  • Existing Model — point the file at a model key that is already in the project.
  • Create New Model — the importer reads a sample of the file, infers the attributes, and hands you straight into the model creation form. For a file over 10 MB it infers from the first 2 MB instead of reading the whole file; that only works for CSV, so a large JSON file falls back to Skip Model Creation.
  • Skip Model Creation — no model file is created or assigned.

Skipping does not cancel the import. The resource entry is still written, with "model": "UNDEFINED_MODEL" when you gave no model key. The rows still load: with the loader's default options a data resource whose model is not declared gets a model of that key, inferred from the rows. Set the entry's model to the key you want before you build on it — the console will not remind you.

3. Nothing loads until a resource entry exists

The checkbox at the bottom of the form, Create a new resource in the loader configuration, is on by default. It adds one data entry to dcupl.lc.json: this URL, the model it feeds, and any tags you gave it. That entry is what makes the file part of an application — a file sitting in the project with no resource entry is never fetched by anything.

Untick it only when you deliberately want to stage a file that no application fetches yet. If you do, remember to add the resource entry by hand later, either in the file editor or by re-running the import.

Tags you enter here are the same tags applications select on. A file tagged catalog reaches every application that selects catalog and every application with no tags, and stays out of applications that select only other tags.