Skip to main content

Domain schemas

note

This feature is in Beta

A domain schema is a plugin-defined PostgreSQL schema whose tables are mapped to Datagrok entities. Unlike plain plugin databases, where you write your own SQL and server code, domain tables get the full platform treatment out of the box:

  • Security: row-level and column-level access control using standard Datagrok groups and permissions — no separate security model to build.
  • Managed CRUD: the server builds all queries, validates values, checks permissions, and writes an audit trail in the same transaction. You never write SQL or a backend.
  • Standard UI: browsing, search, filtering, create/edit dialogs, in-grid editing, import/export, sharing, watching, and history work automatically — see Domains.
  • Typed API: a generic JS client (grok.dapi.domains), generated TypeScript interfaces per table, and the u2 domain controls — Building app UI: u2.

You declare tables once in a JSON manifest; Datagrok creates the database objects on package deployment and upgrades them when the manifest changes.

For working examples, see the Grit (issue tracker), Inventory (batch upload, upsert, aggregation), and PlatesFixture (all security modes) packages.

Declaring a schema​

Add databases/<schema>/schema.json to your package. The directory name must match the manifest's name. The manifest is validated against domain-schema.schema.json.

{
  "name": "grit",
  "version": "1.0.0",
  "tables": {
    "project": {
      "businessKey": ["key"],
      "columns": {
        "key":         {"type": "string", "required": true, "unique": true},
        "name":        {"type": "string", "required": true, "isName": true},
        "description": {"type": "string"}
      }
    },
    "issue": {
      "securityMode": "master",
      "delegate": "project_id",
      "businessKey": ["project_id", "number"],
      "columns": {
        "project_id": {"type": "ref", "ref": "project", "required": true, "onDelete": "cascade"},
        "number":     {"type": "int", "required": true},
        "title":      {"type": "string", "required": true, "isName": true},
        "status":     {"type": "string", "choices": ["open", "in progress", "resolved", "closed"], "default": "open"},
        "assignee":   {"type": "user"}
      }
    }
  }
}

Every table automatically gets the system columns id (UUID, also the row's entity id), version (optimistic concurrency counter), created_on, updated_on, author_id, and is_deleted (rows are soft-deleted by default). Do not declare them.

Table options​

OptionDefaultDescription
securityModetabletable, master, or row — see security modes
delegate—Master mode: the ref column whose target row's security applies
promotionlazyRow mode: when the row becomes an individually sharable entity (lazy = on first share, eager = on insert, with the author granted View/Edit/Delete/Share)
defaultRowVisibilitytableRow/master modes: whether table-level View shows unshared rows (none hides them from everyone but their author)
businessKey—Natural-key column list: powers deduplication on insert, upsert matching, and search handles
audittrueIn-transaction audit trail with before/after diffs; also enables row history and row-level watch
hierarchyfalseThe table is a tree: it must declare exactly one ref column targeting itself, which becomes its parent column. Enables a row's ancestor path and the under subtree filter term (location_id under "<id>"), and lets domains.tree walk it
softDeletetrueDeletes mark is_deleted instead of removing rows
idempotencyfalseAdds the idempotency_key column for replay-safe creates
extensiblefalseLets users add their own columns to this table
schemas—Property schemas contributing dynamic columns
filters—Default filters shown on the table's filter panel
permissions—Custom permissions an app gates its actions on (["approve"] or {"approve": {"description": "..."}}): grantable on the table like View/Edit, answered as access.can.approve and the per-row ~can_approve column; grants may name them
friendlyName, description—Display metadata

Column options​

Column type is one of string, int, float, bool, datetime, string_list, ref (a reference to a row of another table — in this manifest, a Core table, or another plugin's schema), file (a user-uploaded file in platform file storage), or json (an opaque object; filters and aggregates refuse it). user and group are still accepted as aliases of ref: "Core.users" and ref: "Core.groups".

A file column stores a file://<connection>/<path> string. Row dialogs render a file input: picking a local file uploads it to the shared System:DomainFiles storage under an unguessable per-value path, and grids render the file with its name and size, with download and preview available from the Context Panel. Anyone who obtains the stored path can access the file — access control comes from the row and column security of the column that carries it.

"issue": {"columns": {"attachment": {"type": "file", "friendlyName": "Attachment"}}}
OptionDescription
requiredRejects null values
uniqueUnique among live (not soft-deleted) rows
refTarget table (for type: "ref"): a table of this manifest, or <Schema>.<table> for any registered table (Core.queries, grit.issue)
onDeleteReferential action on delete: cascade, restrict, or setnull
min, maxNumeric range validation
choicesControlled dictionary of allowed values (renders as a combo box)
defaultDefault value for new rows and dialog inputs
autoNumbertrue or {"scope": "<ref column>", "start": N}: the engine numbers new rows from a counter (int only, see Auto-numbering)
isNameMarks the primary display-name column (one per table, string only); its value titles cards, tooltips, and entity views. Without it, a string column literally named name is used by convention
semType, friendlyName, description, formatDisplay and semantic metadata
editorInput editor hint for forms: textarea, switch, slider, color, tags, or markdown (reaches Property.inputType; markdown renders as a text area for now)
searchableThe query search key ({search: "alp"}) matches this string column, case-insensitively; a table declaring none searches its name column
filterref columns: a filter expression over the target table narrowing the picker's candidates; $name binds to the owning row's column of that name — "country_id = $country_id" makes a dependent picker

Validation runs twice with the same code: client-side in dialogs (instant feedback) and server-side on every write (authoritative). Integrity constraints (unique, foreign keys, required) are additionally enforced by the database.

Table-level rules go in constraints, keyed by name. A check is raw SQL over the table's relational columns (whitelisted tokens only). An expr is the platform's filter grammar — literals, lists, null, and a bare column name on the right of a comparison — compiled to the same database CHECK and, unlike check, validated by forms before the write, with message shown on violation:

"constraints": {
  "dates_ordered": {"expr": "end_date >= start_date", "message": "End before start"},
  "positive":      {"check": "weight IS NULL OR weight > 0"}
}

Auto-numbering​

An int column with autoNumber gets its value from a server-side counter — globally per table ("autoNumber": true), or per master row when scope names a ref column of the same table: Grit numbers issues per project, so GRIT-1, GRIT-2, ... and PUB-1 count independently. start (default 1) is the first number handed out.

"issue": {
  "businessKey": ["project_id", "number"],
  "columns": {
    "project_id": {"type": "ref", "ref": "project", "required": true},
    "number": {"type": "int", "autoNumber": {"scope": "project_id"}}
  }
}

The rules:

  • A row inserted without the column (or with null) is numbered inside the insert; a supplied value is kept — imports and ingestion scripts bring their own numbers — and the counter catches up, so the next blank insert continues after it. Numbers are never reused: a deleted row keeps its number, and a unique index over (scope, number) enforces that.
  • autoNumber implies immutable: once a row has a number, changing it is refused. Do not combine it with required or unique — the counter fills every new row, and the index is implied. The scope column must be required; one auto-numbered column per table.
  • The typed clients reflect this: <T>Row.number is a number, <T>Insert.number is optional. Batch upsert payloads must carry the column (the engine cannot know which rows will match); batch insert may omit it. A batch payload that names the column must give every row a value — rows without one are refused per row.
  • Adding a new auto-numbered column is an ordinary additive change. Turning auto-numbering on or off for an existing column (or changing its scope) is a structural change: a plugin publish needs a migration script next to schema.json (the deploy is refused and names the required transition otherwise), and a runtime schema change asks for confirmation. Turning it on seeds the counters from the existing maximum per scope. Existing duplicates (deleted rows keep their numbers) are checked before anything runs by a runtime apply and a debug publish, which refuse; a release publish runs your migration script as written, so deduplicate the column in the script or beforehand — otherwise the unique index creation fails.

Property schemas and column security​

Besides relational columns, a table can carry dynamic, strongly-typed values grouped into property schemas — the mechanism behind column-level security:

{
  "tables": {
    "item": {
      "columns": {"sku": {"type": "string", "required": true}},
      "schemas": ["chemistry", "procurement"]
    }
  },
  "propertySchemas": {
    "chemistry":   {"cas_number": {"type": "string"}, "hazard_class": {"type": "string", "choices": ["1", "2", "3"]}},
    "procurement": {"unit_cost": {"type": "float", "min": 0}, "reorder_point": {"type": "int"}}
  }
}

A user sees (and can edit) a column only when a group they belong to holds View (or Edit) on one of the column's property schemas. In the example, chemists granted the chemistry schema see cas_number and hazard_class, while procurement sees unit_cost — on the same rows. Hidden columns never leave the server: they are absent from query results, exports, and filters. Relational columns belong to the table's built-in "core" schema, which is granted to all users on deployment.

References​

A ref column links each row to one row of a target table. The target is a table of the same manifest by name, or any registered table in the qualified <Schema>.<table> form: a Core table (Core.users, Core.groups, Core.queries, Core.connections, Core.scripts, Core.spaces, ...) or a table of another plugin's schema (grit.issue). Every reference behaves the same way — filters and facets travel it, expand returns the target's columns, row dialogs pick from the target's catalog, and grids show the target's display name:

"source_query": {"type": "ref", "ref": "Core.queries"},
"issue":        {"type": "ref", "ref": "grit.issue"}

How strongly the reference is enforced follows from the target — it is never declared:

  • A target in the same schema, or Core.users / Core.groups, is a hard reference: a physical foreign key, so a referenced row cannot be hard-deleted while rows point at it.
  • Any other target is a soft reference: an indexed id column with no foreign key, so the platform's own lifecycle of the target (garbage collection, package uninstall, core migrations) is never blocked by your rows. Such a reference may dangle after its target is hard-deleted: the row keeps the id, grids show that id instead of a name, and filters through the reference match no rows. Nothing sweeps dangling references. onDelete is not available on a soft reference.

A cross-plugin reference needs its target registered first — deploying the referrer before the target plugin is refused. While another schema references one of your tables, dropping that table or purging your schema is refused, naming the referrer.

type: "user" and type: "group" remain accepted as aliases of ref: "Core.users" and ref: "Core.groups" — both spellings produce the same column, with the User / Group semantic type that drives the platform's user renderer and picker.

grok api resolves a qualified reference at build time — Core tables from the declaration shipped with the CLI, another plugin's tables from its databases/<schema>/schema.json in node_modules (so that plugin must be a dependency) — and types the <Table>Expand entry exactly like a same-schema reference.

Many-to-many relations​

A ref column links each row to one target row. To link a row to many target rows — issues to labels, samples to projects — declare a relation on the owning table. The junction is an ordinary domain table you declare yourself. The relation adds no DDL and no migration — it only tells the platform how to travel the junction:

{
  "tables": {
    "label": {
      "columns": {"name": {"type": "string", "required": true, "isName": true}}
    },
    "issue_label": {
      "securityMode": "master",
      "delegate": "issue_id",
      "businessKey": ["issue_id", "label_id"],
      "columns": {
        "issue_id": {"type": "ref", "ref": "issue", "required": true, "onDelete": "cascade"},
        "label_id": {"type": "ref", "ref": "label", "required": true, "onDelete": "cascade"}
      }
    },
    "issue": {
      "relations": {"labels": {"via": "issue_label", "target": "label"}},
      "columns": {"title": {"type": "string", "required": true, "isName": true}}
    }
  }
}

That declaration alone powers the whole stack — a chips column in grids, a tags editor in row dialogs and in the grid cell popup, a filter facet, related-row panes, and the API surface below — with zero UI code. viaSelf and viaTarget name the junction's FK columns explicitly when it carries more than one ref to a side (a self-referential relation must name both). allowCreate: true lets users create and link a new target row directly from the tags editor. A relation name shares the expand/filter namespace with columns, so it may not collide with one, and the junction's businessKey must cover both FK columns — that unique index is what makes re-linking idempotent.

Working with relations from the API:

  • Read: expand: ['labels'] returns each row's links as an array of {id, name}, ordered by display name. DataFrame reads (queryDf, Open in Table View) flatten the names into a chips-rendered labels column instead.
  • Filter: dotted paths travel the relation — labels.name = "bug", or chains that continue from the target table. The semantics are per-link: labels.name != "bug" means "has a label other than bug", not "has no bug label". The set-level questions live on the id leaf: labels.id = null matches rows with no links at all, and excluding a list of ids matches rows linked to none of them.
  • Write: insert and patch accept the relation as the full set of linked ids — {"labels": [id1, id2]}. The server diffs it against the links the caller can currently see and applies the difference through ordinary junction writes, so audit, deduplication, and referential actions come free. Consequences: [] clears only the links visible to you, an absent key leaves the relation untouched, and a replace never removes a link its author cannot see. batch refuses relation keys — load the junction table directly instead. To create a target and link it atomically, use a transaction with $ref placeholders inside the id list.

Who may link is decided by the junction table's own security. The recommended shape is the one above — a master-mode junction delegating to the owner, so Edit on an issue is Edit on its links (and unlinking needs no separate Delete grant). A relation whose junction or target table you cannot View is invisible everywhere, exactly like a name nobody declared. Do not set defaultRowVisibility: "none" on a junction or a relation target — rows created there would reach only their own author, nobody else could link them, and the manifest is rejected.

Default filters​

Without configuration, the table's filter panel is constructed automatically from the column types and value cardinality. To control it, declare a per-table filters section — it replaces the automatic selection for that table, and the declared filters appear pre-opened, in order (users can still add more from the panel):

"filters": [
  {"column": "sample_state"},
  {"column": "volume", "type": "histogram", "bins": 24},
  {"column": "plate_id.barcode", "label": "Plate barcode"},
  {"column": "created_on", "type": "range"}
]
KeyDescription
columnA declared column, a property-schema column, a system column (such as created_on), a dotted reference path like plate_id.barcode, or a relation id leaf like labels.id (relation facets appear automatically only in the automatic panel — a declared filters section must list them explicitly to keep them)
typecategories, histogram, range, text, or bool. Omit to pick automatically from the column type
binsHistogram bucket count (1–200). Requires an explicit histogram or range type
labelDisplay caption for the filter

The section is validated on deployment, and every problem fails the publish with a distinct error:

  • Unknown columns, unknown descriptor keys, and duplicate columns are rejected.
  • type must match the column type: histogram and range apply to int, float, and datetime columns, text to string, bool to bool (categories applies to any).
  • bins without an explicit histogram or range type is rejected.
  • In a dotted path, every segment but the last must be a ref column, and paths are capped at three hops. Paths point forward only — from the table to the tables it references — and support the categories type only.

Security modes​

Each table declares how its rows are protected:

ModeTypical useBehavior
tableLookup and reference tables (default)One permission check against the table itself: a View grant shows all rows, Edit allows writes
masterDetail tables (issue → project, well → plate)Each row inherits the security of the row it references through the delegate column; chains up to two hops deep
rowRegistration masters (studies, plates), user-owned records (models, files)Individual rows can be shared with users and groups; a row's author always sees, edits, deletes and shares it; unshared rows otherwise follow the table-level grant, or stay hidden from everyone else with defaultRowVisibility: "none" (private to the author, shareable by them)

Grants use the standard permissions (View, Edit, Delete, Share) on the schema, table, and property-schema entities. Grant them from the UI (the table's Sharing pane) or programmatically — grok.dapi.domains.table('s.t').grants()/grant()/revoke() for a table, grok.dapi.domains.schema('s').grants()/grant()/revoke() for the schema entity. Note that schema-level grants gate schema operations (apply requires Edit, delete requires Delete, sharing requires Share) — they do not grant access to row data; grant per table for that. In row mode, sharing a row for the first time transparently promotes it to a full platform entity — after that it behaves like any entity: sharing dialog, favorites, comments, global search.

On deployment, the publishing user receives the full permission set on the schema, its tables, and property schemas.

Deployment and upgrades​

The schema deploys when the package is published (grok publish) and updates on every subsequent publish:

  • Additive changes (new tables, new columns, new property schemas, metadata edits) are applied automatically.
  • Destructive changes (dropping or retyping columns) are refused with an error — declare explicit statements under the manifest's migrations key to apply them deliberately.
  • Uninstalling the package keeps the data (the schema is orphaned); reinstalling re-adopts it. A full purge is available to administrators.

Extending a plugin schema​

Every deployment is different, and users routinely need one more field. A schema can invite that instead of forcing a fork: opt in from the manifest, and users you trust add their own tables and columns to your database at runtime.

{
  "name": "grit", "version": "1.2.0",
  "extensible": {"tables": true},
  "tables": {
    "issue": {"extensible": true, "columns": {"title": {"type": "string"}}}
  }
}
  • "extensible": {"tables": true} at the root lets users add their own tables.
  • "extensible": true on a table lets them add their own columns to it.

Both are off by default, and both are yours to revoke: turning a flag off blocks new extensions while everything already added keeps working.

Users also need the Extend permission on the schema entity — grant it from the schema's Share... dialog, or with grok.dapi.domains.schema('grit').grant(groupId, 'Extend'). Extend is a schema-level permission; it does not by itself grant access to any row data.

What users may do is deliberately narrow:

  • Their own tables get the full manifest vocabulary, and they manage what they created.
  • Their own columns on your tables stay optional and non-unique, never join the business key, never become the display-name column, and may only restrict or setnull on delete — your existing rows can never be invalidated or deleted by someone else's column.
  • Everything you declared is immutable to them: your columns, your table metadata, your property schemas.

Republishing your package is safe. The deploy diff sees only your objects: user tables and columns are retained untouched, and a plugin column or table that would collide with one of theirs is refused with a named error rather than silently adopting their data. Uninstalling orphans the schema and keeps their work; reinstalling re-adopts it.

Two things to know about how extension columns are stored:

  • They live under an x_ physical prefix in PostgreSQL (x_customer_id) so they can never collide with a column you add later. Every API surface — insert, query, filter, patch, batch, d42, the audit trail — uses the declared name (customer_id). The prefix does surface in raw PostgreSQL messages that quote an identifier verbatim, such as a constraint-violation error naming fk_issue_x_customer_id. Because of the prefix, no column you declare may start with x_; the manifest validator rejects it.
  • They exist only in the server registry, so grok api never generates them into your typed clients — your generated code keeps describing exactly what your manifest declares. Users reach their columns through the generic client, grok.dapi.domains.table('grit.issue').

Extension applies use the same endpoint as any other schema change (schema.apply) with an extend section, and the same dry-run/confirm flow for anything destructive:

await grok.dapi.domains.schema('grit').apply({
  tables: {customers: {columns: {name: {type: 'string', isName: true}}}},
  extend: {issue: {columns: {customer_id: {type: 'ref', ref: 'customers', onDelete: 'setnull'}}}},
});

The extend section is full state for the user's own columns of that table: omitting one they previously added proposes its drop. Optimistic concurrency runs on the schema's extension counter (ifVersion ↔ extVersion, echoed in the dry-run plan) — your package's own version is never touched by an extension. See dapi/domains/extend-schema.js in ApiSamples.

Working with data from JS​

The generic client is grok.dapi.domains. Address tables as '<schema>.<table>':

const issues = grok.dapi.domains.table('grit.issue');

// Filtered, sorted query (same grammar as entity search; 10k-row cap)
const open = await issues.query({
  filter: 'status = "open" and title starts "Crash"',
  sort: '!created_on',
  limit: 100,
});

// The same query as a typed DataFrame (10M-row cap); columns carry semantic types,
// choices, and property tags, so grids render them correctly out of the box
const df = await issues.queryDf({filter: 'assignee = @current'});

// Single row by id; null when absent or not visible
const issue = await issues.get(id);

// Insert (per-row report; business-key duplicates are reported, not duplicated)
const [r] = await issues.insert({project_id: projectId, number: 42, title: 'Crash on save'});

// Update with optimistic concurrency: fails with a version conflict if the row
// changed since you read it
await issues.update(r.id, {status: 'resolved'}, {version: issue.version});

// Soft delete (declared referential actions are enforced)
await issues.delete(r.id);

// The trash: a deleted row is still addressable, and Delete is also the grant that restores it
const trashed = await issues.query({filter: `id = "${r.id}"`, deleted: 'only'});   // carries ~is_deleted
await issues.restore(r.id);                                                        // audit op 'undelete'

// One value into many rows, one transaction (per-row validation, version and audit line);
// the filter is required, and the caller's Edit permission narrows the selection silently
const {updated, hasMore} = await issues.updateWhere(
  `id in ("${a}", "${b}")`, {status_id: closedId}, {limit: 1000});

// On a table declaring `"hierarchy": true`: the row's ancestors, root first, row excluded
const path = await grok.dapi.domains.table('stockroom.location').pathTo(shelfId);

// Aggregate over the rows and columns the caller can see
const counts = await issues.aggregate({
  groupBy: ['status'],
  measures: [{fn: 'count'}],
});

// Row history (audit-enabled tables)
const trail = await issues.audit(r.id);

For a typed DataFrame of aggregation results, use DomainQuery in aggregate mode (below): one row per group, columns typed from the resolved measures. A boolean groupBy column comes back as strings ('true'/'false'), and aggregate outputs carry no property tags — they are not registry columns.

The same select and aggregate surface is packaged as the DomainQuery function — the reproducible query the platform records behind Open in Table View and data-synced domain dashboards (see Queries and dashboards). It is callable from JS like any function:

// Select mode: typed DataFrame, ref-column captions applied (10M-row ceiling)
const open = await grok.functions.call('DomainQuery', {
  schema: 'grit', table: 'issue',
  filters: ['status = "open"'], orderBy: ['!created_on'], limit: 100,
});

// Aggregate mode: one row per group
const byStatus = await grok.functions.call('DomainQuery', {
  schema: 'grit', table: 'issue', groupBy: ['status'], aggregations: ['count'],
});

Note that only frames opened through the UI (Open in Table View, or a DomainQuery run from the console) record their generation script for project data sync; a frame fetched programmatically via grok.functions.call carries no creation script — save it as static data, or re-run the query in the UI to make it data-sync-ready.

Reads only ever return rows and columns the current user can see — there is no way to opt out client-side. Writes are validated, permission-checked, and audited server-side.

Batch upload and upsert​

batch loads large payloads efficiently (a million-row CSV registers in under a minute) and merges by the table's business key in upsert mode:

const report = await items.batch(csvString, {mode: 'upsert', allOrNothing: false});
// {inserted, updated, skipped, errorCount, rows: [{index, id, status, errors?}, ...]}

Accepted payloads: a DG.DataFrame, a CSV string, an array of row objects, or raw bytes (d42, or parquet converted client-side via the Arrow package). With allOrNothing: true (the default) any bad row aborts the whole batch; with false, good rows are applied and bad ones are reported per row.

Transactions​

Multiple operations — across tables, and across schemas when table is qualified as <schema>.<table> — commit or roll back atomically. An op can name its new row's id with ref for other ops to reference, in any order: the server runs the ops in dependency order (an op using $p after the op declaring p; a delete of a child table before a delete of its parent, so onDelete: restrict never vetoes what the same request removes) while the results array and the error's opIndex keep the request order:

await grok.dapi.domains.transaction('grit', [
  {op: 'insert', table: 'issue', values: {project_id: '$p', number: 1, title: 'First issue'}},
  {op: 'insert', table: 'project', ref: 'p', values: {key: 'GRIT', name: 'Grit'}},
  {op: 'insert', table: 'audit.event', values: {kind: 'project-created', subject: '$p'}},
]);

Add onDuplicate: 'error' to an insert op to fail the whole transaction (409) when the row's business key already exists, instead of merging it into the existing row.

Expansions​

Fetch related rows in one query: expand: ['project_id'] adds the master row's columns prefixed project_id.<name>; expand: ['details:comment'] adds capped child-row arrays (JSON queries only). The expanded table's own row and column security applies.

Facets and saved filters​

facets powers filter panels: one batched request computes category counts, histogram buckets, value ranges, the row count, and column profiles in a single round trip. Category counts, buckets, and count are computed under the passed filter with the conditions on each facet's own column stripped, so a filter control shows counts under all other filters (classic faceted search). The exception is the stable-axis rule: minMax, plan, and histogram bounds are computed under your access predicate only, ignoring filter — a narrowing filter never re-derives a filter's axis. All results respect row-level access and column security, so two users can legitimately get different counts for the same data.

const wells = grok.dapi.domains.table('plates.plate_well');

const res = await wells.facets({
  filter: [{property: 'sample_state', operator: '=', value: ['filled', 'dosed']}],
  facets: [
    // → {categories: [{value, display?, total, filtered}], hasMore}
    {id: 'state', kind: 'categories', column: 'sample_state', limit: 100},
    // Dotted reference path (up to 3 hops); groups by id, `display` carries the name
    {id: 'plate', kind: 'categories', column: 'plate_id.barcode', search: 'P-1'},
    // → {min, max, buckets, nulls}; datetime bounds are ISO-8601 strings
    {id: 'vol', kind: 'histogram', column: 'volume', bins: 24},
    // → {count}
    {id: 'n', kind: 'count'},
    // → {columns: [{name, distinct, min?, max?}]} — for choosing filter types
    {id: 'plan', kind: 'plan', columns: ['sample_state', 'volume']},
  ],
});
grok.shell.info(`${res.facets['n'].count} rows, buckets: ${res.facets['vol'].buckets}`);

At most 32 facets per request. Category lists are capped (default 100, hasMore set when more remain) — narrow with search (compiled server-side as a bound substring match) instead of raising the cap.

Saved filter presets are small shareable entities carrying the filter panel's state maps verbatim (the shape DG.FilterGroup saves):

const preset = await wells.filters.save('Filled wells',
  {'Sample state': {type: 'categorical', column: 'sample_state', selected: ['filled']}});
const presets = await wells.filters.list();  // presets visible to the caller, by name
await wells.filters.delete(preset.id);

Saving with {id} updates a preset in place, preserving its original author. Share a preset with users or groups like any other entity.

Typed clients​

grok api (already part of standard package build scripts) detects databases/*/schema.json and generates src/generated/db.ts with per-table row and insert interfaces, column-name unions, expand maps, a typed transaction union, and a lazy per-schema client:

import {gritDb, IssueStatus} from './generated/db';

const projects = await gritDb.projects.query({sort: 'name'});   // ProjectRow[]
await gritDb.issues.insert({project_id: projects[0].id, number: 7, title: 'Typed!'});
// gritDb.issues.insert({}) — compile error: required columns are enforced

Table properties on the schema client are the camelCase plurals of the declared table names (gritDb.issues, gritDb.issueLabels). The declared singular names stay everywhere data is addressed: table('grit.issue'), transaction ops, and expand keys.

The generated surface is truthful about the wire:

  • Datetimes are dayjs. Declared datetime columns and created_on/updated_on are typed Dayjs and materialize as dayjs objects on JSON reads, including expanded master fields and detail child rows (inserts also accept ISO strings). Untyped table('s.t') clients materialize dayjs too — datetime columns are resolved from the table's registry metadata, with no per-client configuration. Regenerating db.ts across this change is breaking — fix call sites that treated datetimes as strings (a.created_on.valueOf() instead of localeCompare).
  • choices columns are literal unions (IssueStatus = 'open' | 'in progress' | ...) used in both row and insert types — a typo in a status value no longer compiles.
  • Column names and expand keys are compile-checked through the client generics: filter conditions, columns, groupBy, and expand reject unknown names.

Fluent queries and bound conditions​

Bare query() returns an awaitable builder; query(spec) is unchanged:

const top = await gritDb.issues.query()
  .where('project_id', '=', projectId)
  .where({status: 'open'})                        // equality map, AND-combined
  .orderBy('number', true)
  .top(20);

const one = await gritDb.issues.query().where('number', '=', 7).first();  // row | null
const df = await gritDb.issues.query().where({status: 'open'}).df();      // typed DataFrame
const n = await gritDb.issues.query().where({status: 'open'}).count();

Condition values are bound server-side, never interpolated — any string value works, including apostrophes that the filter-string grammar cannot express:

await gritDb.projects.query().where('name', '=', "O'Brien's project");
await gritDb.issues.query({filter: DG.or(
  DG.cond('status', '=', 'open'), DG.cond('priority', '=', 'critical'))});

.expand('details:comment') types the child arrays into the awaited rows. .select(...) narrows the projection — system columns always ride along, and call it before .expand().

Typed errors​

Failures are DG.DomainError subclasses discriminated by class and code — never match message text:

try {
  await gritDb.issues.update(id, {status: 'resolved'}, {version});
} catch (e) {
  if (e instanceof DG.DomainVersionConflictError)
    grok.shell.info(`expected v${e.expectedVersion}, current v${e.currentVersion}`);
}

The family: DomainValidationError (per-row .rows, .isDuplicate), DomainVersionConflictError, DomainRestrictError, DomainFilterError, DomainForbiddenError, DomainNotFoundError, DomainManifestValidationError. A failed transaction carries .opIndex — the index of the failing op.

Optimistic concurrency​

const saved = await gritDb.projects.save({key: 'GRIT', name: 'Grit'}); // insert-or-update
await gritDb.issues.updateWithRetry(id, (fresh) =>
  fresh.status === 'open' ? {priority: 'high'} : null);   // null skips the write
await DG.retryOnVersionConflict(async () => {/* fresh read + transaction write */});

save addresses rows by identity — a business-key duplicate applies your values to the existing row under a versioned update. An idempotency-key replay applies nothing — the original insert already did — and resolves the existing row's fresh version. updateWithRetry re-reads and retries on conflict (default five retries after the initial attempt). Typed transactions get per-op result types from a tuple ops literal: const [upd, ins] = await gritDb.transaction([...]);.

Bulk delete​

const stamp = `test-${Date.now()}`;
while ((await gritDb.issues.deleteWhere(DG.cond('title', 'like', stamp + '%'))).hasMore);

Soft-deletes up to 1000 matching rows you may delete per call, oldest first, in one transaction. Declared referential actions run per row, and a restrict reference rejects the whole call with a DomainRestrictError. Never write per-row delete loops.

Schema lifecycle, grants, and watching​

await grok.dapi.domains.createSchema('inv', {friendlyName: 'Inventory'});
const handle = grok.dapi.domains.schema('inv');
await handle.apply({tables: {/* manifest fragment */}}, {dryRun: true}); // change plan
await handle.apply({tables: {/* manifest fragment */}});
const events = await handle.audit({limit: 50});  // row + ddl history, newest first
await handle.delete();                           // full purge

Table-scoped access control lives on the table client: grants(), grant(group, permission), revoke(group), and column security via shareColumn/restrictColumn/restoreColumnVisibility. Schema-entity grants (handle.grant(...)) gate schema operations — apply requires Edit, delete requires Delete, sharing requires Share — and do not grant access to row data. Grant per table for that.

watch()/unwatch()/isWatching() subscribe the current user to change notifications for a table or one row (row watch requires the table's audit trail). audit(id) reads a row's history, auditLog({limit}) the table-wide trail.

Samples​

Runnable in the platform's samples gallery: crud, typed-client, aggregate, batch, transaction, filters, dataframe, idempotency, trash, bulk-edit, hierarchy, schema, platform-grid.

From the command line​

The grok s CLI (npm install -g datagrok-tools) reaches the same server API without a browser, so scripts, CI jobs and one-off fixes can read and write domain tables with the same permissions and audit trail as the UI. A schema is addressed as <schema>, a table as <schema>.<table>:

grok s domains list                                            # registered schemas
grok s domains get grit.issue                                  # a table's columns
grok s domains query grit.issue --filter 'status = "open"' --sort '!created_on' --limit 20
grok s domains insert grit.issue title="Crash on save" status=open
grok s domains update grit.issue <row-id> status=closed --version 3
grok s domains upload grit.issue ./issues.csv --upsert         # csv, d42 or json; merge by business key
grok s domains download grit.issue -O ./issues.csv --filter 'status = "open"'
grok s domains grant grit.issue Chemists --access Edit
grok s domains create inventory && grok s domains apply inventory --json schema.json --dry-run

Validation errors are printed per row, a schema apply shows its change plan with --dry-run and refuses destructive changes until --confirm-destructive, and --output json makes every command scriptable.

The table UI, and its addresses​

Every registered table has a UI with no code, at stable addresses:

AddressOpens
/domains/<schema>/<table>the table — list, search, filters, ribbon
/domains/<schema>/<table>/<keyOrId>one row's page — its fields, its child rows, its history (the business key where it is unambiguous, the id otherwise)
/domains and /domains/<schema>the domain gallery and the schema diagram
Browse > Platform > Domains > <schema> > <table>the same table view as the first row

The table and row addresses open the u2 domain app (@datagrok-libraries/u2), which is what domains.table(address).app() gives a plugin — the same view, the same ribbon, the same gates, whether it is reached through /domains/..., through Browse, or mounted by a package at /apps/<Package>/<App>. Resolution goes through a //tags: domainRoutes package function (PowerPack:domainRouteView), so the platform loads it on demand and a stand without that package falls back to the built-in Dart view. Settings > Beta > Dart domain UI brings the frozen Dart domain UI routes back for a stand that needs them; the domain gallery and the schema diagram are Dart in either case.

What the app gives on top of browsing — trash and restore, bulk edit, a CSV/frame import wizard, a tree over a self-referencing table — is described in libraries/u2/docs/recipes/crud-app.md and hierarchies.md, and is the same surface a plugin composes from (see Building app UI: u2 below).

Customizing the UI​

The default UI (cards, tooltips, context panel, entity view) works for every table with no code. Plugins customize it through the standard mechanisms only — rows of each table carry the semantic type <schema>.<table>:

  • Custom rendering and views: register an ObjectHandler for the semantic type. Extend DG.DomainObjectHandler — override just what you customize, and everything else (cards, tooltips, grid decoration, context panel, permission-gated actions) keeps the platform defaults:

    class IssueHandler extends DG.DomainObjectHandler {
      constructor() { super('grit.issue'); }
      renderCard(x) { return ui.card(ui.divText(`#${x.values.number} — ${x.values.title}`)); }
      renderGrid(grid) { super.renderGrid(grid); grid.col('title').width = 300; }
    }
    DG.ObjectHandler.register(new IssueHandler());

  • Context actions: any package function with an input of semantic type <schema>.<table> appears in the row's Actions pane automatically.

  • Info panels: #panel-tagged functions with the same input appear in the Context Panel.

  • Search patterns: claim identifier patterns (like GRIT-123) in the handler, and they resolve from global search.

Building app UI: u2​

To build your own UI over domain tables — forms, lists, editable grids, whole browse/CRUD apps — use the domain controls of @datagrok-libraries/u2 (a relative-path dependency, "@datagrok-libraries/u2": "../../libraries/u2", wired the way the Stockroom package is). Everything in it is reflective: the controls take columns, labels, choices, validation rules and permissions from the runtime registry, so an app is one await and one line:

import {domains} from '@datagrok-libraries/u2/src/dg/index.js';

const issues = await domains.table('grit.issue');        // schema + capabilities, one round-trip
grok.shell.addView(issues.app());                         // list ⇄ entity page, URL, ribbon, gate

Three tiers, each a package in this repository:

  • Zero code — t.app() from schema.json alone: Stockroom. grok add app --domain <schema>.<table> scaffolds it.
  • Configuration — the same app as a dg-ui/1 spec with the u2-domain-* tags, edited in the designer: Stockroom/src/app.spec.json.
  • Code — grok api --ui generates typed handles (getGritDb() → a DomainTable<IssueRow> per table, one await) on which the app declares actions, validators, a card renderer and a DomainApp subclass with presets and shortcuts: Grit.

The recipes in libraries/u2/docs/recipes/ walk through the surface: crud-app.md (what the schema declares, what app() gives, and the ⋯ menu — import, bulk edit, trash/restore), spec-app.md, custom-app.md, and hierarchies.md (a self-referencing table as a tree, the under subtree filter). What every backend behind these controls must agree on is libraries/u2/docs/domain-backend-contract.md.

Platform building blocks​

The lower-level blocks behind the standard UI ship in datagrok-api itself:

  • DG.DomainView.create({schema: 'grit', table: 'issue'}) opens the Dart table view (search, filters, render modes, editing) programmatically — the view the /domains/... routes open with Settings > Beta > Dart domain UI on. To open what those addresses normally open, use the u2 app: (await domains.table('grit.issue')).app().
  • grid.attachEditor(editor) hosts the domain editing state in any DG.Grid you own — dirty / invalid / conflict cell markers and per-column writability — and DG.DomainObjectHandler.decorateGrid(grid, table) applies the table's rendering to it.
  • DG.DomainFrameEditor.attachTo(df, schema, table) attaches the same editing state machine to a DataFrame you render yourself — for fully custom hosts; several editors save as ONE transaction through new DG.DomainSession(editors).save(), which owns the conflict and validation flows.

See the platform-grid sample.

See also: