# Legend Cube — Implementation Plan > **Status:** approved 2026-10-05 · branch `cubeV1` (rebased on master `a32e5c0fb`, where the spec landed as > `docs/design/WIP-CUBE-SPEC.md` in #5589) > **Inputs:** [docs/design/WIP-CUBE-SPEC.md](../../../docs/design/WIP-CUBE-SPEC.md), the planning brief, and an investigation of > `legend-studio` + `legend-engine` (HEAD `93d92b4`) with ~1,500 checks against a live engine on `localhost:6300`. > **Evidence markers:** ✅ verified live against the engine · 📄 traced in code · 💭 inference, to be verified in the > milestone that needs it. > **Implementation status** is tracked in [PROGRESS.md](PROGRESS.md), not here. --- ## 0. Decisions | # | Decision | Source | | --- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------------- | | D1 | Saved cubes will live in a **new dedicated Cube store** in `legend-engine-application-query`. **Not in v1.** v1 defines and tests the saved spec format (codec + round trip) only; actual saving and the engine store come after the POC works end to end. | user | | D2 | The slice runs **locally** against a **Cube-owned Northwind fixture sent as an inline model** (no depot). Entry points and the sources modal get expanded later, once designed. v1 scope: one model, one Database element, one runtime per query. | user (default) | | D3 | **ag-grid Enterprise license is available** in every deployment. Use `@finos/legend-lego/data-grid` (enterprise modules). | user | | D4 | NULL semantics: **joins use SQL semantics** (NULL keys never match); **negated filters include NULL rows** (made explicit by the emitter, §8.4: the engine does it only for columns it types `[0..1]`; documented in the UI); **Count = non-null count** of the column. | user (default; wording 2026-10-06) | | D5 | Engine-driven changes to authoritative sections are accepted: Slice is `[start, stop)`; Join gains **FULL OUTER (in the slice)**; window aggregates use SQL's default frame, none saved or written: running with a sort (from the partition's first row to the current one, ties sharing a value), the whole partition without one (M5, §11.6 Q1); Difference keeps the spec's rows (emulated), with native types (M6, §11.7 Q5); Concat across different precise types is rejected, a type next to its ancestor too (the engine accepts it ✅); Convert types casts within numbers, strings or dates (M4, §11.5 Q5). | user (default) | | D6 | Post-slice source order: services → Pure functions → data products → ingest (data products and ingest moved to M3, §6.7–6.8). Data products and ingest are built against mocks until a lakehouse-enabled engine is available. Services snapshot their converted lambda and check for drift. | user (default) | | D7 | Route **`/cube`** inside Legend Query (URL `/query/cube`), hard-wired in the Query router. New module(s) `legend-cube` / `legend-cube-builder` (§3). Further entry points, the sources modal and the final look are revisited in M3. | user + recommendation | | D8 | Cube **works around** Studio and engine defects in its own code and depends on none of them being fixed. Upstream fixes are separate, non-blocking PRs and issues (Appendix B). | user (default) | | D9 | Execution is a **Pure relation-function chain** over store accessors (`#>{db.schema.table}#`), built as **protocol JSON** (never Pure text). Legend SQL is only a possible future "SQL source" node. | recommendation (§8.1) | | D10 | Precise primitives are modeled **inside the host-free domain**. The host adapts the engine's relation-type JSON at the boundary, in a package-local `v1/` folder. | recommendation (§5) | | D11 | **No feature flag.** `/query/cube` is always mounted in Legend Query. (M1.0 first shipped a `TEMPORARY__enableLegendCube` option; it was removed the same day.) _(2026-10-10, reversed by the user: Cube is still a proof of concept, so `/query/cube` mounts only where the config sets `extensions.core.NonProductionFeatureFlag`, Query's option for features not ready for production. It is set by hand in a local `dev/config.json`, not by `yarn setup`, since it also changes the rest of Query.)_ | user | | D12 | **Types: Cube's own registry for the slice, legend-graph's types from M2.0**, for consistency with the rest of Legend. M2.0 first fixes legend-graph's precise primitives (own PR), then rebases `CubeType` on legend-graph's `GenericType` and narrows the core rule to "metamodel only, no `V1_*`, no UI or app packages" (a §2.2 departure). Until then the type seam stays narrow (§4.1) so the switch stays internal. Replaces D10 from M2.0. | user + recommendation | | D13 | **First merge after M1.8a, as one PR** (2026-10-07), so new sources and operations can then be built in parallel. M1.8b (canvas and editors) joined the same PR before it merged (user, 2026-10-07: it is on the critical path). Show Pure's "numbers as 0" bug is fixed before it. The working docs live in `docs/wip/legend-cube/` (PLAN, PROGRESS, and ISSUES for the known issues later PRs fix); the legend-graph issue list stays out of the repo. | user | --- ## 1. Scope ### 1.1 Milestone 1: the slice Thin vertical slice, working live on a developer machine: - **Sources:** relational database tables from an inline model (Cube Northwind fixture, or a pasted Pure model). - **Transforms:** **Join** (Inner, Left Outer, Right Outer, Full Outer; binary from day one) and **Filter**. - **Types:** full precise-primitive support (`Varchar(n)`, `SmallInt`, `Numeric(p,s)`, `Timestamp`, …). - **Path:** pick tables → canvas → live schema inference and validation → lambda → live execution → results grid → saved-spec codec (export/import JSON as a dev affordance; no store). - **Hosting:** Legend Query route `/query/cube`, mounted only with `NonProductionFeatureFlag` (D11, since 2026-10-10). ### 1.2 Not in the slice Every other transform (Sort, Group, Restrict, Rename, Distinct, Drop, Limit, Slice, Concat, Difference, Partition, Extend). The Join rename autofix (it needs the Rename node; it lands in M2). Every other source kind. The depot and project catalog. Persistence to any server. Drill-down, server-side grid mode, export, column formatting. Publishing (§15 is out of scope altogether). Legacy V0 import (none, ever). ### 1.3 After the slice Section 11.3 lists the milestones, in the recommended order. --- ## 2. Architecture overview ``` legend-application-query (Legend Query app; hard-wired route /cube; implements CubeHost) │ depends on ▼ @finos/legend-cube-builder (React UI, MobX state, Legend engine adapter in src/graph-manager/protocol/pure/v1/, │ source catalogs, grid, canvas) │ depends on → legend-graph, legend-application, legend-art, legend-lego, legend-shared, ▼ @xyflow/react, @dagrejs/dagre, react-dnd, mobx, ag-grid (via legend-lego) @finos/legend-cube (HOST-FREE core: types, schema, query graph, inference + validation, transforms, filters, Cube IR + emitter, saved-spec codec). Zero @finos deps; no React, no MobX. ``` The four artifacts and how they flow: ``` decode/encode (core) emit (core) v1 seam (builder) engine CubeSpec JSON ◄────────────────────► Query graph ─────────────► Cube IR ─────────────────► protocol JSON ──► execute (saved, v1) (truth, immutable) (host-free AST) lambda + SI stamps lambdaRelationType │ │ └──── UI state (builder; mostly derived, never the source of truth) ◄──────┘ rows, types, errors ``` - The **query graph** is the only source of truth. Everything else is derived from it, or round-trips through it. - The **executed lambda** is derived from the graph. Since §11.18, Import Pure also reads a lambda back into a graph: one canonical cube per lambda, which then holds the truth again. - **UI state** is derived (layout) or ephemeral. Only presentation metadata is saved (§10.4). --- ## 3. A. Code organization ### 3.1 Packages | Package (folder) | Purpose | Dependencies | Template to copy | | ------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------ | | `@finos/legend-cube` (`packages/legend-cube`) | Host-free core. Spec §2.2 `cube-domain`, plus the IR emitter and the spec codec. | **none** at runtime. Dev: `@finos/legend-dev-utils`, `@jest/globals`, `jest`, `typescript`, `eslint`, `rimraf`, `npm-run-all`, `cross-env`. | `packages/legend-storage` (headless) | | `@finos/legend-cube-builder` (`packages/legend-cube-builder`) | UI and Legend adapter. Spec §2.2 `cube-engine` + `cube-sources` + `cube-ui` + `cube-persistence`, as folders. | `@finos/legend-cube`, `legend-graph`, `legend-application`, `legend-art`, `legend-lego`, `legend-shared`, `@xyflow/react@12.4.4`, `@dagrejs/dagre@1.1.4`, `react-dnd@16.0.1`, `mobx@6.13.6`, `mobx-react-lite@4.1.0`, `react@19.0.1` (peer) | `packages/legend-data-cube` (UI lib) | The dependency lists are the target. Each step adds the dependencies it first uses, so nothing is declared unused: M1.0 declares only `@finos/legend-cube`, React and React DOM in the builder. Why two packages and not five (spec §19.3): - The rule that the core has **no host imports** is the spec's most important structural rule (§2.2). A package boundary enforces it best. - Each extra package costs about 11 config files, root edits, a changeset and a release line. Engine, sources, UI and persistence are folders inside `legend-cube-builder` and can be split later if needed. **Who depends on whom.** - `legend-application-query → legend-cube-builder → legend-cube`. - **`legend-query-builder` does not depend on Cube in v1**, for three reasons: 1. Routes live in `legend-application-query`. The query builder is a library with no routes. 2. `legend-query-builder` is consumed by Studio, the Data Cube app, data-space and others. They would all inherit the canvas stack. 3. Keeping the edge out leaves `legend-cube-builder` free to reuse query-builder pieces later (parameter value editors, the saved-query loader for a "Legend Query" source). That would be a cycle if the query builder depended on Cube. - If "Open in Cube" from the query builder (inside Studio) is wanted later, add the edge then, as `legend-query-builder → legend-data-cube` already does. `legend-cube-builder` must never import `legend-query-builder` after that. - No library package depends on `legend-application-*` (AGENTS.md). ### 3.2 Folder layout ``` packages/legend-cube/src/ types/ CubeType, type registry (precise primitives), families, comparison classes, display values/ LiteralValue, coercion + validation per type schema/ SchemaColumn (name, type, nullable), Schema graph/ QueryNode, Connection, Query (invariants, operations, ids), CubeDocument (context + query + meta) inference/ buildSchemasAndValidity, validation combinators, sentinels, query-level rules nodes/ registry; sources/RelationalTableSource; transforms/Join, Filter; UnknownNode filter/ Filter tree, operators, availability matrix, builder helpers (§8.5) messages/ §16 catalogue (verbatim) + additions ir/ Cube IR (host-free Pure AST), emitter per node, join algorithm, debug printer spec/ CubeSpec v1 codec, Meta, rest-preservation, node codecs; the migrations are a list in CubeSpecCodec (CUBE_SPEC_MIGRATIONS, empty in v1) utils/ internal helpers, e.g. the exhaustive-switch assertion index.ts packages/legend-cube-builder/src/ graph-manager/CubeEngine.ts CubeEngine port + CubeModelOutline / CubeResult / CubeEngineError (no V1_* symbols) graph-manager/CubeModelOutlineHelper.ts getRuntimesForDatabase: the runtimes keyed by exactly that database (§6.2.5) graph-manager/protocol/pure/CubeEngineBuilder.ts buildCubeEngine(config, tracerService): CubeEngine. The ONLY place that constructs V1_LegendCubeEngine (precedent: QueryBuilder_PureGraphManagerExtensionBuilder.ts) graph-manager/protocol/pure/v1/ V1_CubeLambdaSerializer (IR → protocol JSON + sourceInformation stamps), V1_CubeRelationTypeAdapter (relation-type JSON → CubeType), V1_CubeModelOutlineBuilder (parsed model → databases, flags, runtimes), V1_CubeExecutionResultReader (lossless), V1_CubeEngineErrors (payload → node error), V1_LegendCubeEngine (implements the port; builds its own client; imports only the port, legend-graph, legend-shared and @finos/legend-cube) stores/ CubeEditorState, CubeExecutionState, CubeNodeEditorState (the floating node editor), CubeSourcePickerState, CubeShowPureState, CubeSpecTransferState (Export/Import), LocalModelCatalog (the bundled model texts; talks only to the port; loadModel parses a model context once and returns its databases and runtimes as plain data), CubeHost interface, editors/ (node drafts and their registry, §7.4), fixtures/ (Cube Northwind model as a TS string) components/ CubeEditor (layout), CubeButton, CubeNodeIcon (node icons), canvas/, palette/, editors/ (Join, Filter, Source, and the editor registry), source-picker/, grid/, show-pure/, spec-transfer/ __lib__/ labels, help text (§17.9), command config (§3.5), test ids __test-utils__/ Cube-local axios engine helpers for engine-backed tests (§3.4), the fake engine, page and canvas helpers for jsdom tests packages/legend-cube-builder/style/index.scss built to lib/index.css, which the host imports ``` **The repo's `@finos/legend/enforce-module-import-hierarchy` lint rule** (error level) forbids imports in **both** directions between `stores/`/`components/` and `graph-manager/protocol/*/v*/` ([enforce-module-import-hierarchy.js](../../../packages/eslint-plugin/src/rules/enforce-module-import-hierarchy.js)) 📄. That is why the port sits in `graph-manager/`, the factory sits just outside `v1/`, and `stores/` and `components/` never import from `v1/`. The Query host calls `buildCubeEngine` from the builder's index, so no `V1_*` symbol appears in `legend-application-query`. ### 3.3 Guarding the core's purity The core gets four layers of protection, because `tsc` resolves _undeclared_ workspace packages through the root `node_modules/@finos` symlinks 📄. All four have been in place since M1.0 ✅. Each was checked with a probe file that imports `mobx` and reads `window`: lint, the build and the test all reject it. 1. **No dependencies.** `packages/legend-cube/package.json` declares no runtime, peer or optional dependencies. Its only dev dependencies are tooling: `@finos/legend-dev-utils`, Jest, TypeScript, ESLint and the script helpers. The test in item 4 checks this. 2. **Relative imports only (ESLint).** A root `eslint.config.js` block covers the non-test files under `packages/legend-cube/src/**`. - `no-restricted-imports` rejects every specifier that doesn't start with `./` or `../`. That is stricter than a list of host packages and needs no upkeep. - `no-restricted-globals` flags `window`, `document`, `localStorage`, `sessionStorage`, `navigator`, `location`, `fetch`, `XMLHttpRequest`, `process`, `console`, `setTimeout`, `setInterval` and `structuredClone`, for feedback while typing. 3. **ECMAScript globals only (build).** `packages/legend-cube/tsconfig.json` sets `lib: ["esnext"]` and `types: ["node"]`, because the tests need Node. `tsconfig.build.json` also sets `types: []` and leaves out the test folders. - So `yarn build` fails on **any** browser or Node global outside tests, not just the names above: `URL`, `TextEncoder`, `crypto`, `queueMicrotask` and the like are out too. - The core needs none of them. Write plain ECMAScript instead; for example, copy values without `structuredClone`. 4. **A unit test** (`src/__tests__/LegendCubeHostFree.test.ts`) runs in CI even when lint is skipped. - It parses each non-test source file and rejects any module reference that isn't relative: static imports and re-exports (type-only ones too), `import()`, `require()`, `import x = require()` and triple-slash directives. - It compiles the non-test sources against the ECMAScript library with no ambient types, and checks the package's dependencies. - Each check also runs against small bad fixtures, so the test proves it can fail. ### 3.4 Build, test and lint conventions What the repo actually enforces: - **Per package:** `package.json`, `tsconfig.json`, `tsconfig.build.json`, `tsconfig.publish.json`, `_package.config.js`, `jest.config.js`, `.npmignore`, `README.md`, `CHANGELOG.md`, `src/index.ts`. The builder also needs `style/index.scss` and the `build:sass`/`dev:sass` scripts. - **Root:** both packages are referenced in `tsconfig.json` and `tsconfig.build.json`. Each dependent package references them in both of its tsconfigs. `yarn check:ts` only sees a new package once it is `git add`-ed 📄. - **Tests:** - Core: `getBaseJestProjectConfig` (node environment). Builder: `getBaseJestDOMProjectConfig`. - Engine-backed tests are named `*.engine-roundtrip-test.ts`. CI runs them against the docker engine. - `fetch` is blocked in Jest (`legend-dev-utils/jest/blockFetch.js`), and `V1_EngineServerClient` uses `fetch`. So engine tests use axios against the hard-coded `http://localhost:6300/api` ([EngineTestSupport.ts](../../../packages/legend-graph/src/graph-manager/__test-utils__/EngineTestSupport.ts)) 📄. - **How the engine tests are wired:** - Engine tests drive the real `V1_LegendCubeEngine`. They spy its `V1_EngineServerClient` methods and route them to Cube-local axios helpers in `legend-cube-builder/src/__test-utils__/`, with `axios` as a devDependency as in legend-graph. The precedent is [LegendDataCubeStoreTestUtils.tsx:470-503](../../../packages/legend-application-data-cube/src/components/__test-utils__/LegendDataCubeStoreTestUtils.tsx#L470). - Cube-local helpers are needed because `EngineTestSupport` has no `/lambdaRelationType/batch` helper, and its `execute` helper returns already-parsed JSON. That would bypass the lossless result reader M1.7 must test. - The helpers cover: - `grammarToJson/model` and `grammarToJson/lambda` (the A.11 golden and the printIR-parse comparison); - `lambdaRelationType/batch` (a plain JSON POST); - `compilation/compile` (the corpus compile check; a `text` context works ✅); - `jsonToGrammar/lambda`; - `GET /api/server/v1/info` (the engine commit, logged by Part A); - `execute` (`responseType: 'text'`), returning `{ ok: true, status: 200, text: async () => body }` so the lossless reader runs exactly as in the browser. - **CSS:** - From M1.8b, the canvas imports `@xyflow/react/dist/style.css` from its `.tsx` file, as Studio's database diagram does, so the stylesheet loads with Cube's lazy chunk and not in Query's global sheet: xyflow 12 and the lineage viewer's reactflow 11 share the `.react-flow__*` class names (decided in M1.8b S13). - Legend Query's bootstrap `style/index.scss` imports `@finos/legend-cube-builder/lib/index.css`. - Tailwind only scans `../legend-*/src/**/*.tsx`: the package folder must start with `legend-`, and Tailwind classes go in `.tsx` files 📄. - **Release hygiene:** - New files carry `Copyright (c) 2026-present, Goldman Sachs` (from the template, not copied from a neighbour). - New packages start at `0.0.1` with a `patch` changeset. `legend-application-query` also gets a `patch` entry. - **Not enforced:** - `yarn constraints` is a no-op: the rules sit in `.yarn/constraints.pro`, which Yarn 4 does not read 📄. Version alignment is by convention, so pin the exact versions already in `yarn.lock` (§3.6). - There is **no lint rule for package dependency direction** 📄. §3.3 covers the core. ### 3.5 Legend Query integration (slice) - **Route:** - `CUBE: '/cube'` is in `LEGEND_QUERY_ROUTE_PATTERN` ([LegendQueryNavigation.ts:69](../../../packages/legend-application-query/src/__lib__/LegendQueryNavigation.ts#L69)) and mounted in [LegendQueryWebApplication.tsx](../../../packages/legend-application-query/src/components/LegendQueryWebApplication.tsx#L142) since M1.0. M1.8a only swaps the route element for a Query-side wrapper that builds the host, loaded lazily (Settled before M1.8). - Query's `baseUrl` is `/query/`, so the URL is **`/query/cube`** 📄. - Plugin page entries are not used: they force an `/extensions/` prefix 📄. - `/cube/:cubeId` is reserved for saved cubes (M8). - **No feature flag** (D11): the route is always mounted. _(2026-10-10: it now mounts only with `extensions.core.NonProductionFeatureFlag`, D11.)_ - **Keyboard shortcuts:** Legend binds keys only through plugins' `getExtraKeyedCommandConfigEntries()`, collected at app start 📄. F9 is already bound in Query to the query builder's compile command, which is only registered while the query builder is mounted. - The builder exports `LEGEND_CUBE_COMMAND_CONFIG` (execute: F9; undo: Control+KeyZ / Meta+KeyZ). - Query contributes it via `getExtraKeyedCommandConfigEntries()` on a core application plugin. This is the only plugin hook Cube uses. Query has two: `Core_LegendQuery_LegendApplicationPlugin` (`src/application/`) and `Core_LegendQueryApplicationPlugin` (`src/components/`); before M1.8b neither overrode the hook, and Query's test helper installs only the second. **M1.8b (S20) added the override to `Core_LegendQuery_LegendApplicationPlugin`**, the one the app installs first (a new Query key binding must be merged into that override's entries), and a Query test shows that F9 on `/cube` reaches Cube's Execute with the query builder's plugin also binding F9 (the query builder's Compile is registered only while it is mounted). - The Cube page registers the commands with `applicationStore.commandService`, as the query builder's `useCommands` does. - The undo command's `trigger` returns `false` while focus is in an input, textarea or contenteditable. - **Hosts** (§11.19, folded in at AN.6): Legend Query (`/query/cube`) and Legend Analytics (`/analytics/`, its own application, `@finos/legend-application-analytics`) host the page through one `LegendCubeHost` in the builder, built from a plain `LegendCubeHostConfig` of the deployment's servers (the engine; the depot, lakehouse, platform and marketplace where it has them), so both offer the same sources by the same rules. Each app keeps only its config, its shell and its shortcuts' plugin; Analytics also copies the rules of Query's light mode that the page's legend-art components use. Studio's embedded cube (#5690) keeps a host of its own. - **Host code:** `legend-application-query/src/components/cube/` and `src/stores/cube/` hold a thin `LegendQueryCubeHost`, now a subclass of the builder's `LegendCubeHost` given Query's config (`buildLegendQueryCubeHostConfig`). It implements `CubeHost` (defined in the builder) from Query's application store: - engine server client config (Query's `engineServerUrl`, as at [QueryEditorStore.ts:629](../../../packages/legend-application-query/src/stores/QueryEditorStore.ts#L629)); - notifications and alerts; - telemetry; - the bundled model catalog. The engine URL exists only on Query's config 📄, which is why the host injects it. - **Entry links** (landing-page action, editor menu item, deep links) are **not** in the slice. They are designed in M3. - **Look (provisional):** - Legend Query's look: legend-art components, resizable panels, `ContextMenu`, `Dialog`. - Tailwind utilities for layout and legend-art colour tokens. Root CSS class `.legend-cube`; the `.data-cube` class and the cube icon belong to Data Cube. - Components stay thin so the look can change in M3. ### 3.6 UI libraries (all already in `yarn.lock`; no new third-party packages) | Need | Library (version) | Precedent | | ------------- | ---------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | Canvas | `@xyflow/react` 12.4.4 (MIT) | Studio database diagram [DatabaseDiagramCanvas.tsx](../../../packages/legend-application-studio/src/components/editor/editor-group/database-editor/DatabaseDiagramCanvas.tsx) | | Layout | `@dagrejs/dagre` 1.1.4 (MIT) | [DatabaseDiagramHelper.ts](../../../packages/legend-application-studio/src/components/editor/editor-group/database-editor/DatabaseDiagramHelper.ts) (left-to-right, centre → top-left conversion) | | Drag and drop | `react-dnd` 16.0.1 (MIT); `DndProvider` already wraps every Legend app | query builder, legend-lego | | Grid | ag-grid 35.0.0 via `@finos/legend-lego/data-grid` (enterprise, D3) | [DataGrid.tsx](../../../packages/legend-lego/src/data-grid/DataGrid.tsx) | | State | `mobx` 6.13.6, `mobx-react-lite` 4.1.0 | repo-wide | Not used: `reactflow` 11 (legacy, lineage viewer only), elkjs (not installed; EPL), Data Cube's floating-window layout manager. ### 3.7 Housekeeping found along the way (separate, optional) - [AGENTS.md:120](../../../AGENTS.md#L120) says DataCube consumes metamodel only, but `legend-data-cube` uses `V1_*` throughout, outside `v1/` folders 📄. Cube follows the stricter rule: V1 symbols appear only under `legend-cube-builder/src/graph-manager/protocol/pure/v1/`, as `legend-query-builder` does. AGENTS.md should be clarified separately. **Test exception** (user, 2026-10-08): the engine-backed tests in `legend-cube-builder/src/__tests__/` (`LegendCubeNorthwind`, `CubeSpecCorpus` and `CubeEditorState` `.engine-roundtrip-test.ts`) import `V1_*` there. They need both the `v1/` adapter and `stores/` (the Northwind fixture, and `CubeEditorState` itself), and the import-hierarchy rule (§3.2), which lints tests too, forbids `v1/` from importing `stores/`. Product code keeps the rule. - `.yarn/constraints.pro` is dead; see Appendix B. --- ## 4. Domain model (`@finos/legend-cube`), detailed for the slice Everything in this section is host-free and test-driven. Spec sections are cited where behaviour is kept **verbatim**. M2's unary operations (§11.4) are folded in: the column-name rule in §4.2, row order in §4.4, the nodes in §4.5, the Join autofix in §4.7, the grid's Filter by rule in §4.8 and the new messages in §4.11. M4's Group and Concat (§11.5) are folded in the same way: output names and converted types in §4.2, port labels in §4.3, row order in §4.4 and the nodes in §4.5. ### 4.1 Types See §5 for the rationale. In summary: ```ts type TypeFamily = | 'BOOLEAN' | 'STRING' | 'INTEGER' | 'FLOAT' | 'DECIMAL' | 'NUMBER' | 'STRICT_DATE' | 'DATETIME' | 'DATE' | 'STRICT_TIME' | 'VARIANT' | 'ENUM' | 'OPAQUE'; abstract class CubeType { abstract readonly path: string; equals(o: CubeType): boolean; toString(): string; } class PrimitiveType extends CubeType { path; params: readonly number[]; info: PrimitiveTypeInfo; } // interned by (path, params) class EnumType extends CubeType { path; values: readonly string[]; } // equal iff same path (spec §3.1) class OpaqueType extends CubeType { path; params: readonly number[]; } // unknown engine types; carried, never compared ``` - `PrimitiveTypeInfo` comes from a static registry (§5.3) and holds the short name, family, parent, parameter arity and integer range. - Primitives are interned singletons per `(path, params)`, which keeps spec §3.1's "singleton" intent. `Varchar(5)` and `Varchar(40)` are distinct values. - Enum value qualification helpers stay as in §3.1 (`qualifyEnumValue` / `unqualifyEnumValue`). - **Equality is structural** (spec §3.1): same family and same full name, which for an enumeration is its path. Type guards go by the family, never `instanceof`, so types also work across copies of the module. - **The seam (D12).** Code outside `types/` uses only the type API: `family`, `equals`, `displayName`, `fullName`, `path`, `params`, the type guards, and the functions in `types/` and `values/`. It never reaches into `info` or builds types ad hoc: types come only from `resolveCubeType`, `PrimitiveType.get` and `new EnumType` (in the engine adapter, the spec codec and test helpers). That keeps the M2.0 switch to legend-graph's types internal to `types/`. ### 4.2 Schema - `SchemaColumn { name: string (non-empty); type: CubeType; nullable: boolean }`. **`nullable` is new**: outer joins and later aggregations need it, and the engine reports it wrongly (§4.7, §5.7). - `Schema` keeps the §3.2 operations: `lookup`, `type`, `names`, `equals`. - `equals` is order-sensitive and compares names and types. Nullability is ignored, matching the engine's `concatenate` ✅. - Concat's output (M4) is therefore schema1 with `nullable = nullable1 || nullable2` per column, as the engine merges it ✅. With Convert types, a column whose types differ takes their least common ancestor (§11.5 Q5). - **New:** a schema asserts unique column names, because the engine rejects duplicates everywhere ✅. - Source schema **drift detection** compares name, type **and** nullability, not `equals`. - **Column names** (M2, `schema/ColumnName.ts`; why in §11.4): `isValidColumnName` replaces spec §7.5's `/^[A-Za-z0-9_ ]{1,100}$/u` (Appendix A). A name is not empty, is trimmed, has no `"`, no `\`, no backtick (Databricks and MemSQL quote names with backticks, and the engine escapes none: M6d, §11.14) and no control character, and has at most 128 code points (`MAX_COLUMN_NAME_LENGTH`). Rename and the Join autofix (§4.7) use it, and so do Group's output names (M4); Extend will. `foldColumnName` (NFKC, then case folded through upper case, so `ß` meets `SS` and fullwidth letters meet ASCII) is close to how SQL Server, MemSQL and DuckDB compare names, so Cube never gives a column a name that folds to another column's: a Rename's new names, the autofix's names, Group's output names and the temporary columns (`cube_rn`, `cube_d`, and Convert types' `cube_cast`) are all checked folded. ### 4.3 Query graph Spec §4 is kept verbatim (`QueryNode {key, id, type, ports}`, `Connection`, `Query {nodes, connections, selected}`, the five invariants, every operation and every `canX` predicate), plus the following fixes and extensions: | Change | Why | | ------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | **Acyclicity invariant (6)** | §5.1 says cycles are impossible. They aren't: `connect(F1,F2)` then `connect(F2,F1)` satisfies invariant 3 📄. `canConnect` and `connect(…, port)` reject a target that is the source or upstream of it. `canMove` is **unchanged** from §4.4: move isolates the node first, so it cannot create a cycle, and a stricter rule would forbid moves the spec allows. The constructor asserts the graph is acyclic; `visit()` keeps an in-progress set. | | **`connect(source, target, port?)`** | The canvas lets users drop on a specific Left or Right handle. With no port, behaviour is spec's "first free port". | | **Per-type `generateId`** | §4.4's global max cannot produce Appendix C's ids (`join101` + `filter101`) 📄. Take `max(100, ids of nodes of that type) + 1`; the collision fallback is unchanged. | | **Port labels in metadata** | §17.3 pitfall 4. Join's `portLabels` getter, `['Left', 'Right']` (inherited from `BinaryNode`); Concat's `['First', 'Second']` (M4, §11.5 Q8). | Settled in M1.2 (spec §4.4 leaves these open or assumes unary nodes): - **Port invariant (7):** a connection must target one of its target's ports. Port capacity stays out of the constructor, as in the spec. - **Healing** (`remove`, `move`): the node's first connected input, in port order, feeds its output's target on the same port. A binary node's other input is dropped. - **`move`** heals like `remove` before splicing, since the spec only says "disconnect". Moving a node after the selected node selects it, as `add` does; otherwise the selection is unchanged. - **Nodes that don't accept new inputs** (`acceptsNewInputs`, false for Unknown, §4.10): `canAdd(…, after)`, `canMove`, `canConnect(*, it)` and `canSwapInputs` are false. Changes that keep its ports are allowed: healing into it on removal, and splicing a node in front of it. - **`replace(node)`** swaps the node with the same id for a new node object (spec §4.1's key check) and needs every port in use. Every operation asserts its `canX` and throws. - **`generateId`** counts only ids that are the type followed by digits, for nodes of that type (stricter than the spec's lenient `parseInt`). Past safe integers it falls back to the scan, which covers 1–10000. - **Undo** creates a new object identity, as §17.4 requires: `query.clone()`. - **`swapInputs`** also applies the node's `withSwappedInputs()`, so settings that name inputs by side follow them (added in M1.3 for Join, §4.7). The node it gives must keep its id. - `CubeDocument` holds `{ context, query, meta }`, where `context = { model: ModelContext; runtime?: string }` is query-level (§6.2), and the model is the engine's model context as plain JSON (§6.2.2). ### 4.4 Inference and validation engine - §5 is kept verbatim: `buildSchemasAndValidity`, port-ordered input schemas, the three sentinel messages, `isSchemasError`, and the `validate` / `validateAllItems` combinators. - Two additions: - **Query-level rules pass** (new; runs before the per-node pass, and its errors are added to the offending nodes). v1 has one rule: all relational sources must address the same Database element as the first source. Message: `Sources from different databases are not supported yet; "" differs from "".` - **Host-supplied issues** (engine errors mapped back by node id, §8.7) live in a separate map and do not take part in schema propagation. The UI merges them for display. - Settled in M1.2: - A node with a rule error is invalid, so it has no schema and downstream nodes report `ERR_SCHEMAS`. Rule errors come first in its list, before a sentinel or its own errors. The rule takes the first relational source in `query.nodes` order, and every relational source counts, connected or not. - A node that validates but gives no schema gets `ERR_OTHER`, the converse of "invalid ⇒ no schema". - `isIncompleteError` joins `isSchemasError` (which stays `ERR_SCHEMAS` only). - Settled in M2 (`inference/RowOrder.ts`; why in §11.4): - **Row order.** A `RowOrder` lists `OrderKey {column, direction, sortId, keyIndex}`, most significant first: the column as named at that node, and the Sort that declared the key with its place among that Sort's keys. It is `[]` when nothing orders the rows and `undefined` when that can't be known. `computeRowOrders(query)` gives every node's order from its inputs' through `outputOrder` (§4.5): a source, a Join, a Group and a Concat give `[]`; Filter, Distinct, Limit, Drop, Slice and Partition (M5) pass their input's on; Rename renames its keys; Restrict keeps the longest prefix whose columns it keeps; Sort puts its own keys first, then the input's keys on its other columns; an Unknown node gives `undefined`, which the nodes that pass an order on carry down. When running a query, the emitter hands a node that `consumesInputOrder` (Limit, Drop, Slice) its input's order, written as a sort just before it or in the row numbers' `over()` (§8), and sorts the capture by its own before the run's limit; typing lambdas carry no order. Orders are derived, never stored. - **Lost orders.** `findLostSortOrders(query, rowOrders?, schemas?)` follows each Sort's order down its chain to the first node that consumes it, or to the chain's end, and maps the Sort's id to the loss, if any. A full loss names the first node whose rows hold none of its keys, e.g. a Join, a Group, a Concat (on either input), a Restrict that drops the Sort's first key, or a later Sort on all the same columns. A partial loss names the first node that loses some keys, and gives the nodes (Restricts) that removed some (`removals`) and the kept columns that no longer order the rows, as they came after a removed one (`cutColumns`, which needs `schemas`). A later Sort on some of the same columns is no loss, and nothing is reported where the order becomes unknown. The builder shows losses as warnings (§4.11), never validation errors, so Execute stays enabled. ### 4.5 Node contracts and registry This replaces the per-class statics of spec §4.1/§6.1 with registries that the palette, context menu and codec all read from (§17.2's single source of truth). ```ts interface NodeDefinition { type: string; label: string; icon: string; beta: boolean; // menu metadata (§4.1) ports(node: N): readonly string[]; // per instance: static for known types, synthetic for Unknown (§4.10) portLabels?: readonly string[]; create(id: string): N; decode(json: NodeSpecJson): N; encode(node: N): NodeSpecJson; // spec codec (§10.3) emit(node: N, inputs: IR.RelationExpr[], ctx: EmitContext): IR.RelationExpr; // §8 isolationBoundary?: boolean; // true for window-producing nodes (§8.6): Partition, from M5 } interface SourceDefinition extends NodeDefinition { fromCoordinates(id: string, coordinates: unknown): S; // unresolved source from the picker resolve(node: S, result: SourceResolution): S; // new node with schema or error (replaces resolveV1) } ``` Node instances implement spec §5.2 `validate(inputSchemas, errors)` / `schematize(inputSchemas)` plus `describe()`. From M2 they also give their row order (§4.4): `outputOrder(inputOrders)`, by default `[]` (no order, so a new node never claims one), and `consumesInputOrder`, true only for Limit, Drop and Slice, which take rows by that order. As built in M1.2: a `NodeDefinition {type, label, icon, beta}` with `TransformDefinition {kind: 'transform', create}` and `SourceDefinition {kind: 'source', fromCoordinates, resolve, queryRules?}`. `decode`/`encode` arrive with the codec (M1.6) and `emit` with the IR (M1.5). Port labels live on node instances (`portLabels`, Left/Right for binary nodes). The registry is built by `createNodeRegistry()`, not held as a module-level singleton, and reserves the `unknown` type. As built in M2 (§11.4): seven `UnaryNode`s in `src/nodes/transforms/`, each on the port `tds`, registered with the spec's palette labels in spec menu order (Sort, Filter, Restrict, Rename, Distinct, Drop, Limit, Slice, then Join). Each keeps its spec section's validation and `describe()` except as noted. A blank column name reports the generic `