Bounded Morphir YAML input

This portable package reads the normative Morphir YAML profile on JS, native, Wasm and Wasm GC. It needs no host process or daemon.

parse(source, limits~) returns the duplicate-aware JSON-shaped value tree, retaining numeric lexemes. read_versioned(source, limits~) uses the existing version-specific IR decoder. read(source, limits~) also applies the shared migration to current IR. Single files support versions 1 through 4 under that decoder's support table. Quoted and block scalars stay strings, including dates.

read_tree(files, limits~) accepts a map of relative physical paths to text. It assembles v4 and exactly v3.1.0 trees under their version-specific contracts. JSON trees use the existing bounded JSON reader. YAML accepts .yaml and .yml. The manifest selects one physical profile; ambiguous files, mixed profiles, incorrect roles, stray package files and mismatched versions are refused. Modules are sorted by logical path. The reader supports inline and listed nodes, canonical filename escapes, fileNames, dependency specifications and Application definitions. It ignores reserved $meta and unrelated non-tree extensions.

Read a bounded tree

Use open_bounded_tree(source, limits~) to read plain v3.1.0 and supported v4 document trees one page or member at a time. For example, a generator for acme/orders can page through the domain module with reader.members(package_name, module_name, limit=2). Its next page contains the value total-in-USD, whose logical document is pkg/acme/orders/domain/total-in-_usd.value. Call reader.read_member with the package, module, member kind, and canonical name to decode that value. The returned TreeMemberBody carries the original version and role, including classic v3 inferred-type attributes. A type and a value named order are independent selections.

TreeSource supplies a revision identifier, exact physical path probes, and extensionless module paths ordered lexically under each package prefix. Its metadata callbacks must avoid reading file bodies. Each callback must serve the revision it receives or raise TreeSourceError::RevisionChanged. The read callback must enforce the supplied UTF-8 byte cap before allocating an oversized buffer, and raise TreeSourceError::DocumentTooLarge so the reader can report the affected logical document. Source adapters own immutable storage, path confinement and any filesystem or index work. File probes are case-sensitive. JSON uses .json. YAML uses .yaml or .yml. The manifest selects one profile and the reader refuses multiple physical spellings of a selected logical document.

reader.package_name() and reader.dependency_packages() expose the names declared by the manifest. reader.modules(package, limit~) pages one package in lexical logical-path order. reader.members(package, module, limit~) pages types before values and follows the array or mapping order within each kind. Inline members use their module's logical document identity. Each page has up to 256 entries and may return an in-process cursor. Pass that cursor to the same reader with the same selection to continue. A changed revision, another reader, or a different selection causes a refusal.

The bounded reader retains manifest routing metadata, one capped document at a time, and the current page. Module and member discovery can read module metadata. Split definitions and specifications are opened only on selection. Inline entries are physically part of the capped module document and receive typed payload validation when selected. Document reads apply the byte, depth, node and assembly limits below per file. Callers control how many returned pages and decoded bodies they retain. Discovery leaves unread bodies unvalidated. Use read_tree_versioned or read_tree for whole-model assembly and validation, including checks for stray tree files.

Read file-local name tables

Use open_bounded_tree(source, encoding=ContextualDraft1) to admit the draft.1 contextual envelope in JSON or YAML. Plain reading is the default. The encoding selection is explicit and separate from the manifest's IR version. Within an opted-in tree, a file without an envelope keeps its plain representation.

When the orders generator selects total-in-USD, the value file's fqNames table resolves its reference to acme/orders:domain#order. The reader returns that typed reference without opening order.value. A $name member inside a document literal remains ordinary data. Each context has four separate typed tables, at most 16,384 entries per table, and at most 1,024 UTF-8 bytes per spelling. The envelope, including ignored optional context content, counts toward the encoded file's byte and assembly limits.

Context follows documentOpen one capped documentValidate its envelope and tablesResolve aliases in typed name slotsReturn selected versioned memberReopen the same revision under the same cap

A typed name in a legacy pair array can carry an alias. This includes classic record fields, recursive bindings and constructors, plus v4 constructor and value specification inputs. JSON object keys stay spelled out. The v4 value definition's input map also keeps its names expanded when read in the tolerated pair-array spelling.

The general reader admits a context after the document with a second bounded read of the same revision. It releases the first parse tree before that read. ContextualDraft1Ordered requires both the schema marker and context before the document. This option checks input order. The reader still parses one capped document at a time.

Use reader.read_original_document with a manifest, module or member identity to pass through the original UTF-8 text. Inline members select their containing module. The reader validates the selection and reopens the artifact under the same revision and byte limit, preserving optional content, formatting and numeric spellings.

Stage an expanded JSON export

read_expanded_document returns a selected, already scoped artifact as plain JSON with a caller-supplied output byte cap. An inline member selects its containing module. The same revision and input caps apply to every read, including the extra read for a context after its document.

write_expanded_tree stages one file at a time using the bounded module and member pages. It emits the manifest, then the root package and its declared dependencies. Each inline module is written once. A document and the combined output have separate UTF-8 byte caps. The tree retains only the current documents and bounded pages. The callback owns the private stage. Publish it only after the walk returns successfully. A late limit, invalid document, callback error, or revision change can fail the walk.

Use read_original_document for exact pass-through, including optional envelope data and YAML presentation. Expanded export deliberately emits plain JSON and keeps typed name values, numeric lexemes and meaningful member order.

Limits

Default limits are finite and configurable:

Resource Default Check
Source bytes 16 MiB Before parsing. Whole-tree APIs also cap the total
Events and scanner tokens 400,000 each Before the next pull or token insertion
Physical nodes 100,000 Before scalar placement or container allocation
Depth 128 Before scanner/parser stack growth and container placement
Scalar UTF-16 units 4 Mi Before each scanner buffer append, including folding buffers
Semantic assembly work 100,000 Nodes and text units before typed shorthand expansion; cumulative for whole-tree assembly

Configured depth cannot exceed the portable hard ceiling of 512. Diagnostic cursors and physical paths are also bounded to 4,096 UTF-16 units. Whole-tree APIs accept at most 4,096 captured files. Adapters must bound filesystem traversal and capture before supplying a map. The engine exposes the same capture bound as data.model_tree_file_limit.

The physical grammar comes from the MIT-licensed MoonBit event parser, with pinned provenance and local changes. The Morphir layer resolves scalars itself and refuses anchors, aliases, tags, directives, merge keys, duplicate members, multiple documents, non-string keys and forbidden numeric forms. Typed float decoding also refuses nonfinite values and nonzero underflow. Arbitrary-precision document numbers retain their lexemes.

Normalization does not preserve YAML presentation. Callers retain original bytes separately through the shared model-document owner for exact downloads. This package implements input, not a canonical YAML writer.

In-process reader observations

An explicit MCK adapter can use tree.observation() for admitted manifest facts and call observation() on each TreeModule or TreeMember returned by the existing paged reader. Keep TreeCursor opaque and pass it back to the same snapshot and selection. Inventory observations open no member bodies. tree.observe_member(package, module, kind, name) reads one selected typed member and returns its canonical identity, logical document, role and version-specific payload. Classic payloads retain type and expression attributes. Direct value references, integer literals and output type references also have normalized fact fields. Missing optional fields say nothing about nested expressions. This API does not evaluate expressions.

To follow a direct reference, call direct_value_reference() on the typed TreeMemberBody and pass that FQName's package, module and local name to read_member or observe_member with ValueMember. A type with the same name has a separate identity. These reads retain the existing revision, selection and per-document admission checks. The adapter supplies a bounded revision-bound TreeSource; it need not construct a complete map of bodies.

The observation API contains no fixture expectations, runner, receipt schema or implementation-revision claims. The owning runner records provenance, checks fixed expectations and determines correctness separately from resource qualification. observation_fixture_test.mbt embeds all 72 documents of the eight parent orders profiles and the independently authored expected.json from finos/morphir 3652032a53adeb6bfa4a5d672142487de45c57c5. The focused observation_test.mbt emits normalized JSON facts for an eventual adapter and checks two domain pages, selected original bytes, roles, escaped IDs, classic attributes and a typed value reference resolving to integer 73. Controls change that literal, reuse a cursor across selections or readers, change the source revision and refuse missing or oversized selected documents.

These tiny in-process cases use an in-memory fixture source. They establish reader behavior, not a filesystem snapshot lease, host generation, WASM execution or a measured memory ceiling. Compaction uses the existing public wire.compact_names and wire.expand_names APIs and the independent frozen policy cases in wire/compaction_policy_wbtest.mbt; no second policy checker is introduced here.

View source on GitHub