shex.js

Language-aware editors for the ShEx / ShExMap web apps — implementation plan

Status (2026-07-14): phases 0–3 implemented behind ?editors=1.

Goal: replace the plain textareas (schema and data in shex-simple; bindings, statics and output schema in shexmap-simple) with language-sensitive editors that can

  1. highlight the shape declaration corresponding to a shape-map node (and vice versa), and
  2. highlight validation errors in both panes with precise ranges — e.g. “schema says <p> should be an @<foo>” anchors on that triple constraint in the schema editor and on the offending triple in the data editor.

The core insight: the editor doesn’t need to understand the languages

The instinct that deep parser↔editor integration might not be necessary is right. Modern editors (CodeMirror 6 here) separate three concerns:

So millan and the ts-jison ShExC parser never talk to CodeMirror directly; they run exactly as they do today (on the cache-refresh path), and a thin adapter converts their ranges into diagnostics/decorations. No Lezer grammar, no language server protocol, no editor plugins to write beyond ~200 lines of glue.

What prototyping established (2026-07-14)

Schema side: the ShExC parser already half-does this

packages/shex-parser (ts-jison) already exposes source locations — no ts-jison changes were needed to get:

const schema = ShExParser.construct(base, {}, {index: true}).parse(text);
schema._locations
// { 'http://a.example/S': { filename, first_line: 4, first_column: 0,
//                           last_line: 7, last_column: 1 }, ... }
schema._sourceMap   // Map: shapeRef/tripleExprRef -> [yylloc, ...] per @<X> site

That is feature 1 (shape-map ↔ shape highlighting) essentially for free.

The machinery: the grammar brackets shapeExprDecl with a mark nonterminal capturing yy.lexer.yylloc, and sprinkles yy.addSourceMap(...) on reference productions. The ts-jison runtime also computes a merged location (yyval._$) for every reduction, and its lexer supports a ranges option (currently off) that adds character offsets alongside line/column.

Interface work needed (the anticipated ts-jison/grammar tweaks):

Why identity-keyed maps work end-to-end: validation errors reference the schema objects by reference. Verified in packages/shex-validator/lib/shex-validator.js:

{ type: "TypeMismatch",
  triple: { type: "TestedTriple", subject, predicate, object },  // LD terms
  constraint: misses.get(t).constraint,   // <-- the parser's own TC object
  errors: [...] }

constraint → _exprLocations.get(constraint) → schema range. Done.

Data side: millan’s rdfjs-interface branch is purpose-built

../../ericprud/millan (branch rdfjs-interface, commit b50f0be “Add RDF/JS adapter with source-location provenance”) provides:

const {dataset, errors, semanticErrors, emitterErrors} =
  millan.rdfjs.parseTurtleToRdfjs(text, {baseIRI, sourceURL});

Prototype output (scratchpad/millan-proto.js):

L4:8-L4:21 [85-98] http://a.example/x :p "not a number" (object @ [85-98])
...
=== parse errors (tolerant): 1

Branch polish list found while prototyping:

Editor: CodeMirror 6, delivered through the existing webpack bundles

Architecture

             ┌────────────────────────────────────────────────────┐
             │ @shexjs/editor-services (new package, node+browser) │
             │                                                    │
 ShExC text ─┤ shexc.parse(text, base)                            │
             │   → {schema, diagnostics[], locate: {              │
             │       shape(label), expr(obj), ref(label)}}        │
 Turtle text ┤ turtle.parse(text, base)   [millan rdfjs]          │
             │   → {dataset, diagnostics[]}                       │
             │ mapErrors(valResult, schemaLocate, dataset)        │
             │   → {schemaDiagnostics[], dataDiagnostics[],       │
             │      pairs[]}   // paired ids for cross-pane flash │
             └────────────────────────────────────────────────────┘
                      ↑ ranges                      ↓ diagnostics/decorations
        ShExParser/_locations/_exprLocations   CodeMirror 6 EditorPanes
        millan quad/term .source               (schema, data, bindings, outSchema)

mapErrors walks the validation Failure structure (c.f. ShExUtil.errsToSimple for the traversal patterns), resolving:

Phases

Phase 0 — editors in, behavior unchanged (spike) CM6 EditorPane adapter behind ?editors=1 in shex-simple: schema + data panes with Turtle/stream-ShExC modes, JSON panes in shexmap-simple. All existing tests must stay green (browser tests drive the textarea path; add a jsdom test for the adapter contract).

Phase 1 — shape-map ↔ shape highlighting (uses today’s parser output) Expose _locations/_sourceMap through SchemaCache; hover/select on a fixed-shape-map entry decorates the shape’s declaration range; clicking a shape decoration filters/scrolls the shape map. No grammar changes.

Phase 2 — live parse diagnostics Data pane parses with millan in parallel with N3 (validation path untouched); millan + structural ShExC parse errors become linter diagnostics in both panes. Debounced on-change parsing; millan’s error tolerance keeps stale highlights minimal.

Phase 3 — validation-error mapping (the headline) Grammar work for _exprLocations (+ optional lexer ranges); implement mapErrors; validate → paired squiggles: the failing TripleConstraint in the schema pane, the offending triple/term in the data pane, hover text from errsToSimple. ShExMap: MaterializationError.failures reference TCs by identity too, so the output-schema pane gets the same treatment for materialization failures.

Phase 4 — consolidation & polish

Risks / open questions

Prototype artifacts

Both currently live outside the repo (session scratchpad); re-run or promote into test/ fixtures during Phase 0.

This site is open source. Improve this page.