Grammar reference

Grammar formats and syntax.

GLRMask accepts JSON Schema, GLRM, Lark, and EBNF.

Constraint constructors

input	Python constructor	typical use
JSON Schema	`Grammar.from_json_schema(...)`	structured JSON, tools, typed API responses
GLRM	`Grammar.from_glrm(...)`	native grammar/lexer control, reusable subgrammars
Lark	`Grammar.from_lark(...)`	existing Lark grammars
EBNF	`Grammar.from_ebnf(...)`	conventional CFG descriptions

These constructors return immutable source Grammar values. Call grammar.compile(vocab) for a runnable constraint or grammar.compile_unlinked(vocab) for a reusable open parent.

JSON Schema

GLRMask supports a subset of JSON Schema. Unsupported features may be rejected.

GLRM

GLRM is GLRMask’s native grammar format. A grammar begins with glrm 1; and a start declaration. Rules use = and end with ;.

glrm 1;
start value;

t WS = /[ \t\r\n]+/;
ignore WS;

t NUMBER = /-?(0|[1-9][0-9]*)/;
nt value = NUMBER | "null";

Use explicit eps for epsilon. Regex terminals use full-match semantics. Unsupported or non-regular regex constructs are rejected.

External subgrammars

Declare a child grammar by name and bind a compiled constraint in Python:

extern grammar payload;

parent = glrmask.Grammar.from_glrm(grammar)
payload = glrmask.Grammar.from_json_schema(payload_schema)
document = parent.bind("payload", payload).compile(vocab)

For a compiled child, use parent.compile_unlinked(vocab).bind("payload", child).link(). Source bindings accept Grammar values; compiled bindings accept runnable Constraint values.

Inline and externally bound subgrammars have the same semantics:

g inner = {
    start value;
    nt value = "null";
};

Special tokens

To use a special token in a GLRM grammar, declare it by name:

glrm 1;
start message;
extern token TOOL_CALL;
nt message = TOOL_CALL call;
nt call = "lookup()";

Bind it outside the grammar:

constraint = glrmask.Grammar.from_glrm(grammar).bind(
    "TOOL_CALL", vocab.token(tool_call_token_id)
).compile(vocab)

Direct finite automata

A terminal or nonterminal can use an explicit finite automaton body:

t WORD = fa {
    start begin;
    accept done;
    begin -> middle: "a";
    middle -> done: "b";
};

End tokens

Supply exact end-token IDs with grammar.compile(vocab, end_tokens=[eos_id]). The mask allows these IDs only when the grammar body is accepting. Committing a configured end token lets the decoder stop based on its chosen ID. End-token policy belongs to the final root and is not inherited from compiled children.