Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
28 changes: 21 additions & 7 deletions GLR.md
Original file line number Diff line number Diff line change
Expand Up @@ -97,6 +97,8 @@ Initialize the generated context with initial user data (or `with_default_userda

In GLR mode, user data is branch-local. When the parser forks because of a conflict, the current semantic stack and user data are cloned, and each resulting branch owns and mutates its own `UserData` value independently. Token values, semantic return values, and user data must therefore implement `Clone` for GLR branch-local parsing. Results returned from `accept_all()` include the final user data for each successful branch.

`NoAction` is the only feed failure that means the CFG cannot consume the lookahead terminal. `ReduceAction` means the terminal was grammatically feedable, but a runtime semantic action failed. GLR recovery is entered only when every active branch fails with `NoAction`; if at least one branch survives, the feed succeeds and any sibling branch failures are returned in `FeedSuccess::errors`.

For example, this ambiguous grammar creates two branches for the same input terminal symbol. Each branch mutates its own cloned `Vec<&'static str>`:

```rust
Expand Down Expand Up @@ -143,15 +145,26 @@ fn main() {
// Feed tokens to the GLR parser
for token in input {
// basic feeding:
if let Err(e) = context.feed(token) {
eprintln!("Fatal parse error: {}", e);
return;
match context.feed(token) {
Ok(success) => {
if let Some(branch_errors) = success.errors {
eprintln!("Some GLR branches were pruned: {}", branch_errors);
}
}
Err(e) => {
eprintln!("Fatal parse error: {}", e);
return;
}
}

// or location-aware feeding (if %location is configured in the grammar):
// let span = MySpan { start: ..., end: ... };
// if let Err(e) = context.feed_location(token, span) {
// eprintln!("Fatal parse error: {}", e);
// if let Ok(success) = context.feed_location(token, span) {
// if let Some(branch_errors) = success.errors {
// eprintln!("Some GLR branches were pruned: {}", branch_errors);
// }
// } else {
// eprintln!("Fatal parse error");
// return;
// }
}
Expand All @@ -172,7 +185,8 @@ fn main() {

### Key API Components
- **`EContext::new(userdata)`**: Initializes a new GLR state context with initial user data. Use `EContext::with_default_userdata()` when `UserData: Default`.
- **`context.feed(token)`**: Feeds a token into all active parsing stacks.
- **`context.feed_location(token, location)`**: Feeds a token with its location span into all active parsing stacks (requires `%location` in the grammar).
- **`context.feed(token)`**: Feeds a token into all active parsing stacks and returns `FeedSuccess` when at least one branch survives.
- **`context.feed_location(token, location)`**: Feeds a token with its location span into all active parsing stacks (requires `%location` in the grammar) and returns `FeedSuccess` when at least one branch survives.
- **`FeedSuccess::errors`**: Optional branch-failure information from GLR branches pruned while the feed still succeeded.
- **`context.accept()`**: Finalizes parsing (feeding the end-of-file symbol) and returns the first successful `(parse_result, userdata)` pair.
- **`context.accept_all()`**: Finalizes parsing and returns an iterator over successful `(parse_result, userdata)` pairs from all active branches.
4 changes: 4 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -188,12 +188,16 @@ context.feed(token);
context.feed_location(token, token_location);
```

Feed failures are classified by cause. `NoAction` means the CFG cannot consume the lookahead terminal; it leaves the parser stack unchanged and is the only failure kind that can enter panic-mode recovery. `ReduceAction` means the terminal was grammatically feedable, but a runtime semantic action failed. In GLR mode, recovery is attempted only when every active branch fails with `NoAction`.

---

## GLR Parsing

RustyLR provides native support for Generalized LR (GLR) parsing. When you add the `%glr;` directive to a grammar, RustyLR generates a non-deterministic parser that forks state branches upon encountering shift/reduce or reduce/reduce conflicts. This is particularly useful for ambiguous grammars or complex programming languages.

GLR `feed()` and `feed_location()` return a success value that may contain errors from branches pruned while another branch survived. Treat the feed as failed only when the returned `Result` is `Err`; inspect the success value when branch-level diagnostics matter.

In GLR mode, user data is branch-local. When the parser forks, the current user data is cloned so each active branch owns and mutates an independent `UserData` value.

For more details, see [GLR.md](GLR.md).
Expand Down
10 changes: 8 additions & 2 deletions SYNTAX.md
Original file line number Diff line number Diff line change
Expand Up @@ -395,7 +395,9 @@ Expr: '{' Stmt* '}'
;
```

The reserved terminal `error` is used to implement panic-mode recovery. When the parser encounters an unexpected token:
The reserved terminal `error` is used to implement panic-mode recovery. Recovery is entered only for `NoAction`, which means the CFG cannot consume the lookahead terminal. `ReduceAction` means the terminal was grammatically feedable, but a runtime semantic action failed.

When the parser encounters an unexpected token:
1. **Pop Stack**: The parser pops states off the parser stack until it finds a state that can shift the `error` symbol. If no such state is found, parsing aborts immediately with a `ParseError::NoAction`.
2. **Shift `error`**: The parser shifts the `error` symbol onto the stack. The initial span/location of the `error` token is set to the span of the popped tokens.
3. **Handle Lookahead & Merge**:
Expand All @@ -408,7 +410,7 @@ The reserved terminal `error` is used to implement panic-mode recovery. When the
#### Notes
- The `error` token does not carry a semantic value.
- You can access the entire span of all discarded/merged tokens using the location binder `@error`.
- In GLR mode, the parser will prefer non-error paths and only trigger error recovery if all other active branches fail.
- In GLR mode, the parser will prefer non-error paths and only trigger error recovery if all active branches fail with `NoAction`.
- **Comparison to Bison**: Unlike Bison, which runs a blocking pull-loop internally to discard tokens until it finds a synchronizing token, RustyLR integrates with a push-based model (`feed()`). If a token cannot follow `error`, RustyLR discards it and merges its location span into the `error` token's span, returning control to the caller. This enables reactive parsing and easy diagnostics via `@error` containing the full range of discarded tokens.

### Operator Precedence
Expand Down Expand Up @@ -461,6 +463,8 @@ Expr

Defines a custom error type returned by reduce actions. If your reduce action returns a `Result`, returning an `Err(custom_error)` will stop execution (or prune the GLR branch) and bubble the error up wrapped in `ParseError::ReduceAction`.

A reduce-action error is semantic failure, not a syntax `NoAction`. It does not enter panic-mode error recovery. In GLR mode, if another branch successfully shifts the same terminal, the feed succeeds and reports the pruned-branch error in the GLR feed success value.

## No Optimization

```
Expand Down Expand Up @@ -500,6 +504,8 @@ To feed locations, use `feed_location` instead of `feed`:
context.feed_location(token, span);
```

`NoAction` means the CFG cannot consume the lookahead terminal and leaves the parser stack unchanged. `ReduceAction` means the terminal was grammatically feedable, but runtime execution failed. Only `NoAction` can enter panic-mode recovery.

Within reduce actions, access symbol spans using the `@` prefix:
```rust
Expr(i32) : e1=Expr '+' e2=Expr {
Expand Down
3 changes: 3 additions & 0 deletions USING_RUSTYLR_WITH_AI.md
Original file line number Diff line number Diff line change
Expand Up @@ -110,6 +110,8 @@ fn parse(tokens: impl IntoIterator<Item = parser::Token>) -> Result<i64, String>
}
```

Feed failures have two main runtime causes. `NoAction` means the CFG cannot consume the lookahead terminal, leaves the parser stack unchanged, and is the only failure kind that can enter panic-mode recovery. `ReduceAction` means the terminal was grammatically feedable, but runtime execution failed.

## Agent Checklist

Before implementing:
Expand All @@ -129,6 +131,7 @@ While implementing:
- Use `%left`, `%right`, or `%precedence` before changing grammar shape solely to fix expression conflicts.
- Use the `error` terminal for recovery points such as delimiters, statements, or object members.
- Include `error` in precedence declarations if recovery productions introduce a shift/reduce conflict.
- Treat reduce-action `Err` values as semantic failures, not recovery triggers. In GLR mode, a successful feed may still return branch errors through its success value when other branches were pruned.
- Keep generated parser files committed only if the project convention does so.

When debugging:
Expand Down
4 changes: 4 additions & 0 deletions llms.txt
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,10 @@ RustyLR targets Rust 2024 and requires Rust 1.85 or newer.

Generated `<Start>Context` types are the normal parser contexts downstream code should use. They initialize parsing for the selected start symbol and return the typed start value from `accept()` or `accept_all()`.

Feed failures have two main runtime causes. `NoAction` means the CFG cannot consume the lookahead terminal, leaves the parser stack unchanged, and is the only failure kind that can enter panic-mode recovery. `ReduceAction` means the terminal was grammatically feedable, but runtime execution failed.

Reduce-action errors are semantic failures, not panic-mode recovery triggers. GLR feed success values may contain branch errors when one branch survives and sibling branches are pruned.

Generated contexts implement `Clone` when their runtime storage and user data satisfy the required `Clone` bounds. Choose token and semantic value types that implement `Clone` when cloning contexts or using GLR branch-local user data; user data must also implement `Clone` in those cases.

Context `Debug` output reports parser stack state and user data; the root parser state is implicit, and GLR output is grouped by active branch. Use the explicit tree APIs for syntax-tree inspection when the `tree` feature is enabled.
Expand Down
89 changes: 89 additions & 0 deletions rusty_lr/tests/lr_glr_operation.rs
Original file line number Diff line number Diff line change
Expand Up @@ -376,4 +376,93 @@ mod reduce_action_errors {
}
assert!(too_large.accept().is_err());
}

mod glr_recovery {
use rusty_lr::lr1;

// Keep optimization disabled so `Rejected` remains a distinct reduction that fails before
// the following terminal is shifted. This makes the recovery boundary observable.
lr1! {
%glr;
%nooptim;
%tokentype char;
%userdata Vec<&'static str>;
%error &'static str;
%start Item;

Item(char)
: Rejected 'y' {
Rejected
}
| error 'y' {
data.push("recovered");
'?'
}
;

Rejected(char): 'x' {
if true {
return Err("rejected token");
}
'x'
};
}

#[test]
fn glr_reduce_action_error_does_not_enter_recovery_mode() {
let mut ctx = ItemContext::new(Vec::new());
ctx.feed('x').unwrap();

// `error 'y'` could recover from a grammatical `NoAction`, but the failing branch is
// grammatically valid and dies from a reduce-action error instead.
let err = ctx.feed('y').unwrap_err();

assert_eq!(err.reduce_action_errors, ["rejected token"]);
assert!(ctx.userdata_all().all(|userdata| userdata.is_empty()));
}
}

mod glr_partial_errors {
use rusty_lr::lr1;

// This grammar keeps one shift path alive while a competing reduce path fails
// semantically, exercising successful feeds with pruned-branch diagnostics.
lr1! {
%glr;
%nooptim;
%tokentype char;
%error &'static str;
%start S;

S(&'static str)
: A 'y' {
"reduce path"
}
| 'x' 'y' {
"shift path"
}
;

A(char): 'x' {
if true {
return Err("bad reduce path");
}
'x'
};
}

#[test]
fn glr_feed_success_reports_pruned_branch_errors() {
let mut ctx = SContext::with_default_userdata();
assert!(ctx.feed('x').unwrap().errors.is_none());

// Feeding `y` succeeds through the direct shift branch, while the reduce branch reports
// its semantic failure through `FeedSuccess::errors`.
let success = ctx.feed('y').unwrap();
let errors = success.errors.expect("reduce branch should be reported");

assert_eq!(errors.reduce_action_errors, ["bad reduce path"]);
assert_eq!(ctx.accept().unwrap(), ("shift path", ()));
}
}
}
Loading
Loading