Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Sequence peeking input is being ignored #64

Open
phorward opened this issue Jun 29, 2022 · 1 comment
Open

Sequence peeking input is being ignored #64

phorward opened this issue Jun 29, 2022 · 1 comment
Assignees
Labels
bug Something isn't working

Comments

@phorward
Copy link
Member

phorward commented Jun 29, 2022

Given this program

T_EOL : @{
    peek EOF
    peek ' '
}

'Hello' T_EOL _ print("World")

and the input Hello, the call to peek EOF is ignored and afterwards rejected as it doesn't consume any input, even if its valid.
This bug is a play between peek and the Op::ForwardIfConsumed operation which doesn't recognize that input was consumed because peek consumed input but reset it afterwards.

There is a workaround for this problem by defining T_EOL this way:

T_EOL : @{
    peek EOF accept
    peek ' '
}

It can also be reproduced when the two sequence are flipped, so

T_EOL : @{
    peek ' '
    peek EOF
}

Doesn't work either, but

T_EOL : @{
    peek ' '  accept
    peek EOF
}

does. Latter one does also work because the result is not reset, and the parselet accepts next:

.T_EOL@6     000 Frame(0)
.T_EOL@6     001 Fuse(11)
.T_EOL@6     002 Frame(0)
.T_EOL@6     003 Frame(0)
.T_EOL@6     004 CallStatic(6)
.T_EOL@6     005 Reset
.T_EOL@6     006 Close
.T_EOL@6     007 Accept
.T_EOL@6     008 Collect(0, 5)
.T_EOL@6     009 Close
.T_EOL@6     010 ForwardIfConsumed(7)
.T_EOL@6     011 Reset
.T_EOL@6     012 Frame(0)
.T_EOL@6     013 Offset(Offset { offset: 40, row: 3, col: 10 })
.T_EOL@6     014 CallStatic(7)
.T_EOL@6     015 Reset
.T_EOL@6     016 Close
.T_EOL@6    >017 Close
.T_EOL@6    --- Stack ---
.T_EOL@6     000 Empty
.T_EOL@6     001 Range(1..6, None, 0)
.T_EOL@6     002 Empty
.T_EOL@6    --- Frames ---
.T_EOL@6     000 Frame { fuse: None, capture_start: 3, reader_start: Offset { offset: 6, row: 1, col: 12 } }
.T_EOL@6     001 Frame { fuse: Some(12), capture_start: 3, reader_start: Offset { offset: 6, row: 1, col: 12 } }
.T_EOL@6    ip = 17 state = Ok(Next)
.T_EOL@6    returns None
@phorward phorward added the bug Something isn't working label Jun 29, 2022
@phorward phorward self-assigned this Oct 17, 2022
@phorward phorward added the tokay label Feb 15, 2023
@phorward phorward removed the tokay label Oct 8, 2023
@phorward
Copy link
Member Author

phorward commented Nov 6, 2024

The problem still exists in Tokay v0.6.6, with this updated version

T_EOL : @{
    Peek<EOF>
    Peek<' '>
}

'Hello' T_EOL _ print("World")

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

1 participant