Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Increase buffer size #20

Open
DavidNemeskey opened this issue Dec 21, 2016 · 3 comments
Open

Increase buffer size #20

DavidNemeskey opened this issue Dec 21, 2016 · 3 comments
Assignees

Comments

@DavidNemeskey
Copy link
Contributor

Is it possible to increase the buffer size? Now it is around 80-90k, any continuous (read: no empty lines) text longer than that results in the following error:

quex/quex/code_base/buffer/BufferFiller.i:655:  terminate called after throwing an instance of 'std::runtime_error'
  what():  Distance between lexeme start and current pointer exceeds buffer size.
(tried to load buffer forward). Please, compile with option

    QUEX_OPTION_INFORMATIVE_BUFFER_OVERFLOW_MESSAGE

in order to get a more informative output. Most likely, one of your patterns
eats longer as you inteded it. Alternatively you might want to set the buffer
size to a greate value or use skippers (<skip: [ \n\t]> for example).

Aborted

There are several documents in the MNSZ that trigger this. Some authors had the wonderful idea of writing VERY long sentences...

@mittelholcz
Copy link
Collaborator

@DavidNemeskey Buffer size is increased to 1MB. Is it enough?

@DavidNemeskey
Copy link
Contributor Author

@M-Ivan Some errors were fixed by the change, but not all.

mittelholcz pushed a commit that referenced this issue Jan 3, 2017
@mittelholcz
Copy link
Collaborator

@DavidNemeskey I increased the buffersize to 2MB. If it is enough, please close the issue. Otherwise you can either make a pull request (for change buffersize see Makefile: line 74), or send a sample file with the long sentences which caused the error.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants