Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support recursive references in json schema #98

Merged
merged 5 commits into from
Dec 6, 2024
Merged

Conversation

torymur
Copy link
Contributor

@torymur torymur commented Nov 26, 2024

Closes #96

Implementing recursive references feature by walking the json shema and counting ins and outs with defined max recursion depth of 3. Defining recursion depth higher than that probably should not be done, since each +1 step on the depth blows up regex's size exponentially. For example, for simple referential json schema level 5 will produce regex size over 700K, which seems counterproductive and likely to introduce performance issues. It also breaks even regex sensible defaults with CompiledTooBig error.

Also, for this reason having more sophisticated ref counter (counting different entities with their own recursion depths) won't work in current json schema implementation.

TODO:

  • port main json_schema tests to rust
  • encapsulate private parsing logic under parser struct
  • add recursive feature with defined default depth
  • adapt errors from anyhow to thiserror, handle errors
  • add more tests
  • add basic documentation

@torymur torymur added the bug Something isn't working label Nov 26, 2024
@torymur torymur changed the title Error instead of segfault on recursive references in json schema Support recursive references in json schema Dec 3, 2024
@torymur torymur marked this pull request as draft December 3, 2024 14:54
@torymur torymur added documentation Improvements or additions to documentation enhancement New feature or request testing labels Dec 3, 2024
@torymur
Copy link
Contributor Author

torymur commented Dec 6, 2024

This is ready for review of the recursive feature itself, which is relatively limited in code.

Refactoring and encapsulation changes, ported tests, errors and combined coverage check could be ignored in review.
Tests confirm that the main logic wasn't affected in any way with the exception of the addition of recursive feature.

The rest of the work defined above will be done in a follow up to speed up merging this PR.

@torymur torymur marked this pull request as ready for review December 6, 2024 13:27
@torymur torymur force-pushed the json-schema-segfault-96 branch from 2fa560a to 5449ec5 Compare December 6, 2024 13:59
Copy link
Contributor

@dpsimpson dpsimpson left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've gone through the recursive parts and everything looks right to me except for a bounds check that i commented on.

src/json_schema/parsing.rs Show resolved Hide resolved
@torymur
Copy link
Contributor Author

torymur commented Dec 6, 2024

Appreciate you taking a look here! @dpsimpson 🚀

@torymur torymur force-pushed the json-schema-segfault-96 branch from 5449ec5 to 9c5936f Compare December 6, 2024 15:52
@torymur torymur merged commit ac3b41f into main Dec 6, 2024
7 of 8 checks passed
@torymur torymur deleted the json-schema-segfault-96 branch December 6, 2024 16:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working documentation Improvements or additions to documentation enhancement New feature or request testing
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Implement fixed-depth recursion
4 participants