Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use Boost.Charconv #993

Draft
wants to merge 2 commits into
base: develop
Choose a base branch
from

Conversation

grisumbras
Copy link
Member

@grisumbras grisumbras commented Mar 19, 2024

  1. Removes embedded older version of Charconv and instead uses the library proper. Fix Remove GCC<5 stuff after Boost 1.87 #1032.
  2. Removes embedded version of ryu and uses Charconv for double formatting.

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

1 similar comment
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 227539e to be9028c Compare December 14, 2024 17:06
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

1 similar comment
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

Copy link

codecov bot commented Dec 14, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 99.23%. Comparing base (7f0bceb) to head (be9028c).

Additional details and impacted files

Impacted file tree graph

@@             Coverage Diff             @@
##           develop     #993      +/-   ##
===========================================
+ Coverage    93.70%   99.23%   +5.52%     
===========================================
  Files           91       78      -13     
  Lines         9139     8052    -1087     
===========================================
- Hits          8564     7990     -574     
+ Misses         575       62     -513     
Files with missing lines Coverage Δ
include/boost/json/basic_parser_impl.hpp 98.50% <100.00%> (ø)

... and 1 file with indirect coverage changes


Continue to review full report in Codecov by Sentry.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 7f0bceb...be9028c. Read the comment docs.

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot
Copy link

The check was added because of older GCC versions which are no longer
supported
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 91a7792 to 9e788cd Compare February 19, 2025 11:11
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@grisumbras grisumbras force-pushed the refactor/charconv branch 2 times, most recently from 3b7526f to c58ed7b Compare February 19, 2025 11:17
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

2 similar comments
@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

@sdarwin
Copy link
Collaborator

sdarwin commented Feb 19, 2025

This is crashing the s390 runners. out-of-memory.
The PR was opened 10 months ago. Have you rebased it on develop?
I now adjusted those runners from 2 jobs to 1 job. Try it again.

Screenshot from 2025-02-19 07-13-23

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

@cppalliance-bot
Copy link

@grisumbras
Copy link
Member Author

I did rebase it on develop.

It is quite interesting that the build is crashing s390 runners. This might be a problem with Boost.Charconv.

@cppalliance-bot
Copy link

An automated preview of the documentation is available at https://993.json.prtest2.cppalliance.org/libs/json/doc/html/index.html

@cppalliance-bot
Copy link

@sdarwin
Copy link
Collaborator

sdarwin commented Feb 19, 2025

This might be a problem with Boost.Charconv.

It used more than 8GB of memory.

I have just kicked off another run on 'develop', to compare that.

@sdarwin
Copy link
Collaborator

sdarwin commented Feb 19, 2025

Depending on the test, even the develop branch may get to nearly 8GB.
Charconv is not the cause.
I don't think JSON was using that much memory a couple years ago.
In any case the runner is now parallel-jobs=1 so more memory is available.

@grisumbras
Copy link
Member Author

I've recorded memory usage when building and running tests. The outlier is pretty much compiling parse_into.cpp:

Config Max mem usage
toolset=gcc variant=release cxxstd=20 3.22GB
toolset=gcc variant=release cxxstd=17 3.15GB
toolset=gcc variant=debug cxxstd=20 3.10GB
toolset=gcc variant=debug cxxstd=17 2.95GB
toolset=clang variant=debug cxxstd=20 2.02GB
toolset=clang variant=debug cxxstd=17 2.00GB
toolset=clang variant=release cxxstd=20 1.51GB
toolset=clang variant=release cxxstd=17 1.42GB

Everthing else is below 0.9GB (https://gist.github.com/grisumbras/5985960c7781c4f657a93605d9e2b0ed).

Unfortuantely, this is inherent to parse_into.cpp. It's very template-heavy. I think, the way we should deal with this is to move it to a separate build target, and run b2 twice: one for regular jobs with regular job count limit, and then the heavy tests one job at a time.

@grisumbras
Copy link
Member Author

BTW, for posterity, the info was collected thusly:

# user-config.jam
using gcc : 13 : /usr/bin/time "-f%C|%M" -o/home/grisumbras/dev/stats -a g++-13 ;
using clang : 20 : /usr/bin/time "-f%C|%M" -o/home/grisumbras/dev/stats -a clang++-20 ;

And then b2 was invoked with

rm /home/grisumbras/dev/stats
b2 test example toolset=gcc,clang variant=debug,release cxxstd=17,20 warnings=extra testing.launcher="/usr/bin/time -f'%C|%M' -o/home/grisumbras/dev/stats -a"

After that the file /home/grisumbras/dev/stats contained collected statistics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Remove GCC<5 stuff after Boost 1.87
3 participants