Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Cleaner stack trace with visitors #982

Draft
wants to merge 8 commits into
base: main
Choose a base branch
from
Draft

Cleaner stack trace with visitors #982

wants to merge 8 commits into from

Conversation

krlmlr
Copy link
Member

@krlmlr krlmlr commented Aug 21, 2022

  • Ensures that all visited functions have a name
  • This name now appears in the stack trace thanks to make_visit_one()

We could extend this idea to implementing a make_visit() function, pre_visit() would be visitor <- make_visit(pre = TRUE) then visitor(pd_nested) . This would save a few function calls, give more compact code, and also better stack traces for pre_visit_one() (which would vanish). The 80-20 rule forbids to do this right now, maybe later. Also, this turns the whole visiting logic into a metaprogram, this is a slippery slope.

For #759.

@codecov-commenter
Copy link

codecov-commenter commented Aug 21, 2022

Codecov Report

Merging #982 (5d5a7cc) into main (0592e8e) will decrease coverage by 0.02%.
The diff coverage is 100.00%.

❗ Current head 5d5a7cc differs from pull request most recent head e95d346. Consider uploading reports for the commit e95d346 to get more accurate results

@@            Coverage Diff             @@
##             main     #982      +/-   ##
==========================================
- Coverage   91.07%   91.06%   -0.02%     
==========================================
  Files          46       46              
  Lines        2713     2708       -5     
==========================================
- Hits         2471     2466       -5     
  Misses        242      242              
Impacted Files Coverage Δ
R/testing.R 67.36% <100.00%> (+0.69%) ⬆️
R/transform-files.R 97.50% <100.00%> (+0.01%) ⬆️
R/visit.R 94.25% <100.00%> (-0.54%) ⬇️

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if cb15e52 is merged into main:

  •   :ballot_box_with_check:cache_applying: 31.6ms -> 30.4ms [-9.05%, +1.54%]
  •   :ballot_box_with_check:cache_recording: 1.36s -> 1.38s [-0.83%, +3.42%]
  •   :ballot_box_with_check:without_cache: 3.69s -> 3.74s [-1.28%, +4.09%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@lorenzwalthert
Copy link
Collaborator

Do we need stack traces only for understanding where styler spends more time or do you want to merge that I to the default branch? Ie maybe it’s not necessary to go the slippery slope for the production code and this version of the code can reside in a side branch?

@krlmlr
Copy link
Member Author

krlmlr commented Aug 21, 2022

We could go with two parallel files, and activate only one of them depending on some switch. Do you prefer that?

Do I read the benchmark correctly: we're faster when applying the cache, but slower without cache?

@lorenzwalthert
Copy link
Collaborator

We could go with two parallel files, and activate only one of them depending on some switch. Do you prefer that?

Well that means more code to maintain. What's exactly the need for clean(er) stack traces than what we have now? If the analysis of the traces is a more or less one-off thing, then I don't think we should introduce meta programming into the default branch.

@lorenzwalthert
Copy link
Collaborator

Do I read the benchmark correctly: we're faster when applying the cache, but slower without cache?

Yes, but you see the confidence interval contains zero, so it's not a significant change. Gray check boxes in {touchstone} just mean no significant deviation in either direction.

@krlmlr
Copy link
Member Author

krlmlr commented Aug 21, 2022

The stack traces also affect error messages as in rlang::last_trace(). I think it's a win.

We're talking about the two functions pre_visit() and post_visit(), only those would be tested in a separate GHA matrix, yes. I don't mind keeping it in a branch for now, as long as the other PRs are good.

@krlmlr
Copy link
Member Author

krlmlr commented Aug 21, 2022

"Not significant" might just mean "we need more data to be sure". I do believe that the meta code takes a tad longer overall.

@lorenzwalthert
Copy link
Collaborator

The stack traces also affect error messages as in rlang::last_trace(). I think it's a win.

Our end users should not see these anyways.

@lorenzwalthert
Copy link
Collaborator

We're talking about the two functions pre_visit() and post_visit(), only those would be tested in a separate GHA matrix, yes. I don't mind keeping it in a branch for now, as long as the other PRs are good.

Other PRs are good yes, thanks again.

@lorenzwalthert
Copy link
Collaborator

lorenzwalthert commented Aug 22, 2022

Yes, you can run {touchstone} for more iterations, then you'll always get a statistically significant result. However, if the change is +0.1% with a confidence interval around it of < 0.001%, it's not really significant from a user experience point of view, although statistically different from 0. So we tried to run for as many iterations that the two significance dimension align (so if user experience is slowed down significantly, it also creates statistical significance). I believe from a speed perspective, the changes your PR introduces are not problematic.

@krlmlr krlmlr marked this pull request as draft September 11, 2022 05:32
@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 2c2aaaf is merged into main:

  •   :ballot_box_with_check:cache_applying: 31.7ms -> 31.5ms [-1.37%, +0.02%]
  • ❗🐌cache_recording: 1.45s -> 1.49s [+1.87%, +2.83%]
  • ❗🐌without_cache: 3.87s -> 3.9s [+0.44%, +1.17%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if e222ace is merged into main:

  •   :ballot_box_with_check:cache_applying: 31.4ms -> 31.2ms [-1.62%, +0.98%]
  •   :ballot_box_with_check:cache_recording: 1.47s -> 1.46s [-3.64%, +2.16%]
  •   :ballot_box_with_check:without_cache: 3.77s -> 3.79s [-0.44%, +1.52%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if ec7ca0a is merged into main:

  •   :rocket:cache_applying: 39.3ms -> 37.4ms [-8.31%, -1.22%]
  • ❗🐌cache_recording: 1.86s -> 1.89s [+0.76%, +3.03%]
  •   :ballot_box_with_check:without_cache: 5.22s -> 5.25s [-0.31%, +1.11%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

github-actions bot commented Nov 7, 2022

This is how benchmark results would change (along with a 95% confidence interval in relative change) if a98c069 is merged into main:

  • ❗🐌cache_applying: 23.8ms -> 27.4ms [+14.46%, +15.75%]
  • ❗🐌cache_recording: 754ms -> 774ms [+2.29%, +3.05%]
  • ❗🐌without_cache: 1.89s -> 1.9s [+0.1%, +0.83%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 940010d is merged into main:

  •   :ballot_box_with_check:cache_applying: 210ms -> 212ms [-0.26%, +1.54%]
  • ❗🐌cache_recording: 957ms -> 987ms [+2.07%, +4.32%]
  • ❗🐌without_cache: 1.97s -> 1.99s [+0.63%, +1.4%]

Further explanation regarding interpretation and methodology can be found in the documentation.

@github-actions
Copy link
Contributor

This is how benchmark results would change (along with a 95% confidence interval in relative change) if 0592e8e is merged into main:

  •   :ballot_box_with_check:cache_applying: 289ms -> 291ms [-1.77%, +3.07%]
  • ❗🐌cache_recording: 1.3s -> 1.35s [+2.18%, +4.62%]
  •   :ballot_box_with_check:without_cache: 2.44s -> 2.44s [-1.28%, +1.59%]

Further explanation regarding interpretation and methodology can be found in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants