Fixes for speedier Python-Julia interaction #32
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rmshaffer
reviewed
Aug 27, 2024
Comment on lines
+80
to
+86
| def setup_pool(): | ||
| global __JULIA_POOL__ | ||
| __JULIA_POOL__ = Pool(processes=1) | ||
| __JULIA_POOL__.apply(setup_julia) | ||
| atexit.register(__JULIA_POOL__.join) | ||
| atexit.register(__JULIA_POOL__.close) | ||
| return |
There was a problem hiding this comment.
is this thread safe? what if two Python threads call this at the same time? (i.e. should you wrap this in some kind of lock?)
Co-authored-by: Ryan Shaffer <[email protected]>
Co-authored-by: Ryan Shaffer <[email protected]>
AbeCoull
reviewed
Aug 27, 2024
.github/workflows/benchmark.yml
Outdated
| - name: Set up Python | ||
| uses: actions/setup-python@v5 | ||
| with: | ||
| python-version: 3.9 |
There was a problem hiding this comment.
Do we want to benchmark across multiple python version?
There was a problem hiding this comment.
We certainly can, I just wanted to be sparing with the amount of resource use at first.
|
|
||
| def setup_pool(): | ||
| global __JULIA_POOL__ | ||
| __JULIA_POOL__ = Pool(processes=1) |
There was a problem hiding this comment.
Assuming this will be used by the batched executions, should this take in a param for setup_pool called processes which has a default of 1?
There was a problem hiding this comment.
No, because Julia handles the batches itself behind the scenes. We only need one process to chat back and forth with Julia.
rmshaffer
approved these changes
Aug 27, 2024
|
Moved benchmarking to a new PR: #35 |
kshyatt-aws
added a commit
that referenced
this pull request
Aug 28, 2024
* change: nobraket branch * change: Remove threading tests thanks to new pythoncall version * fix: make tests compatible with xdist * fix: Use proper BraketSimulator.jl version * Use a `ProcessPoolExecutor` to talk to Julia (#30) * fix: Use new BraketSimulator.jl version * fix: Force a Julia registry update in CI to catch new versions * update tests (#31) * update tests * add a test optimization * refactor pyproject file to have test dependencies in their own file * Update requirements-test.txt * Update requirements-test.txt --------- Co-authored-by: Coull <[email protected]> * Fixes for speedier Python-Julia interaction (#32) * change: Use multiprocessing Pool rather than ProcessPoolExecutor * fix: No sigterm screaming * fix: Force close and join at exit * fix: Faster * fix: handle julia error in serializable way * change: Add benchmark workflow to CI * change: Point to latest unblocking branch * fix: linting * fix: Don't turn off juliapkg in tox * Update src/braket/simulator_v2/julia_workers.py Co-authored-by: Ryan Shaffer <[email protected]> * Update src/braket/simulator_v2/julia_workers.py Co-authored-by: Ryan Shaffer <[email protected]> * fix: one-line pip in benchmark * fix: typo * fix: Use benchmark-json option correctly * change: Add initial output.json * fix: Actually include benchmark script * fix: Don't deploy benchmark results to gh-pages * fix: restore GH token * fix: Remove benchmarks for now * change: Point to new BraketSimulator 0.0.4 --------- Co-authored-by: Ryan Shaffer <[email protected]> * fix: Don't turn off pyjuliapkg to make sure dpes do install * fix: Don't return the julia Main module to unlock jupyter * fix: Return to not using JuliaError type in case Julia isn't loaded --------- Co-authored-by: Coull <[email protected]> Co-authored-by: Abe Coull <[email protected]> Co-authored-by: Ryan Shaffer <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Issue #, if available: N/A
Description of changes:
Testing done:
toxpassed locallyMerge Checklist
Put an
xin the boxes that apply. You can also fill these out after creating the PR. If you're unsure about any of them, don't hesitate to ask. We're here to help! This is simply a reminder of what we are going to look for before merging your pull request.General
Tests
By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.