fix: Run batch execution after session starts #2327
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Your org has enabled the Graphite merge queue for merging into mainAdd the label “flow:merge-queue” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “flow:hotfix” to add to the merge queue as a hot fix. You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link. |
github-actions
bot
added
comp:manager
fregataa
force-pushed
the
fix/run-batch-execution-after-session-starts
branch
from
82d6f94
to
75e93a6
Compare
achimnol
approved these changes
Jul 10, 2024
achimnol
reviewed
Jul 10, 2024
There was a problem hiding this comment.
hm... in my setup, it seems not working well. Investigating more.
- Missing session events in the event stream (only kernel events)
- According to the manager log, "SessionStarted" event is fired.
- There is no way to check if the batch execution task is actually running or not. The container log shows nothing?
Update:
After restarting my dev services, it seems to work as expected, except that SessionStarted events are often missing. 🤔
…ssion-starts' into fix/run-batch-execution-after-session-starts
achimnol
requested changes
Jul 10, 2024
achimnol
approved these changes
Jul 13, 2024
lablup-octodog
pushed a commit
that referenced
this pull request
Jul 13, 2024
Co-authored-by: Joongi Kim <[email protected]> Backported-from: main (24.09) Backported-to: 24.03 Backport-of: 2327
github-merge-queue bot
pushed a commit
that referenced
this pull request
Jul 13, 2024
Co-authored-by: Sanghun Lee <[email protected]> Co-authored-by: Joongi Kim <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
resolves #2326
How the lifetime of the batch session changes
RUNNING, it callssrc.ai.backend.manager.registry.trigger_batch_execution()API.trigger_batch_execution()RPC function to an agent where a main kernel is allocated.SessionSuccessevent when the batch execution finishes and a manager that consumes the event destroys the batch session.Checklist: (if applicable)