Dedicated Resource Manager, expose QueuePairs #272
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
facebook-github-bot
added
the
CLA Signed
|
This pull request was exported from Phabricator. Differential Revision: D76370567 |
Summary: Pull Request resolved: pytorch-labs#272 This diff refactors the RDMA codebase to introduce the RdmaManagerActor as a dedicated resource manager for each endpoint, responsible for managing memory mappings and related communicators (QueuePairs). Currently, the process is simple: we use CPU memory and expose the entire address space to the NIC, with a single QueuePair per route (i.e., all traffic between endpoints A and B goes through the same QP). As we move toward supporting GPUs, the requirements will become more complex. This refactor cleans up the API and abstractions in preparation for that transition. In the new model, RdmaBuffers are extremely lightweight and should only be created by RdmaManagerActors. It's important to note that an RdmaBuffer is designed to carry only the minimum data necessary to the remote caller; under the hood, an RdmaBuffer represents a slice of a registered memory region with associated rkey and lkey. Moving data between endpoints requires using the RdmaQueuePair's GET/PUT API. To obtain a QueuePair, it must be requested from an RdmaManagerActor. core API for RdmaManagerActor: request_buffer(addr: u32, size: usize) -> RmdaBuffer. // This acts a light-weight memory handle that can be serialized. In practice this a slice of IBV MR maintained by RdmaManagerActor request_queue_pair(remote: ActorRef<RdmaManagerActor> -> RmdaQueuePair // light-weight wrapper around QP. Example API usage: // setup envs, send remote_buffer -> local env. // establish RMDA connection with Peer if required. let qp = local_rmda_manager.request_queue_pair(client, remote_rmda_manager).await()? // get/put api example, returns immediately. qp.get(local_buffer, remote_buffer)?; // ... poll for completion Differential Revision: D76370567
|
This pull request was exported from Phabricator. Differential Revision: D76370567 |
dstaay-fb
force-pushed
the
export-D76370567
branch
from
4ad43f2 to
4b53308
Compare
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary:
This Diff refactors RDMA codebase to have RmdaManagerActor to act as a dedicated resource manager for each endpoint; and manages memory mappings and related communicators (QueuePairs). Today this process is relatively simple (we are using CPU, and exposing entire address space to NIC), and assume one QueuePair per route (ie. all traffic from A<->B will go though same QP), but as we transition supporting GPUs it will get more complex, so cleaning up api and abstractions now will help.
In this new model, RdmaBuffer's are extremely light-weight and should only be created by RdmaManagerActors. Its worth highlighting the RmdaBuffer is really designed to move the minimum data to remote caller, under the hood, RmdaBuffers represent a slice of a Memory Region w/ a given rkey/lkey.
Moving buffers requires leveraging RmdaQueuePairs GET/PUT API; again to obtain an QP it must be requested from an RmdaManagerActor.
core API for RdmaManagerActor:
request_buffer(addr: u32, size: usize) -> RmdaBuffer. // This acts a light-weight memory handle that can be serialized. In practice this a slice of IBV MR maintained by RdmaManagerActor
Example API usage:
// setup envs, send remote_buffer -> local env.
// establish RMDA connection with Peer if required.
let qp = local_rmda_manager.request_queue_pair(client, remote_rmda_manager).await()?
// get/put api example, returns immediately.
qp.get(local_buffer, remote_buffer)?;
// ... poll for completion
Differential Revision: D76370567