Skip to content

Latest commit

 

History

History
128 lines (96 loc) · 5.31 KB

spec_6.adoc

File metadata and controls

128 lines (96 loc) · 5.31 KB
Raw

6/Flux Remote Procedure Call Protocol

This specification describes how Flux Remote Procedure Call (RPC) is built on top of request and response messages defined in RFC 3.

  • Name: github.com/flux-framework/rfc/spec_6.adoc

  • Editor: Jim Garlick <[email protected]>

  • State: raw

Language

The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in RFC 2119.

Goals

Flux RPC protocol enables comms modules, utilities, or other software communicating with a Flux comms session to call the methods implemented by comms modules. Flux RPC has the following goals:

  • Support location-neutral service addressing, without a location broker.

  • Support a high degree of concurrency in both clients and servers

  • Avoid over-engineered mitigations for timeouts, congestion avoidance, etc. that can be a liability in high performance computing environments.

  • Provide a mechanism to abort in-progress RPC calls.

Implementation

A remote procedure call SHALL consist of one request message sent from a client to a server, and zero or one response messages sent from a server to a client. The client and server roles are not mutually-exclusive—​comms modules often act in both roles.

+--------+    Request      +--------+
|        | --------------> |        |
| Client |                 | Server |
|        | <-------------- |        |
+--------+    Response     +--------+

Per RFC 3, the request message SHALL include a nodeid and topic string used to aid the broker in selecting appropriate routes to the server. The client MAY address the request in a location-neutral manner by setting nodeid to FLUX_NODEID_ANY, then the tree-based overlay network will be followed to the root looking for a matching service closest to the client.

The server SHALL send zero or one responses to each request, as established by prior agreement between client and server (e.g. defined in their protocol specification). Responses SHALL contain topic string and matchtag values copied from the request, to facilitate client response matching. Responses MAY be sent in any order.

The server SHALL set errnum in the response to zero to indicate success, or a nonzero value to indicate failure, using POSIX.1 errno encoding.

Payload frames are OPTIONAL in both request and response messages. However, a response with errnum set to a nonzero value SHALL NOT return a payload frame.

Matchtag Field

RFC 3 provisions request and response messages with a 32-bit matchtag field. The client MAY assign a unique (to the client) value to this field, which SHALL be echoed back by the server in responses. The client MAY use this matchtag value to correlate responses to its concurrently outstanding requests.

Note that matchtags are only unique to the client. Servers SHALL NOT use matchtags to track client state unless paired with the client UUID.

The client MAY set matchtag to FLUX_MATCHTAG_NONE (0) if it has no need to correlate responses in this way, or a response is not expected.

Group RPC

A group RPC is a request sent to multiple nodeids.

The upper 12 bits of the matchtag field are designated as the group id portion of the matchtag. If the group id is nonzero, the lower 20 bits SHALL be ignored in response matching and pass through to the group RPC implementation, which MAY use the lower 20 bits in an implementation dependent way; for example, as a key to a lookup table.

Exceptional Conditions

If a request cannot be delivered to the server, the broker MAY respond to the sender with an error. For example, per RFC 3, a broker SHALL respond with error number 38 "Function not implemented" if the topic string cannot be matched to a service, or error number 113, "No route to host" if the requested nodeid cannot be reached.

Although overlay networks use reliable transports between brokers, exceptional conditions at the endpoints or at intervening broker instances MAY cause messages to be lost. It is the client’s responsibility to implement any timeouts or other mitigation to handle missing or delayed responses.

Cancellation

If a client wishes to give up on an in-progress RPC, it MAY send a request to the server with a topic string of "service.disconnect".

It is optional for the server to implement the disconnect method. As usual, if the method is unimplemented, the server SHALL respond with error number 38, "Function not implemented".

If the server implements the disconnect method, it SHALL cancel any pending RPC requests from the sender, without responding to them, and respond to the disconnect request with success, no payload. Upon receipt of a successful disconnect response, the client MAY reuse the canceled messages' matchtags, if any.

The server MAY determine the sender identity for any request by reading the first source-address routing identity frame (closest to routing delimiter frame) from the request message. Servers which respond to requests out of order SHOULD retain state for pending requests, allowing them to be canceled by sender id as described above.