Skip to content

Commit

Permalink
A bit of documentation
Browse files Browse the repository at this point in the history
  • Loading branch information
nibanks committed Dec 6, 2024
1 parent f73a471 commit b743020
Show file tree
Hide file tree
Showing 2 changed files with 69 additions and 3 deletions.
62 changes: 62 additions & 0 deletions docs/Execution.md
Original file line number Diff line number Diff line change
Expand Up @@ -109,3 +109,65 @@ graph TD
end
end
```

## Custom Execution

MsQuic also supports scenarios where the application layer creates the threads that MsQuic uses to execute on.
In this mode, the application creates one or more execution contexts that MsQuic will use to run all its internal logic.
The application is responsible for calling down into MsQuic to allow these execution contexts to run.

To create an execution context, the app much first create an event queue object, which is a platform specific type:

- Windows: IOCP
- Linux: epoll
- macOS: kqueue

On Windows, the following types are defined:

```c
typedef HANDLE QUIC_EVENTQ;

typedef struct QUIC_CQE {
OVERLAPPED_ENTRY Overlapped;
void (*Completion)(struct QUIC_CQE *Cqe);
} QUIC_CQE;
```

You will also notice the definiton for `QUIC_CQE` (CQE stands for completion queue event), which defines the format that all completion events must take so they may be generically processed from the event queue (more on this below).

Once the app has the event queue, it may create the execution context with the `ExecutionCreate` function:

```c
HANDLE IOCP = CreateIoCompletionPort(INVALID_HANDLE_VALUE, nullptr, 0, 1);
QUIC_EXECUTION_CONTEXT_CONFIG ExecConfig = { 0, &IOCP };

QUIC_EXECUTION_CONTEXT* ExecContext = nullptr;
QUIC_STATUS Status = MsQuic->ExecutionCreate(QUIC_EXECUTION_CONFIG_FLAG_NONE, 0, 1, &ExecConfig, &ExecContext);
```
The above code createa a new IOCP (for Windows), sets up an execution config, indicating an ideal processor of 0 and the pointer to the IOCP, and then calls MsQuic to create 1 execution context.
An application may expand this code to create multiple execution contexts, depending on their needs.
To drive this execution context, the app will need to to periodically call `ExecutionPoll` and use the platform specific function to drain completion events from the event queue.
```c
bool AllDone = false;
while (!AllDone) {
uint32_t WaitTime = MsQuic->ExecutionPoll(ExecContext);
ULONG OverlappedCount = 0;
OVERLAPPED_ENTRY Overlapped[8];
if (GetQueuedCompletionStatusEx(IOCP, Overlapped, ARRAYSIZE(Overlapped), &OverlappedCount, WaitTime, FALSE)) {
for (ULONG i = 0; i < OverlappedCount; ++i) {
QUIC_CQE* Cqe = CONTAINING_RECORD(Overlapped[i].lpOverlapped, QUIC_CQE, Overlapped);
Cqe->Completion(Cqe);
}
}
}
```

Above, you can see a simple loop that properly drives a single execution context on Windows.
`OVERLAPPED_ENTRY` objects received from `GetQueuedCompletionStatusEx` are used to get the completion queue event and then call its completion handler.

In a real application, these completion events may come both from MsQuic and the application itself, therefore, this means **the application must use the same base format for its own completion events**.
This is necessary to be able to share the same event queue object.
10 changes: 7 additions & 3 deletions src/tools/execution/execution_windows.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -39,10 +39,14 @@ main(
if (!_MsQuic.IsValid()) { return 1; }
MsQuic = &_MsQuic;

QUIC_STATUS Status;
IOCP = CreateIoCompletionPort(INVALID_HANDLE_VALUE, nullptr, 0, 1);
QUIC_EXECUTION_CONTEXT_CONFIG ExecConfig = { 0, &IOCP };
if (IOCP == nullptr) {
return 1;
}

QUIC_STATUS Status;
QUIC_EXECUTION_CONTEXT* ExecContext = nullptr;
QUIC_EXECUTION_CONTEXT_CONFIG ExecConfig = { 0, &IOCP };
if (QUIC_FAILED(Status = MsQuic->ExecutionCreate(QUIC_EXECUTION_CONFIG_FLAG_NONE, 0, 1, &ExecConfig, &ExecContext))) {
return 1;
}
Expand Down Expand Up @@ -81,8 +85,8 @@ main(
while (!AllDone) {
uint32_t WaitTime = MsQuic->ExecutionPoll(ExecContext);

OVERLAPPED_ENTRY Overlapped[8];
ULONG OverlappedCount = 0;
OVERLAPPED_ENTRY Overlapped[8];
if (GetQueuedCompletionStatusEx(IOCP, Overlapped, ARRAYSIZE(Overlapped), &OverlappedCount, WaitTime, FALSE)) {
for (ULONG i = 0; i < OverlappedCount; ++i) {
QUIC_CQE* Cqe = CONTAINING_RECORD(Overlapped[i].lpOverlapped, QUIC_CQE, Overlapped);
Expand Down

0 comments on commit b743020

Please sign in to comment.