Chunks are the transport capsules in iceoryx. They store data from a publisher as payload and are sent to one or more subscriber. Furthermore, there is some meta information which is stored alongside the payload, e.g. the size and the origin of the chunk. This data is composed in the ChunkHeader and located at the front of the chunk. Custom meta information can be added to extend the data from the ChunkHeader and tailor the chunk to specific use cases. While this makes the chunk layout more complex, this complexity would otherwise be distributed at different locations all over the code base, e.g. in the request/response feature. Additionally, the adjustments for the user-header makes arbitrary alignments of the user-payload trivial to implement.
| Name | Description |
|---|---|
| Chunk | a piece of memory |
| Chunk-Header | contains meta information related to the chunk |
| Chunk-Payload | the part of the chunk for the payload which is split into user-header and user-payload |
| User-Header | contains custom meta information, e.g. timestamps |
| User-Payload | the user data with custom alignment |
| Back-Offset | offset stored in front of the user-payload to calculate back to the chunk-header (for the most simple case it will overlap with the user-payload offset stored in the Chunk-Header) |
Framing with terminology
+=============================================================================+
| Chunk |
+===================+=========================================================+
| | Chunk-Payload |
| Chunk-Header +===============+=======+====================+============+
| | User-Header | ¦ ᶺ | User-Payload | Padding |
+===================+===============+=====|=+====================+============+
└ Back-Offset
- it's not uncommon to record chunks for a later replay -> detect incompatibilities on replay
- iceoryx runs on multiple platforms -> endianness of recorded chunks might differ
- for tracing, a chunk should be uniquely identifiable -> store origin and sequence number
- the chunk is located in the shared memory, which will be mapped to arbitrary positions in the address space of various processes -> no absolute pointer are allowed
- in order to reduce complexity, the alignment of the user-header must not exceed the alignment of the
ChunkHeader
class ChunkHeader
{
uint32_t chunkSize;
uint8_t chunkHeaderVersion;
uint8_t reserved{0};
uint16_t userHeaderId;
popo::UniquePortId originId; // underlying type = uint64_t
uint64_t sequenceNumber;
uint32_t userHeaderSize{0U};
uint32_t userPayloadSize{0U};
uint32_t userPayloadAlignment{1U};
UserPayloadOffset_t userPayloadOffset; // alias to uint32_t
};
- chunkSize is the size of the whole chunk
- chunkHeaderVersion is used to detect incompatibilities for record&replay functionality
- reserved is currently not used and set to
0 - userHeaderId is currently not used and set to
NO_USER_HEADER - originId is the unique identifier of the publisher the chunk was sent from
- sequenceNumber is a serial number for the sent chunks
- userHeaderSize is the size of the chunk occupied by the user-header
- userPayloadSize is the size of the chunk occupied by the user-payload
- userPayloadAlignment is the alignment of the chunk occupied by the user-payload
- userPayloadOffset is the offset of the user-payload relative to the begin of the chunk
For back calculation from the user-payload pointer to the ChunkHeader pointer, the user-payload offset must be accessible from a defined position relative to the user-payload. Lets call this back-offset. This is solved by storing the offset in the 4 bytes in front of the user-payload. In case of a simple layout where the ChunkHeader is adjacent to the user-payload, this nicely overlaps with the position of userPayloadOffset and no memory is wasted. In more complex cases, the offset has to be stored a second time. If the user-payload alignment requires some padding from the header extension, this memory is used to store the offset.
- No user-header and user-payload alignment doesn't exceed the
ChunkHeaderalignment
sizeof(ChunkHeader) userPayloadSize
|------------------>|--------------------->|
| | |
+===================+======================+==================================+
| Chunk-Header ¦ * | User-Payload | Padding |
+===================+======================+==================================+
| | |
| userPayloadOffset | |
|------------------>| |
| chunkSize |
|---------------------------------------------------------------------------->|
*) userPayloadOffset from ChunkHeader and back-offset are overlapping
- No user-header and user-payload alignment exceeds the
ChunkHeaderalignment
sizeof(ChunkHeader) back-offset userPayloadSize
|------------------>| |<---|------------------->|
| | | | |
+===================+=======================+====================+============+
| Chunk-Header | ¦ | User-Payload | Padding |
+===================+=======================+====================+============+
| | |
| userPayloadOffset | |
|------------------------------------------>| |
| |
| chunkSize |
|---------------------------------------------------------------------------->|
Depending on the address of the chunk there is the chance that ChunkHeader is the still adjacent to the user-payload. In this case, the framing looks exactly like in case 1.
- User-Header is used
sizeof(ChunkHeader) back-offset userPayloadSize
|------------------>| |<---|------------------->|
| | | | |
+===================+===============+=======+====================+============+
| Chunk-Header | User-Header | ¦ | User-Payload | Padding |
+===================+===============+=======+====================+============+
| | |
| userPayloadOffset | |
|------------------------------------------>| |
| |
| chunkSize |
|---------------------------------------------------------------------------->|
- No user-header and user-payload alignment doesn't exceed the
ChunkHeaderalignment
userPayloadOffset = sizeof(ChunkHeader);
- No user-header and user-payload alignment exceeds the
ChunkHeaderalignment, which means the user-payload is either adjacent to theChunkHeaderor there is a padding with at least the size ofChunkHeaderalignment in front of the user-payload and therefore enough space to store theback-offset
chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader);
alignedUserPayloadAddress = align(chunkHeaderEndAddress, userPayloadAlignment);
userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader);
- User-Header is used
chunkHeaderEndAddress = addressof(chunkHeader) + sizeof(chunkHeader) + sizeof(userHeader);
anticipatedBackOffsetAddress = align(chunkHeaderEndAddress, alignof(userPayloadOffset));
unalignedUserPayloadAddress = anticipatedBackOffsetAddress + sizeof(userPayloadOffset);
alignedUserPayloadAddress = align(unalignedUserPayloadAddress, userPayloadAlignment);
userPayloadOffset = alignedUserPayloadAddress - addressof(chunkHeader);
In order to fit the user-header and the user-payload into the chunk, a worst case calculation has to be done. We can assume that a chunk is sufficiently aligned for the ChunkHeader.
- No user-header and user-payload alignment doesn't exceed the
ChunkHeaderalignment
chunkSize = sizeof(chunkHeader) + userPayloadSize;
- No user-header and user-payload alignment exceeds the
ChunkHeaderalignment
Worst case scenario is when a part of the ChunkHeader crosses the user-payload alignment boundary, so that the user-payload must be aligned to the next boundary. The following drawing demonstrates this scenario.
┌ back-offset
+===============+==========|+===============================+
| ChunkHeader | ¦ᵛ| User-Payload |
+===============+===========+===============================+
⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries
⊥ ⊥ ⊥ ⊥ <- user-payload alignment boundaries
<-----------|--------------->
pre user-payload |------------------------------->
user-payload alignment user-payload size
alignment
overhang
The following formula is used to calculate the required chunk size.
preUserPayloadAlignmentOverhang = sizeof(chunkHeader) - alignof(chunkHeader);
chunkSize = preUserPayloadAlignmentOverhang + userPayloadAlignment + userPayloadSize;
- User-Header is used
Similar to case 2, but in this case it is the back-offset which might cross the user-payload alignment boundary.
┌ back-offset with same alignment
| as userPayloadOffset
+===============+===========+==============|+===============================+
| ChunkHeader | UserHeader| ¦ᵛ| User-Payload |
+===============+===========+===============+===============================+
⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- userPayloadOffset alignment boundaries
⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ ⊥ <- ChunkHeader alignment boundaries
⊥ ⊥ ⊥ ⊥ <- userPayload alignment boundaries
<---------------------------|--------------->
pre user-payload alignment user-payload |------------------------------->
overhang alignment user-payload size
The following formula is used to calculate the required chunk size.
chunkHeaderSize = sizeof(chunkHeader) + sizeof(userHeader)
preUserPayloadAlignmentOverhang = align(chunkHeaderSize, alignof(userPayloadOffset));
maxPadding = max(sizeof(userPayloadOffset), userPayloadAlignment);
chunkSize = preUserPayloadAlignmentOverhang + maxPadding + userPayloadSize;
The ChunkHeader has a template method to get user-header. There is a risk to use this wrong by accident, but there is currently no better solution for this.
Since the user-header is always adjacent to the ChunkHeader, the formula to get the user-header is:
userHeader = addressOf(chunkHeader) + sizeof(chunkHeader);
void* userPayload()returns a pointer to the user-payloadtemplate <typename T> T* userHeader()returns a pointer to the user-headerstatic ChunkHeader* fromUserPayload(const void* const userPayload)returns a pointer to theChunkHeaderassociated to the user-payload
The Publisher has additional template parameters for the user-header and user-payload alignment. Alternatively this could also be done with the allocate method, but that increases the risk of using it wrong by accident.
Additionally, to fully integrate the user-header into the regular control flow, a pointer to a ChunkHeaderHook can be passed to the Publisher c'tor.
The ChunkHeaderHook is not in the shared memory and has the following virtual methods:
virtual void allocateHook(ChunkHeader& chunkHeader)can be used to initialize the user-headervirtual void deliveryHook(ChunkHeader& chunkHeader)can be used to set members of the user-header, e.g. a timestamp
Furthermore, the Publisher and Subscriber have access to the ChunkHeader and can use the userHeader() method to gain access to the user-header.
- when the user-payload is adjacent to the
ChunkHeader, it must be ensured thatuserPayloadOffsetoverlaps with theback-offset, which issizeof(userPayloadOffset)in front of the user-payload - to simplify calculation, it is assumed that the alignment of the user-header doesn't exceed the alignment of the
ChunkHeader. This has to be enforced with anassert
- a
ChunkHeaderHookcould be used on the publisher side
template <typename UserHeader>
class MyChunkHeaderHook : public ChunkHeaderHook
{
public:
void deliveryHook(ChunkHeader& chunkHeader) override
{
chunkHeader.userHeader<UserHeader>().timestamp = myTimeProvider::now();
}
};
auto userHeaderHook = MyChunkHeaderHook<MyUserHeader>();
auto pub = iox::popo::Publisher<MyPayload>(serviceDescription, userHeaderHook);
- alternatively, instead of the ChunkHeaderHook class, the publisher could have a method `registerDeliveryHook(std::function<void(ChunkHeader&)>)`
- allow the user only read access to the `ChunkHeader` and write access to the `UserHeader`
- user defined sequence number
- this can probably be done by a
ChunkHeaderHook - alternatively, a flag could be introduce into the
ChunkHeader
- this can probably be done by a
- user defined timestamp
- this should probably be in a user-header
- mempool configuration
- currently we specify the chunk-payload size and the size of the
ChunkHeaderis added automatically - with the new approach, part of the specified chunk-payload size might be used for the user-header
- part of the specified chunk-payload might also be used as padding for the user-payload alignment
- the user will continue to specify the chunk-payload; if a user-header or custom user-payload alignment is used, the user needs to take this into account
- currently we specify the chunk-payload size and the size of the
- is it necessary to store a flag in
ChunkHeaderif a user extension is used?- we could maintain a list of known user-header IDs or ranges of IDs similar to
IANAhttps://datatracker.ietf.org/doc/id/draft-cotton-tsvwg-iana-ports-00.html#privateports - the IDs could be stored in the
ChunkHeaderand everything with an ID larger than0xC000is free to use - to make this somewhat save, the
ChunkHeaderHookmust be mandatory with e.g. avirtual uint16_t getId() = 0;method which will be called in theChunkSender- alternatively, the user-header struct must have a
constexpr uint16_t USER_HEADER_ID; if it's not present, we could set the ID to0xC000which is the first ID free to use
- alternatively, the user-header struct must have a
- we could maintain a list of known user-header IDs or ranges of IDs similar to
- do we want to store the version of the user extension in the
ChunkHeader, similarly tochunkHeaderVersion - for record&replay the user-header is totally opaque, which might lead to problems if e.g. a timestamp is stored in the user-header and needs to be updated for the replay
- if we maintain a list of known user-header IDs and also store the user-header version, a record&replay framework could implement the required conversions
- for record&replay it is necessary to store the alignment of the user-payload; decide if a uint16_t should be used of if just the alignment power should be stored in a uint8_t