WIP: K2IS support #22
Open
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
sk1p
force-pushed
the
k2is
branch
2 times, most recently
from
7472ec9
to
366d2a4
Compare
sk1p
force-pushed
the
k2is
branch
2 times, most recently
from
559f933
to
2c3d13c
Compare
- timestamp_recorder: set local_addr depending on sector - ipc_test: pre-fault the whole shm area - warmup: increase number of blocks sent through - add a "circuit breaker" if the `assembly_channel` fills up too fast - `k2is_live.py`: allow setting a filename pattern and controlling the synchronization mode
Still slightly broken, but useful still for starting to receive multicast traffic (for example for capturing), or for testing the receive code in isolation.
* Split frame module into base, is, and summit modules * main: allocate shm with correct slot size * Fix `decode_u16_vec` to correctly take header into account * Fully implement `K2Frame` trait for `K2SummitFrame` * Slightly improve error handling; shut down assembly threads when one of the workers has died * Lots of clippy fixes
Meaning, work towards multiple acquisitions after another with minimal delay inbetween.
Otherwise, the recv on the second channel will, in the worst case, be limited to the timeout of the first recv. Also work on fixing the missing first frame issue.
Also some cleanup..
If a frame is 'dropped multiple times', that means the acquisition index is counted up too often, which can then result in the re-ordering buffer being non-empty at the end of an acquisition, which is not expected.
It's a lot faster to just use `getattr`/`call0` to call Python functions instead of compiling a Python module each time the function is called.
Otherwise, there is a high chance of having multiple dropped frames at the beginning of an acquisition, because receiving from multicast sockets does not initialize instantaneously.
Extract some configuration into `AcquisitionRuntimeConfig`
Remove non-workspace Cargo.lock files; remove unused `rayon` dependency from `k2o`.
... which is an old name for `libertem_k2is`.
Instead, writing will be implemented as part of the common infrastructure.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Opened to keep track of TODOs.
TODO
ipc-testversionK2Frameinto a thread-local variant with aSlotForWriting(K2FrameForWriting?), and an immutable variant to pass around for further processingmultiversionusage for especially hot functionsparallel-udp-replay-Ctarget-cpu=nativefor optimal performanceparallel-udp-replay