Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimise: avoid payload copy in produce flow. #67

Open
kmrdhruv opened this issue Dec 18, 2023 · 1 comment
Open

Optimise: avoid payload copy in produce flow. #67

kmrdhruv opened this issue Dec 18, 2023 · 1 comment
Assignees
Milestone

Comments

@kmrdhruv
Copy link
Collaborator

File - server/src/main/java/com/flipkart/varadhi/web/v1/produce/ProduceHandlers.java
method - produce()
Below needs to be fixed.

    // TODO:: Below is making extra copy, this needs to be avoided.
    // ctx.body().buffer().getByteBuf().array() -- method gives complete backing array w/o copy,
    // however only required bytes are needed. Need to figure out the correct mechanism here.
    byte[] payload = ctx.body().buffer().getBytes();

Currently above code is doing a copy of the produce payload and adds same to the message buffer. This copy of payload is extra and can be avoided.

Expected:
Investigate, how underlying buffer reference can be passed in the message instead of copy. Intent is to avoid copy of data in Varadhi and instead pass input buffer directly to the underlying messaging stack. Reason for doing this is primarily performance.

Investigate and propose 1 pager for the required changes before starting on implementation.

@kmrdhruv kmrdhruv added this to Varadhi Dec 18, 2023
@kmrdhruv kmrdhruv moved this to Backlog in Varadhi Dec 18, 2023
@kmrdhruv kmrdhruv added Task and removed Task labels Dec 18, 2023
@sahil-chachan sahil-chachan added this to the Produce milestone Dec 20, 2023
@kmudgal25 kmudgal25 moved this from Backlog to In Review in Varadhi Jan 8, 2024
@kmudgal25
Copy link
Contributor

@kmudgal25 kmudgal25 moved this from In Review to Done in Varadhi Jan 10, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
Status: Done
Development

No branches or pull requests

3 participants