Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(pageserver): add direct io pageserver config #8622

Merged
merged 2 commits into from
Aug 7, 2024

Conversation

yliang412
Copy link
Contributor

@yliang412 yliang412 commented Aug 6, 2024

Part of #8130, RFC: Direct IO For Pageserver

Description

Add pageserver config for evaluating/enabling direct I/O.

  • Disabled: current default, uses buffered io as is.
  • Evaluate: still uses buffered io, but could do alignment checking and perf simulation (pad latency by direct io RW to a fake file).
  • Enabled: uses direct io, behavior on alignment error is configurable.

Checklist before requesting a review

  • I have performed a self-review of my code.
  • If it is a core feature, I have added thorough tests.
  • Do we need to implement analytics? if so did you add the relevant metrics to the dashboard?
  • If this PR requires public announcement, mark it with /release-notes label and add several sentences in this section.

Checklist before merging

  • Do not forget to reformat commit message to not include the above checklist

@yliang412 yliang412 changed the title feat(pageserver): add direct io pageserver config [WIP] feat(pageserver): add direct io pageserver config Aug 6, 2024
@yliang412 yliang412 self-assigned this Aug 6, 2024
Copy link

github-actions bot commented Aug 6, 2024

2112 tests run: 2043 passed, 0 failed, 69 skipped (full report)


Code coverage* (full report)

  • functions: 32.5% (7157 of 22003 functions)
  • lines: 50.5% (57850 of 114500 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
7b0689c at 2024-08-07T19:54:02.903Z :recycle:

@yliang412 yliang412 added c/storage/pageserver Component: storage: pageserver t/feature Issue type: feature, for new features or requests labels Aug 7, 2024
@yliang412 yliang412 marked this pull request as ready for review August 7, 2024 01:12
@yliang412 yliang412 requested a review from a team as a code owner August 7, 2024 01:12
@yliang412 yliang412 requested a review from jcsp August 7, 2024 01:12
@yliang412 yliang412 changed the title [WIP] feat(pageserver): add direct io pageserver config feat(pageserver): add direct io pageserver config Aug 7, 2024
Copy link
Member

@skyzh skyzh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you may also change the default neon_local config (in some toml string in the codebase) to enable it by default for neon local users.

@yliang412
Copy link
Contributor Author

you may also change the default neon_local config (in some toml string in the codebase) to enable it by default for neon local users.

I will change the config once I'm certain with my implementation in the read path.

@yliang412 yliang412 enabled auto-merge (squash) August 7, 2024 18:55
@yliang412 yliang412 disabled auto-merge August 7, 2024 18:56
@yliang412 yliang412 enabled auto-merge (squash) August 7, 2024 19:04
@yliang412 yliang412 merged commit 542385e into main Aug 7, 2024
63 checks passed
@yliang412 yliang412 deleted the yuchen/direct-io-ps-config branch August 7, 2024 20:04
jcsp pushed a commit that referenced this pull request Aug 12, 2024
Part of #8130, [RFC: Direct IO For Pageserver](https://github.com/neondatabase/neon/blob/problame/direct-io-rfc/docs/rfcs/034-direct-io-for-pageserver.md)

## Description

Add pageserver config for evaluating/enabling direct I/O. 

- Disabled: current default, uses buffered io as is.
- Evaluate: still uses buffered io, but could do alignment checking and
perf simulation (pad latency by direct io RW to a fake file).
- Enabled: uses direct io, behavior on alignment error is configurable.


Signed-off-by: Yuchen Liang <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
c/storage/pageserver Component: storage: pageserver t/feature Issue type: feature, for new features or requests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants