Skip to content

Commit

Permalink
fix(testing): Use 1 MB shared_buffers even with LFC (#9969)
Browse files Browse the repository at this point in the history
## Problem

After enabling LFC in tests and lowering `shared_buffers` we started
having more problems with `test_pg_regress`.

## Summary of changes

Set `shared_buffers` to 1MB to both exercise getPage requests/LFC, and
still have enough room for Postgres to operate. Everything smaller might
be not enough for Postgres under load, and can cause errors like 'no
unpinned buffers available'.

See Konstantin's comment [1] as well.

Fixes #9956

[1]:
#9956 (comment)
  • Loading branch information
ololobus authored Dec 2, 2024
1 parent d8ebd33 commit 2e9207f
Show file tree
Hide file tree
Showing 2 changed files with 6 additions and 4 deletions.
4 changes: 4 additions & 0 deletions control_plane/src/endpoint.rs
Original file line number Diff line number Diff line change
Expand Up @@ -310,6 +310,10 @@ impl Endpoint {
conf.append("wal_log_hints", "off");
conf.append("max_replication_slots", "10");
conf.append("hot_standby", "on");
// Set to 1MB to both exercise getPage requests/LFC, and still have enough room for
// Postgres to operate. Everything smaller might be not enough for Postgres under load,
// and can cause errors like 'no unpinned buffers available', see
// <https://github.com/neondatabase/neon/issues/9956>
conf.append("shared_buffers", "1MB");
conf.append("fsync", "off");
conf.append("max_connections", "100");
Expand Down
6 changes: 2 additions & 4 deletions test_runner/fixtures/neon_fixtures.py
Original file line number Diff line number Diff line change
Expand Up @@ -3801,13 +3801,11 @@ def create(
assert size_to_bytes(size) >= size_to_bytes(
"1MB"
), "LFC size cannot be set less than 1MB"
# shared_buffers = 512kB to make postgres use LFC intensively
# neon.max_file_cache_size and neon.file_cache size limit are
# set to 1MB because small LFC is better for testing (helps to find more problems)
lfc_path_escaped = str(lfc_path).replace("'", "''")
config_lines = [
"shared_buffers = 512kB",
f"neon.file_cache_path = '{lfc_path_escaped}'",
# neon.max_file_cache_size and neon.file_cache size limits are
# set to 1MB because small LFC is better for testing (helps to find more problems)
"neon.max_file_cache_size = 1MB",
"neon.file_cache_size_limit = 1MB",
] + config_lines
Expand Down

1 comment on commit 2e9207f

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

7144 tests run: 6826 passed, 0 failed, 318 skipped (full report)


Flaky tests (3)

Postgres 17

Postgres 16

Code coverage* (full report)

  • functions: 30.7% (8262 of 26881 functions)
  • lines: 47.8% (65200 of 136467 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
2e9207f at 2024-12-02T20:41:34.886Z :recycle:

Please sign in to comment.