Skip to content

Commit

Permalink
Add timeline offload mechanism (#8907)
Browse files Browse the repository at this point in the history
Implements an initial mechanism for offloading of archived timelines.

Offloading is implemented as specified in the RFC.

For now, there is no persistence, so a restart of the pageserver will
retrigger downloads until the timeline is offloaded again.

We trigger offloading in the compaction loop because we need the signal
for whether compaction is done and everything has been uploaded or not.

Part of #8088
  • Loading branch information
arpad-m authored Oct 8, 2024
1 parent 5bd8e23 commit e8ae376
Show file tree
Hide file tree
Showing 6 changed files with 434 additions and 93 deletions.
4 changes: 3 additions & 1 deletion pageserver/src/http/routes.rs
Original file line number Diff line number Diff line change
Expand Up @@ -703,6 +703,8 @@ async fn timeline_archival_config_handler(
let tenant_shard_id: TenantShardId = parse_request_param(&request, "tenant_shard_id")?;
let timeline_id: TimelineId = parse_request_param(&request, "timeline_id")?;

let ctx = RequestContext::new(TaskKind::MgmtRequest, DownloadBehavior::Warn);

let request_data: TimelineArchivalConfigRequest = json_request(&mut request).await?;
check_permission(&request, Some(tenant_shard_id.tenant_id))?;
let state = get_state(&request);
Expand All @@ -713,7 +715,7 @@ async fn timeline_archival_config_handler(
.get_attached_tenant_shard(tenant_shard_id)?;

tenant
.apply_timeline_archival_config(timeline_id, request_data.state)
.apply_timeline_archival_config(timeline_id, request_data.state, ctx)
.await?;
Ok::<_, ApiError>(())
}
Expand Down
Loading

1 comment on commit e8ae376

@github-actions
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5085 tests run: 4878 passed, 0 failed, 207 skipped (full report)


Flaky tests (4)

Postgres 17

Postgres 16

Postgres 15

Postgres 14

Code coverage* (full report)

  • functions: 31.3% (7510 of 23958 functions)
  • lines: 49.5% (60299 of 121883 lines)

* collected from Rust tests only


The comment gets automatically updated with the latest test results
e8ae376 at 2024-10-09T00:29:41.844Z :recycle:

Please sign in to comment.