Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Return error instead of panicking if rewriting fails #343

Merged
merged 25 commits into from
Dec 7, 2023
Merged

Return error instead of panicking if rewriting fails #343

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
25 commits
Select commit Hold shift + click to select a range
3a75b78
Return error instead of panicing if rewriting fails
v01dstar Nov 2, 2023
77f8beb
Update rust version
v01dstar Nov 3, 2023
34db5d0
Update rust version in github workflow
v01dstar Nov 3, 2023
452d57e
Update src/file_pipe_log/pipe.rs
v01dstar Nov 7, 2023
a20cd43
Update src/file_pipe_log/pipe.rs
v01dstar Nov 7, 2023
43b25ca
Address comments, fix test cases
v01dstar Nov 7, 2023
6fcb077
Fix format error
v01dstar Nov 7, 2023
8cc474d
Move panic inside
v01dstar Nov 8, 2023
1fd5416
Fix clippy
v01dstar Nov 8, 2023
c606f51
Propagate error if writing header fails
v01dstar Nov 9, 2023
61fbdb6
Adjust write header fail expectation, from panic to error
v01dstar Nov 9, 2023
862fe0b
Panic if write header fails since we do not truncate
v01dstar Nov 9, 2023
0d2924b
Failure other than sync should be returned
v01dstar Nov 9, 2023
a59609a
Address comments
v01dstar Nov 15, 2023
0554cd1
Fix test failures
v01dstar Nov 15, 2023
2c81285
Change test exepectations
v01dstar Nov 15, 2023
bd2c3b4
Address comments
v01dstar Nov 25, 2023
005418f
Fix format
v01dstar Nov 25, 2023
2c8d59a
Revert sync() signature
v01dstar Nov 28, 2023
395d530
Add more details to rotate test
v01dstar Nov 28, 2023
cee2d8f
Fix style
v01dstar Nov 28, 2023
8c2eb45
Address comments
v01dstar Dec 4, 2023
445fd1e
Address comments
v01dstar Dec 6, 2023
3106c04
Fix clippy
v01dstar Dec 6, 2023
bb27b29
Trigger Github actions
v01dstar Dec 6, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 1 addition & 3 deletions Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name = "raft-engine"
version = "0.4.1"
authors = ["The TiKV Project Developers"]
edition = "2018"
rust-version = "1.66.0"
rust-version = "1.67.1"
description = "A persistent storage engine for Multi-Raft logs"
readme = "README.md"
repository = "https://github.com/tikv/raft-engine"
Expand Down Expand Up @@ -95,8 +95,6 @@ nightly_group = ["nightly", "swap"]
raft-proto = { git = "https://github.com/tikv/raft-rs", branch = "master" }
protobuf = { git = "https://github.com/pingcap/rust-protobuf", branch = "v2.8" }
protobuf-codegen = { git = "https://github.com/pingcap/rust-protobuf", branch = "v2.8" }
# TODO: Use official grpc-rs once https://github.com/tikv/grpc-rs/pull/622 is merged.
grpcio = { git = "https://github.com/tabokie/grpc-rs", branch = "v0.10.x-win" }

[workspace]
members = ["stress", "ctl"]
2 changes: 1 addition & 1 deletion src/file_pipe_log/log_file.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ impl<F: FileSystem> LogFileWriter<F> {
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a comment to this struct stating it should be fail-safe, i.e. user can still use the writer without breaking data consistency if any operation has failed.


fn write_header(&mut self, format: LogFileFormat) -> IoResult<()> {
self.writer.seek(SeekFrom::Start(0))?;
self.writer.rewind()?;
self.written = 0;
let mut buf = Vec::with_capacity(LogFileFormat::encoded_len(format.version));
format.encode(&mut buf).unwrap();
Expand Down
14 changes: 2 additions & 12 deletions src/file_pipe_log/pipe.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,12 +321,7 @@ impl<F: FileSystem> SinglePipe<F> {
fail_point!("file_pipe_log::append");
let mut writable_file = self.writable_file.lock();
if writable_file.writer.offset() >= self.target_file_size {
if let Err(e) = self.rotate_imp(&mut writable_file) {
panic!(
"error when rotate [{:?}:{}]: {e}",
self.queue, writable_file.seq,
);
}
self.rotate_imp(&mut writable_file)?;
v01dstar marked this conversation as resolved.
Show resolved Hide resolved
}

let seq = writable_file.seq;
Expand Down Expand Up @@ -372,12 +367,7 @@ impl<F: FileSystem> SinglePipe<F> {
// - [3] Both main-dir and spill-dir have several recycled logs.
// But as `bytes.len()` is always smaller than `target_file_size` in common
// cases, this issue will be ignored temprorarily.
if let Err(e) = self.rotate_imp(&mut writable_file) {
panic!(
"error when rotate [{:?}:{}]: {e}",
self.queue, writable_file.seq
);
}
self.rotate_imp(&mut writable_file)?;
v01dstar marked this conversation as resolved.
Show resolved Hide resolved
// If there still exists free space for this record, rotate the file
// and return a special TryAgain Err (for retry) to the caller.
return Err(Error::TryAgain(format!(
Expand Down
Loading