Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(sns): Avoid producing overly large errors upon UpgradeSnsControlledCanister proposal invalidation #2877

Open
wants to merge 2 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
124 changes: 96 additions & 28 deletions rs/sns/governance/src/lib.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
use crate::pb::v1::Subaccount as SubaccountProto;
use std::{convert::TryInto, fmt::Debug};
use std::convert::TryInto;

mod cached_upgrade_steps;
pub mod canister_control;
Expand All @@ -20,29 +20,37 @@ trait Len {
fn len(&self) -> usize;
}

/// Maximum size, in bytes, of a scalar field (e.g., of type `String` or numeric types) that.
/// Scalar values greater than this will be truncated during error reporting.
pub const MAX_SCALAR_FIELD_LEN_BYTES: usize = 50_000;

/// Warning: the len method on str and String is in bytes, not characters. If
/// you want to constrain the number of characters, look at
/// validate_chars_count.
fn validate_len<V>(field_name: &str, field_value: &V, min: usize, max: usize) -> Result<(), String>
where
V: Len + Debug,
V: Len + ToString,
{
let len = field_value.len();

if len < min {
return field_err(
field_name,
field_value,
&format!("too short (min = {} vs. observed = {})", min, len),
);
let defect = &format!("too short (min = {} vs. observed = {})", min, len);

let bounded_field_value = field_value.to_string();

return field_err(field_name, bounded_field_value, defect);
}

if len > max {
return field_err(
field_name,
field_value,
&format!("too long (min = {} vs. observed = {})", min, len),
);
let defect = &format!("too long (max = {} vs. observed = {})", max, len);

let bounded_field_value = field_value
.to_string()
.chars()
.take(max)
.collect::<String>();

return field_err(field_name, bounded_field_value, defect);
}

Ok(())
Expand Down Expand Up @@ -94,19 +102,16 @@ fn validate_chars_count(
let len = field_value.chars().count();

if len < min {
return field_err(
field_name,
field_value,
&format!("too short (min = {} vs. observed = {})", min, len),
);
let defect = &format!("too short (min = {} vs. observed = {})", min, len);

return field_err(field_name, field_value.to_string(), defect);
}

if len > max {
return field_err(
field_name,
field_value,
&format!("too long (max = {} vs. observed = {})", max, len),
);
let defect = &format!("too long (max = {} vs. observed = {})", max, len);
let bounded_field_value = field_value.chars().take(max).collect::<String>();

return field_err(field_name, bounded_field_value, defect);
}

Ok(())
Expand All @@ -121,12 +126,35 @@ fn validate_required_field<'a, Inner>(
.ok_or_else(|| format!("The {} field must be populated.", field_name))
}

/// Return an Err whose inner value describes (in detail) what is wrong with a
/// field value, and where within some (Protocol Buffers message) struct.
fn field_err(field_name: &str, field_value: impl Debug, defect: &str) -> Result<(), String> {
/// Return an Err whose inner value describes (in detail) what is wrong with a field value (should
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
/// Return an Err whose inner value describes (in detail) what is wrong with a field value (should
/// Return an Err whose inner value describes (in detail) what is wrong with a field value (e.g. should

?

/// be bounded), and where within some (Protocol Buffers message) struct.
///
/// Only up to the first `MAX_SCALAR_FIELD_LEN_BYTES` bytes will be taken from `field_value`.
fn field_err(field_name: &str, field_value: String, defect: &str) -> Result<(), String> {
let mut bounded_field_value = String::new();
// Searching for the longest byte prefix that is a valid string is expensive in the worst case.
// This is because unicode characters are unbounded. Instead, concatenate characters one-by-one,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unicode characters are unbounded

Whaa??

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, this comment is misleading. Actually there are ligatures of unbounded size, but they would be represented as multiple chars in Rust's utf-8 encoding. I'll adjust the comment.

// until we either run out of characters or exceed the limit (in which case we pop the last
// character to still comply with the limits). Note that a `char` is always 4 bytes, but
// pushing it to a string does not always increase a string's byte size by 4 bytes.
// For example:
// ```
// println!("bytes = {}", std::mem::size_of_val(&'\u{200D}')); // bytes = 4
// println!("bytes = {}", std::mem::size_of_val("\u{200D}")); // bytes = 3
// ```
for c in field_value.chars() {
bounded_field_value.push(c);
if bounded_field_value.len() > MAX_SCALAR_FIELD_LEN_BYTES {
bounded_field_value.pop();
break;
}
}
Comment on lines +134 to +151
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Uh. You are just trying to limit the length of the defect description, right? How about this: https://sourcegraph.com/github.com/dfinity/ic/-/blob/rs/nervous_system/string/src/lib.rs?L25:8-25:23

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That function doesn't seem safe for arbitrary unicode strings. For example, it would panic in the following case:

clamp_string_len("\u{200D}\u{200D}\u{200D}", 2)

https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=16ace03d1cfbfb292b28b2e96b9cbe1f

Err(format!(
"The value in field {} is {}: {:?}",
field_name, defect, field_value
"The first {} characters of the value in field `{}` are {}: `{}`",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The new wording doesn't make sense to me. It should say what bad property the field value has. E.g. "The value in foo is not prime: 42".

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'll improve the wording.

bounded_field_value.chars().count(),
field_name,
defect,
bounded_field_value
))
}

Expand Down Expand Up @@ -201,6 +229,14 @@ mod tests {
assert_is_ok(validate(&"abcde"));

assert_is_err(validate(&"abcd\u{1F389}"));
assert_eq!(
validate(&"abcdefg"),
Err(
"The first 5 characters of the value in field `field_name` are too long \
(max = 5 vs. observed = 7): `abcde`"
.to_string()
),
);
}

#[test]
Expand Down Expand Up @@ -261,7 +297,7 @@ mod tests {

#[test]
fn test_field_err() {
let result = field_err("my_field", 41, "not the meaning of life");
let result = field_err("my_field", 41.to_string(), "not the meaning of life");
match result {
Ok(()) => panic!("field_err is supposed to always return an Err."),
Err(err) => {
Expand All @@ -271,4 +307,36 @@ mod tests {
}
}
}

#[test]
fn test_giant_field_err() {
let expected_upper_bound = 12_500;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let expected_upper_bound = 12_500;
let barely_not_too_large_len = 12_500;


let run_test_for_value_of_size = |value_size| {
let input_value: String = (0..value_size).map(|_| '🤝').collect();
// Sanity check: We construct a string in which each character is encoded as 4 bytes.
assert_eq!(input_value.len(), 4 * input_value.chars().count());
let observer_err = field_err("foo", input_value.clone(), "bar").unwrap_err();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
let observer_err = field_err("foo", input_value.clone(), "bar").unwrap_err();
let observer_err = field_err("field_name", input_value.clone(), "ugly").unwrap_err();

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not sure I see the point to have these dummy values over others?

(input_value, observer_err)
};

// Scenario A: maximum size that still fits.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Scenario A: maximum size that still fits.
// Value is (barely) not too large to be fully included in the defect description.

{
let (input_value, observer_err) = run_test_for_value_of_size(expected_upper_bound);
assert!(observer_err.contains(&input_value));
}

// Scenario B: minimum size that no longer fits.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// Scenario B: minimum size that no longer fits.
// Value is too large to be fully included in the defect description.

{
let (input_value, observer_err) = run_test_for_value_of_size(expected_upper_bound + 1);
assert!(
!observer_err.contains(&input_value),
"Expected ```{}``` not to contain ```{}```.",
observer_err,
input_value
);
// Only the last character was dropped.
assert!(observer_err.contains(&input_value[..(input_value.chars().count() - 1)]));
}
}
}
30 changes: 13 additions & 17 deletions rs/sns/governance/src/proposal.rs
Original file line number Diff line number Diff line change
Expand Up @@ -1068,30 +1068,26 @@ fn validate_and_render_upgrade_sns_controlled_canister(
const RAW_WASM_HEADER: [u8; 4] = [0, 0x61, 0x73, 0x6d];
// see https://ic-interface-spec.netlify.app/#canister-module-format
const GZIPPED_WASM_HEADER: [u8; 3] = [0x1f, 0x8b, 0x08];
// Minimum length of raw WASM is 8 bytes (4 magic bytes and 4 bytes encoding version).
// Minimum length of gzipped WASM is 10 bytes (2 magic bytes, 1 byte encoding compression method, and 7 additional gzip header bytes).
const MIN_WASM_LEN: usize = 8;
if let Err(err) = validate_len(
"new_canister_wasm",
new_canister_wasm,
MIN_WASM_LEN,
usize::MAX,
) {
defects.push(err);
} else if new_canister_wasm[..4] != RAW_WASM_HEADER[..]
&& new_canister_wasm[..3] != GZIPPED_WASM_HEADER[..]

if new_canister_wasm.len() < 4
|| new_canister_wasm[..4] != RAW_WASM_HEADER[..]
&& new_canister_wasm[..3] != GZIPPED_WASM_HEADER[..]
Comment on lines +1072 to +1074
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This seems orthogonal to the problem(s) mentioned in the PR title+description. Perhaps, mention that there are some additional small entrained fixes?

{
defects.push("new_canister_wasm lacks the magic value in its header.".into());
}

if new_canister_wasm.len()
+ canister_upgrade_arg
if new_canister_wasm.len().saturating_add(
canister_upgrade_arg
.as_ref()
.map(|arg| arg.len())
.unwrap_or_default()
>= MAX_INSTALL_CODE_WASM_AND_ARG_SIZE
.unwrap_or_default(),
) >= MAX_INSTALL_CODE_WASM_AND_ARG_SIZE
{
defects.push(format!("the maximum canister WASM and argument size for UpgradeSnsControlledCanister is {} bytes.", MAX_INSTALL_CODE_WASM_AND_ARG_SIZE));
defects.push(format!(
"the maximum canister WASM and argument size \
for UpgradeSnsControlledCanister is {} bytes.",
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

align. rustfmt is great sometimes. All kneel before the robots!

MAX_INSTALL_CODE_WASM_AND_ARG_SIZE
));
}

// Generate final report.
Expand Down