Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

How to generate arbitrary values of a given type that match a condition? #163

Closed
OrangeTux opened this issue Jan 5, 2024 · 4 comments
Closed

Comments

@OrangeTux
Copy link

Assume you want to generate an arbitrary f32 that is withing certain bounds. Or you want to generate an arbitrary String that matches a certain regex. Or you want to generate an arbitrary value of some other data structure that matches certain rules.

// Generate a f32 between 0 and 100 (including).
fn arbitrary_f32(u: &mut Unstructured) -> Result<f32> {
   let value = f32::arbitrary(u)?;

   if value >= 0 && value =< 100{
     return Result::Ok(value)
   }
   // What to do values not in range of 0..100?
   // Clamp them?
   // Return an Result::Err(..)?
}

// Generate an arbitrary String that matches some regex.
fn arbitrary_string(u: &mut Unstructured) -> Result<String> {
   let regex = ...;
   let value = String::arbitrary(u)?;
   
   lf regex.match(value) {
           return Result::Ok(value);
   }

   // What to do with values not matching the regex?
}

How to handle values that don't match the predicate?

One could return clamp the f32, as is done here.
And one could return an empty String when a String doesn't match the regex. Or some other default.

The drawback is that arbitrary_f32() is likely to return 0 or 100, most of the time. And arbitrary_string() is likely to return an empty string in the majority of the cases.

Or should these functions return some Result::Err for values that don't match the predicate? If so, what error should they return? Arbitrary::Error only has 3 variants, and none of them seem to fit.

Do you have a word of advice?

@OrangeTux OrangeTux changed the title How to generate arbitrary values of a given type that match a predicate? How to generate arbitrary values of a given type that match a condition? Jan 5, 2024
@fitzgen
Copy link
Member

fitzgen commented Jan 5, 2024

You've enumerated the options pretty well. IME with other fuzz generators it is generally better to avoid returning errors, but you can try both and fuzz for a while and see which results in better coverage.

@Manishearth
Copy link
Member

For arbitrary_f32() you can do some math to get your float: generate an arbitrary float, invert it if it's not in the range 0 to 1, and then multiply by 100

@OrangeTux
Copy link
Author

Thanks for the quick feedback!

@langston-barrett
Copy link

Perhaps worth a note that the general case of generating values that match predicates is an active area of research in property-based testing, see e.g., this paper.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants