Integration testing examples and documentation

[WillSquire]

I've been reading through the examples and docs but I can't find material on integration testing? Found a good bit on unit testing here though. Is there material on this?

[hgzimmerman]

I don't think there is any significant material on testing within warp, although I have a few approaches to both unit testing and integration testing utilizing warp that I'll share:

To enable integration testing, you want to structure your server to initialize a State singleton that has a reference passed around to every filter that relies on configuration or needs to talk to a database (which should be nearly every endpoint). This State struct should have a database connection pool (or just a single connection), and an associated function that returns a filter that returns a connection.
This looks like:

let endpoint = warp::path("api/path")
    .and(state.db())
    .and_then(|conn: PooledConnection| {
        // implementation
    });

Then you need to find a way to work with a defined database state in every test.
I opted to delete and recreate the database and re-run the migrations to rebuild the schema for each test, then run a function to populate it with expected data that can be tested against.
Some concurrent locking mechanism should be needed to prevent tests from invalidating each other by making sure that only one test runs at once. You can choose to connect to a test database instead of your usual development or release one when you create the pool.

---

While I haven't gotten around to doing this yet, these are my plans for doing unit testing.

To perform unit testing, you want to remove all external services as best you can. You ideally want to avoid using warp's testing infrastructure, and just test your business logic. So when constructing routes, I use a pattern of doing business logic in a function that is passed to a map filter, and then following the map with an and_then that either converts it to JSON, or rejects the request if the business logic function returned an Err.
This looks like:

let get_users_in_bucket = path!(Uuid / "users")
        .and(warp::get2())
        .and(state.db())
        .map(get_users_in_bucket_handler) // takes a Uuid, and a database conneciton as arguments, returns a Result
        .and_then(json_or_reject);

So now you have your business logic in a handler function that can be tested independently of warp. Now you have to remove your dependency on the database to make it a unit test. Create a trait called Repository that contains all the function definitions that you use to access the database, and implement it on your Connection struct. Then create a struct called DatabaseMock and implement the Repository trait on it. Now, by using trait objects or impl Repository in your business logic handlers, you can use your business logic with either the database connection, or your mock - which you use for unit testing.

This facilitates unit testing, but fills me with existential dread, because you now have the problem of ensuring that the mock object behaves the same as your database. It should be much faster to test with than the integration tests because they run in a threaded manner and don't need to reset the database on every new test, but the effort of creating the mock object and ensuring its parity with the database implementation is more effort than I think faster tests are worth.

[WillSquire]

Thanks for the detailed post @hgzimmerman. I'm kinda more interested in how a server is 'spun up' in a test though. I see the database side as outside of scope, but it would be good information nonetheless - P.S I've got an open source project I'm tinkering with at the moment that has a similar setup here if that's a helpful reference for others. Separation in lib.rs and main.rs is also kinda noteworthy.

Anyway the bit I'm getting my head around is that run(([127, 0, 0, 1], 8000)) puts a thread in an infinite loop, so doing an integration test from the app entrypoint causes it to block an never return. I can think of a way around it (i.e. extracting all routing out before this point and testing) but ideally I'd like to spin it up, send a request and stop the server in an integration test just so it can be as close as possible to the real thing. I already compromised with separating the args parsing to main.rs and the rest to lib.rs.

[seanmonstar]

As you've noticed, there's an introduction to testing at the docs of warp::test. Besides that, all of warp's own tests are built using warp::test, in case you'd like more examples.

---

@hgzimmerman

This facilitates unit testing, but fills me with existential dread, because you now have the problem of ensuring that the mock object behaves the same as your database.

You could make the default tests use the mock object, and have CI run the tests with a real DB.

[..], but the effort of creating the mock object and ensuring its parity with the database implementation is more effort than I think faster tests are worth.

To each their own, but in my experience, having fast tests is important to allow a developer to run them more often, being more confident and productive.

[hgzimmerman]

@WillSquire In that case, I think bind_with_graceful_shutdown might serve your needs in that it allows you to shut down your server externally.

---

@seanmonstar I feel dumb for having phrased it that way. For professional development with collaborators, unit tests are invaluable, whereas I think my common use case involving non-critical throwaway services is better served by relying on integration tests and Rust's types to keep me safe. And thank you for the suggestion, it never occurred to me to put the mock/db choice for testing behind a feature flag.

[seanmonstar]

Is the question answered? Sorry, I got lost in the comments. XD

[WillSquire]

@hgzimmerman thanks I'll take a look into this :) but I was hoping there was a way to use the same method as I'd use in production (run())?

@seanmonstar No problem :P it's not quite there for me yet. I guess I'd expect documentation on both unit testing and integration testing like in the rust docs. Rust seems to have a convention of doing integration tests from the main app entrypoint, which isn't bad, although I understand an integration test in general can be more localised to specific regions on an app. As an example Rocket's got a friendly nod at this in the testing section because it explains how to structure the rocket().launch() separately (the main entry point).

[hgzimmerman]

The Rust docs regarding integration tests you are pointing to pertain to integration tests for libraries, not necessarily for web services, so the approaches are different. While both attempt to cover the publicly facing api of a project, library integration tests exist to ensure that the library works with other libraries or in the context of how it is supposed to be used by consumers of the library. Web service integration tests aim to ensure that the service and all its known dependencies (database, external api calls) work together, and that requests are handled as expected.

Warp's existing test module enables both integration and unit testing, and the distinction partially hinges on if you are mocking your external entities or using them directly. The use of warp's testing module isn't practically different than the provided Rocket example, in that warp::serve()ing a filter is the same as calling .launch() on a Rocket service, and that passing a filter to a test request chain is approximately the same as instantiating a Client and running requests against it.

So instead of the run() function you export from lib.rs, you might want to export another function that takes your config information and returns your filter (everything currently inside your warp::serve in run()), that you then use it like warp::serve(your_filter_here).run(...) in your main.rs. That way, you can test the same lib function that you use in main() using warp's existing test framework.

[WillSquire]

Thanks @hgzimmerman. I think that any kind of test should be as close as possible to how actually going to be used, but this might be a matter of opinion. Appreciate there can be focus on certain areas for integration tests (such as communicating with databases) and I'm following this up on diesel as a problem but as something that's outside the scope of this package.

The reason for this ticket and what could hopefully be a helpful improvement is the documentation around integration testing for the main app entrypoint that handles http requests, i.e. handled by run(). The closer requests fired in tests can be handled in the same manner as they'd be handled in production, the better, but I understand there might be some compromises. So I've tried pulling out the following:

warp::serve(
  // Snip...
)
.run((config.address, 8000));

and returning:

warp::get2()
  .and(warp::path::end().and(juniper_warp::graphiql_filter("/graphql")))
  .or(warp::path("graphql").and(graphql(context(db, hasher, tokeniser))))
  .with(log)

but with(log) returns a type that is internal. I'm currently looking at:

impl Filter<Extract = (warp::log::internal::Logged,), Error = warp::Rejection>

as the return type, but is there a different signature I should be using to this?

[hgzimmerman]

Yeah, without looking too closely, I'm pretty confident that the following should work:

-> impl Filter<Extract = (impl warp::Reply,), Error = warp::Rejection>

or failing that, stick a .boxed() at the end of your filter chain and have a return signature of:

-> BoxedFilter<(impl warp::Reply,)>

Check out the returning example for a better reference point.

[WillSquire]

Brilliant thanks @hgzimmerman that worked great. Wanted to get an example up here to help others, so apologies for going a bit quiet. I haven't got a minimal example yet but hopefully with have soon (I know this sort of thing helps me). Thanks again

[WillSquire]

In case it's useful to others, here's the separation in lib.rs and the integration tests. These aren't quite finished yet as the next thing to tackle is database seeding... but that's a whole separate thing. Thanks again :)

[seanmonstar]

Additionally, the todos.rs example was recently updated to show how to build and app and include tests.