Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

integration tests should mockup actual API calls instead of internal model representation in csv #547

Open
nicolasochem opened this issue Jan 17, 2022 · 0 comments
Assignees
Labels
enhancement New feature or request

Comments

@nicolasochem
Copy link
Contributor

nicolasochem commented Jan 17, 2022

With #545, we extended the current model of testing rewards calculation to all providers - previously tzkt was doing it this way.

Then ithaca happened. All these tests are now stale. Let's take this opportunity to make them better.

Reward api tests did not mock up RPC calls directly: they mocked up the internal representation of the results of these calls in large custom json files. Following the tzkt model, these files are populated if they do not exist, by actually calling the RPCs. When they exist, they are not called again.

It would be better to mock the rpc calls at lower level: add an option to pytest to call the apis "for real" and record the response. Then save all the data returned from these api calls in source control. In normal mode, replay all these recorded api calls, do not call any external provider.

There are several tools in python that can do that. vcr.py looks promising.

This would apply to all three providers.

As a bonus, we will have a simple way to figure out whenever any of the upstream APIs has changed: just rerun the tests agains "real" api endpoints, and compare with what we have saved: should be identical. Otherwise, the upstream apis have changed and we need to adjust the code or let them know.

We should wait for ithaca to be in "steady state" (past cycle 374) and then;

  • find examples of bakers with interesting earnings: blocks stolen, payload producer != block producer, revelation rewards, double baking reward
  • for each of them, run all 3 providers, recording the rpc calls with vcr.py, commit them, and replay them in the CI
  • also commit the resulting csv file (unique between 3 providers, since all 3 providers must return the same results)

The ultimate goal would be to be able to run all tests in airplane mode, without calling any external API at all. But this may not work for the payment consumer, which needs an actual tezos node to simulate operations.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants