You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
With #545, we extended the current model of testing rewards calculation to all providers - previously tzkt was doing it this way.
Then ithaca happened. All these tests are now stale. Let's take this opportunity to make them better.
Reward api tests did not mock up RPC calls directly: they mocked up the internal representation of the results of these calls in large custom json files. Following the tzkt model, these files are populated if they do not exist, by actually calling the RPCs. When they exist, they are not called again.
It would be better to mock the rpc calls at lower level: add an option to pytest to call the apis "for real" and record the response. Then save all the data returned from these api calls in source control. In normal mode, replay all these recorded api calls, do not call any external provider.
There are several tools in python that can do that. vcr.py looks promising.
This would apply to all three providers.
As a bonus, we will have a simple way to figure out whenever any of the upstream APIs has changed: just rerun the tests agains "real" api endpoints, and compare with what we have saved: should be identical. Otherwise, the upstream apis have changed and we need to adjust the code or let them know.
We should wait for ithaca to be in "steady state" (past cycle 374) and then;
find examples of bakers with interesting earnings: blocks stolen, payload producer != block producer, revelation rewards, double baking reward
for each of them, run all 3 providers, recording the rpc calls with vcr.py, commit them, and replay them in the CI
also commit the resulting csv file (unique between 3 providers, since all 3 providers must return the same results)
The ultimate goal would be to be able to run all tests in airplane mode, without calling any external API at all. But this may not work for the payment consumer, which needs an actual tezos node to simulate operations.
The text was updated successfully, but these errors were encountered:
With #545, we extended the current model of testing rewards calculation to all providers - previously tzkt was doing it this way.
Then ithaca happened. All these tests are now stale. Let's take this opportunity to make them better.
Reward api tests did not mock up RPC calls directly: they mocked up the internal representation of the results of these calls in large custom json files. Following the tzkt model, these files are populated if they do not exist, by actually calling the RPCs. When they exist, they are not called again.
It would be better to mock the rpc calls at lower level: add an option to pytest to call the apis "for real" and record the response. Then save all the data returned from these api calls in source control. In normal mode, replay all these recorded api calls, do not call any external provider.
There are several tools in python that can do that. vcr.py looks promising.
This would apply to all three providers.
As a bonus, we will have a simple way to figure out whenever any of the upstream APIs has changed: just rerun the tests agains "real" api endpoints, and compare with what we have saved: should be identical. Otherwise, the upstream apis have changed and we need to adjust the code or let them know.
We should wait for ithaca to be in "steady state" (past cycle 374) and then;
The ultimate goal would be to be able to run all tests in airplane mode, without calling any external API at all. But this may not work for the payment consumer, which needs an actual tezos node to simulate operations.
The text was updated successfully, but these errors were encountered: