Error in readFully method from client-utils: Rejected promise with missing vStorage data

[Jorge-Lopes]

Describe the bug

After the changes introduced to the client-utils package in PR #10566, the readFully method from the Agoric client-utils package fails when attempting to read a vstorage node. Instead of returning the expected value, it produces a rejected promise with the following error message:

  Rejected promise returned by test. Reason:

  Error {
    code: 38,
    codespace: 'sdk',
    message: 'error code 38 reading data of published.committees.Economic_Committee.latestQuestion: no data for vstorage path: not found',
  }

To Reproduce

Steps to reproduce the behavior:

1. Go to Draft PR Jorge/10574 read fully #10575
2. Go to the CI action test-docker-build
3. See error message

Expected behavior

The readFully method should correctly return the values stored in a specific node, recorded after a given block height.

Platform Environment

OS: macOS Sequoia v15.0.1
Node.js: v20.9.0
Agoric-SDK: agoric-upgrade-16av-1172-g3de052bbe8

[Jorge-Lopes]

cc: @Chris-Hibbert

[turadg]

After the changes introduced to the client-utils package in PR #10566, the readFully method from the Agoric client-utils package fails when attempting to read a vstorage node

I can't tell whether you're attributing the cause of this problem to #10566. Are you saying that it worked before? (i.e. Are you saying "Even after the changes…" or "Because of the changes…")

[Jorge-Lopes]

I can't tell whether you're attributing the cause of this problem to #10566. Are you saying that it worked before? (i.e. Are you saying "Even after the changes…" or "Because of the changes…")

I am saying "Even after the changes…".
Sorry for the lack of clarity in the description.

[dckc]

readFully is a sort of awkward API - it lumps the results of a sequence of queries into one array, and changes in the way cosmos-sdk reports the end of the available data have made it somewhat unstable (#9024 etc.).

I hope we can migrate toward something like readHistory that provides an async iterator. If the tail end of the iterator is flaky, at least the iterations before that are useful.

[Jorge-Lopes]

Following @dckc's suggestion, I replaced the use of readFully with readHistory. This change successfully resolved the issue described earlier, enabling the retrieval of the expected history from a vstorage node.

However, during further testing, I noticed inconsistent behavior in the context of the a3p-integration acceptance tests. Specifically, when readHistory is used after the execution of both state sync tests, it fails with the following errors:

• if after state-sync-snapshots-test.sh

message: 'cannot read data of published.committees.Economic_Committee.latestQuestion: Internal Server Error',

• if after genesis-test.sh

message: 'cannot read data of published.committees.Economic_Committee.latestQuestion: fetch failed',

I am unsure whether this behavior is expected or documented.
To facilitate observation and analysis, I added a new test case in z:acceptance/vstorage.test.js, which runs both before and after the state sync tests.
The full logs are available in the CI action report at the same draft PR #10575.

[turadg]

when readHistory is used after the execution of both state sync tests, it fails with the following errors

State-sync prunes history so it's expected that not all the history would be available after state-sync. The "Internal Server Error" is a surprise, but out of scope.

For your purposes I think you'll just need do any tests depending on history before the state-sync test.

[Jorge-Lopes]

For your purposes I think you'll just need do any tests depending on history before the state-sync test.

Thank you for the feedback @turadg
I have followed this approach, I am not blocked by this issue.
My goal was simply to bring this behavior to your attention in case it might be relevant.

[mhofman]

I don't remember what the genesis test does, but it might similarly affect availability of historical data.