You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
When requesting data from the beacon chain using caplin RPC, currently there are two possibilities: the finalized epoch and the previous finalized epoch (and anything newer) is always reachable, meaning that the slots belonging to them can be used in RPC requests. Anything older returns 404, by default.
The second possibility, is to enable archival for caplin, which downloads all historical blocks and then processes them from the beginning towards the current time, and this processing (reported as "State processing progress") takes ~4-5 days on 3.0.0-alpha3 with modern hardware and full nvme setup with 64GB ram. During this processing, only the less interesting part of the chain (the beginning) is reachable, and later blocks become available last. The very recent history (as discussed in "possibility one" above) is always reachable, fortunately. Once, this synchronization succeeds, all part of the beaconchain is queriable with good latency.
For any kind of remote validator monitoring (typical questions like "how much did a validator lose in the last 3 days due to spurious internet outages"), the only setting that is usable currently is the full archive, but in practice, a partly full archive, which e.g. removes blocks after 2 months (or configurable), would be very useful. This feature would even be useful, if no new synchronization logic were to be implemented, simply the archival would "start" when you set up the non-archive caplin, and then slowly fill up your 2 months (in realtime), and from then on the 2 months are always available.
Alternatively, if some kind of torrent technology is implemented, instead of the very slow "State processing progress", then maybe this whole discussion is moot, as the datasize itself is <200G for the full archive, so the only problem is the slow generation of it. Compared to the size of the execution chain, one can definitely accept if the only feature for archival access is a 200G download and storage, if the initialization of this fast (similarly to the execution chain in 3.0.0-alpha3).
Implementation
I unfortunately don't have enough knowledge of the codebase to know if this is a quick change, but I'm willing to poke around if somebody has ideas, I can definitely do small modifications if directed and I can test+report back.
Also, maybe there is some fundamental reason why what I'm asking is not possible, I just very naively figured that if a data can be securely accessed now (with confidence that it has been verified and it's not a lie), then it's possible to archive it for 2 months. In the complicated world of PoS and crypto in general, there might be some issue with my oversimplified reasoning.
The text was updated successfully, but these errors were encountered:
Rationale
When requesting data from the beacon chain using caplin RPC, currently there are two possibilities: the finalized epoch and the previous finalized epoch (and anything newer) is always reachable, meaning that the slots belonging to them can be used in RPC requests. Anything older returns 404, by default.
The second possibility, is to enable archival for caplin, which downloads all historical blocks and then processes them from the beginning towards the current time, and this processing (reported as "State processing progress") takes ~4-5 days on 3.0.0-alpha3 with modern hardware and full nvme setup with 64GB ram. During this processing, only the less interesting part of the chain (the beginning) is reachable, and later blocks become available last. The very recent history (as discussed in "possibility one" above) is always reachable, fortunately. Once, this synchronization succeeds, all part of the beaconchain is queriable with good latency.
For any kind of remote validator monitoring (typical questions like "how much did a validator lose in the last 3 days due to spurious internet outages"), the only setting that is usable currently is the full archive, but in practice, a partly full archive, which e.g. removes blocks after 2 months (or configurable), would be very useful. This feature would even be useful, if no new synchronization logic were to be implemented, simply the archival would "start" when you set up the non-archive caplin, and then slowly fill up your 2 months (in realtime), and from then on the 2 months are always available.
Alternatively, if some kind of torrent technology is implemented, instead of the very slow "State processing progress", then maybe this whole discussion is moot, as the datasize itself is <200G for the full archive, so the only problem is the slow generation of it. Compared to the size of the execution chain, one can definitely accept if the only feature for archival access is a 200G download and storage, if the initialization of this fast (similarly to the execution chain in 3.0.0-alpha3).
Implementation
I unfortunately don't have enough knowledge of the codebase to know if this is a quick change, but I'm willing to poke around if somebody has ideas, I can definitely do small modifications if directed and I can test+report back.
Also, maybe there is some fundamental reason why what I'm asking is not possible, I just very naively figured that if a data can be securely accessed now (with confidence that it has been verified and it's not a lie), then it's possible to archive it for 2 months. In the complicated world of PoS and crypto in general, there might be some issue with my oversimplified reasoning.
The text was updated successfully, but these errors were encountered: