feat(): index storage scaffolding & rocksdb implementation #47
Conversation
|
Decided to ciborium. Discussion here: https://eig3r.slack.com/archives/C070RN7BZPC/p1716879168517689 |
cernicc
added
ready for review
cernicc
added
ready for review
cernicc
added
ready for review
cernicc
added
ready for review
| @@ -0,0 +1,3 @@ | |||
| fn main() { | |||
There was a problem hiding this comment.
Can we please add the crate level doc with text from the readme?
The piecestore module is a simple encapsulation of two data stores, one for
PieceInfoand
another forCidInfo.
| fn cf_handle(&self, cf_name: &str) -> &ColumnFamily { | ||
| self.database | ||
| .cf_handle(cf_name) | ||
| .expect("column family should be present") |
There was a problem hiding this comment.
Potentially this functionality is abstract enough to treat it as a library, because of which IMO library should not panic, but propagate an error so the client/caller app can decide what to do with it...
| fn add_deal_for_piece( | ||
| &self, | ||
| piece_cid: &Cid, | ||
| deal_info: DealInfo, | ||
| ) -> Result<(), PieceStoreError> { | ||
| // Check if the piece exists | ||
| let mut piece_info = self | ||
| .get_value_at_key(piece_cid.to_bytes(), PIECES_CF)? | ||
| .unwrap_or_else(|| PieceInfo { | ||
| piece_cid: *piece_cid, | ||
| deals: Vec::new(), | ||
| }); | ||
|
|
||
| // Check if deal already added for this piece | ||
| if piece_info.deals.iter().any(|d| *d == deal_info) { | ||
| return Err(PieceStoreError::DealExists); | ||
| } | ||
|
|
||
| // Save the new deal | ||
| piece_info.deals.push(deal_info); | ||
| self.put_value_at_key(piece_cid.to_bytes(), &piece_info, PIECES_CF) | ||
| } |
There was a problem hiding this comment.
I have this feeling that we have super abstract functionality which might be separated to a separate crate mixed with business logic related context like here, where we operates with deals on the same layer where we are defining functionality like put_value_at_key etc.
| // verify that getting a piece with a non-existent CID return None | ||
| let info = store.get_piece_info(&piece_cid2).unwrap(); | ||
| assert!(info.is_none(), "expected None, got {:?}", info); |
There was a problem hiding this comment.
I'd suggest to separate this to a different test case.
Description
The PR adds abstraction layer over the specific database implementation used for storage indexing. Rocksdb implementation is also added.
In the rocksdb impl we are using two different column names descriptors. The
piecesnamespace is used for storing infos of pieces. Thecid_infosis used for storing cid infos.If any additional info would help you as a reviewer please let me know.
Important points for reviewers
Notes