Skip to content

Commit

Permalink
feat(monofs): [ongoing] implement NFS server adapter
Browse files Browse the repository at this point in the history
Adds NFS server functionality to monofs, allowing it to be mounted as a network filesystem. Key changes include:

- Implement NFSFileSystem trait for MonofsServer
- Add support for basic NFS operations (create, mkdir, symlink, etc.)
- Add path-to-fileid mapping system using SymbolTable for efficient lookups
- Add comprehensive test suite for NFS operations
- Update documentation with examples and diagrams
  • Loading branch information
appcypher committed Jan 13, 2025
1 parent e5b8ee9 commit 4f1add1
Show file tree
Hide file tree
Showing 15 changed files with 1,306 additions and 21 deletions.
2 changes: 1 addition & 1 deletion _README.md → .md
Original file line number Diff line number Diff line change
Expand Up @@ -95,7 +95,7 @@ This will install both the `monocore` command and its alias `mc`.

### Quick Start

TODO: CLI: demo video of using the CLI.
TODO: demo video of that ties the CLI (throwaway vm), SDK (AI agent), Web UI (app-like experience) and REST API (scaling active sandboxes) together. See https://github.com/browserbase/stagehand?tab=readme-ov-file

### SDK

Expand Down
30 changes: 20 additions & 10 deletions .todo.md
Original file line number Diff line number Diff line change
@@ -1,33 +1,43 @@
## monofs

- [ ] Implement tombstone for deletes.
- [x] Implement tombstone for deletes.

- [ ] Implement merging although it is a bit tricky.

- [ ] How do we handle "previous" field conflicts?
- [ ] Handling deletes with tombstones

- [ ] Fix file size issue.

## monocore
- [ ] We should be able to get the file size from a files metadata.
- [ ] This requires making changes to `IpldStore` and `Layout`.
- [ ] `IpldStore::get_cid_size(cid: &Cid) -> Result<u64, Error>`?
- [ ] `Layout::get_cid_size(cid: &Cid) -> Result<u64, Error>`?

- [ ] Children entities should inherit sync type from their parents when they get added.

- [ ] Change MonofsServer `fileid_map` and `path_map` to use LruCache instead of HashMap.

## monoutils-store

- [ ] Improve remove and copy performance.
- [ ] Implement a content-defined chunker that uses Gear Hashing.

- [ ] Sibling entries should be processed in parallel.
- [ ] Do the same for PermissionsGuard Drop implementation.
- [ ] QuickCDC?
- [ ] Make it the default chunker for stores.

- [ ] Implement `BalancedDagLayout` and make it the default layout for stores.

## monocore

- [x] Fix copy and remove permission issues on Linux.

- [ ] Use sqlitedb for maintaining running services state.

- [ ] Fix issue with services running even after the config is deleted.
- [ ] We should be able to guarantee that service is dead when the config is deleted.

- [ ] Treating microvm management like a package manager.

- [ ] Store service rootfs, state, logs locally in a .mc directory kind of like ./node_modules.
- [ ] Store reference rootfses (oci & monofs) in home_dir with a special store that links to them from forked rootfses.

- [ ] Support multiple registries.
- [ ] Use `Reference` type for image_ref where it makes sense: https://docs.rs/oci-spec/0.7.1/oci_spec/distribution/struct.Reference.html
- [x] Use `Reference` type for image_ref where it makes sense: https://docs.rs/oci-spec/0.7.1/oci_spec/distribution/struct.Reference.html
- [ ] Qualify image names fully where needed. <registry>/<repo>:<tag>
- [ ] Instead of sanitizing image refs, we should just hash them instead.
51 changes: 51 additions & 0 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions monofs/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,11 @@ serde_json.workspace = true
monoutils.workspace = true
serde_ipld_dagcbor.workspace = true
pretty-error-debug.workspace = true
async-trait.workspace = true
tracing.workspace = true
tracing-subscriber.workspace = true
nfsserve = "0.10"
intaglio = "1.10"
libipld.workspace = true
hex.workspace = true
tempfile.workspace = true
8 changes: 7 additions & 1 deletion monofs/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,9 +40,15 @@

## 🚀 Getting Started

### Installation

```sh
curl -sSfL https://install.monofs.dev | sh
```

### Quick Start

TODO: Demo of running multiple servers on different paths syncing up with each other.
TODO: Demo of running multiple servers on different paths syncing up with each other and use with monocore.

### API

Expand Down
13 changes: 11 additions & 2 deletions monofs/lib/filesystem/dir/ops.rs
Original file line number Diff line number Diff line change
Expand Up @@ -264,10 +264,10 @@ where
Ok(())
}

/// Removes an entity at the specified path and returns it.
/// Removes an entity at the specified path.
///
/// This method completely removes the entity from its parent directory, leaving no trace.
/// For a version that marks the entity as deleted but keeps it in the structure, use `remove`.
/// For a version that marks the entity as deleted but keeps it in the structure, use [`remove`](Self::remove).
///
/// ## Examples
///
Expand Down Expand Up @@ -318,6 +318,15 @@ where
/// This method marks the entity as deleted by setting its deleted_at timestamp,
/// but keeps it in the directory structure. The entity will be skipped by find operations.
///
/// #### Why keep deleted entities?
///
/// monofs is designed for local-first applications where changes from multiple sources need to be merged.
/// The deleted_at timestamp serves as a tombstone marker - without it, there would be no way to
/// distinguish between an entity that was deleted in one location versus newly added in another
/// during a merge operation.
///
/// For a version of `remove` that completely removes the entity, use [`remove_trace`](Self::remove_trace).
///
/// ## Examples
///
/// ```
Expand Down
1 change: 1 addition & 0 deletions monofs/lib/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,3 +10,4 @@ pub mod config;
pub mod filesystem;
pub mod store;
pub mod utils;
pub mod server;
7 changes: 7 additions & 0 deletions monofs/lib/server/mod.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
mod server;

//--------------------------------------------------------------------------------------------------
// Exports
//--------------------------------------------------------------------------------------------------

pub use server::*;
Loading

0 comments on commit 4f1add1

Please sign in to comment.