Skip to content

Commit

Permalink
Copy edited and proof read docs
Browse files Browse the repository at this point in the history
  • Loading branch information
gdanezis committed Jun 11, 2024
1 parent 0909941 commit 3c9391b
Show file tree
Hide file tree
Showing 8 changed files with 82 additions and 49 deletions.
5 changes: 3 additions & 2 deletions docs/SUMMARY.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,12 +26,13 @@
- [Setup]()
- [Prerequisites]()
- [Installation]()
- [Configuration]()
- [Configuration](configuration.md)
- [Interacting with Walrus](interacting.md)
- [Using the client CLI](client-cli.md)
- [Using the client JSON API](json-api.md)
- [Using the client HTTP API](web-api.md)
- [Examples]()
- [Examples](examples.md)
- [Python](python.md)

# Walrus sites

Expand Down
30 changes: 17 additions & 13 deletions docs/components.md
Original file line number Diff line number Diff line change
@@ -1,29 +1,33 @@
# Components

From a developer perspective Walrus has some components that are objects and smart contracts on
Sui, and some components that are an independent set of services. As a rule Sui is used to manage
blob and storage node metadata, while off-sui components are used to actually store and read blob
data, which can be large.
From a developer perspective, some Walrus components are objects and smart contracts on
Sui, and some components are Walrus-specific binaries and services. As a rule Sui is used to
manage blob and storage node meta-data, while Walrus-specific services are used to store and
read blob contents, which can be very large.

Walrus defines a number of objects and smart contracts on Sui:

- A shared *system object*, records and manages the current committee of storage nodes.
- *Storage resources*, represent empty storage space that may be used to store blobs.
- *Blob resources*, represent blobs being registered and certified as stored.
- Changes to these objects emit *Walrus-related events*.

The system object ID for Walrus can be found in the Walrus `client_config.yaml` file. You may use
any Sui explorer to look at its content, as well as explore the content of blob objects. You can
find more information in the [quick reference to the Walrus Sui structures](sui-struct.md).
The Walrus system object ID can be found in the Walrus `client_config.yaml` file
(see [Configuration](configuration.md)). You may use
any Sui explorer to look at its content, as well as explore the content of blob objects.
There is more information about these in the
[quick reference to the Walrus Sui structures](sui-struct.md).

Walrus is also composed of a number of services and binaries:
Walrus is also composed of a number of Walrus-specific services and binaries:

- A client (binary) can be executed locally and provides a
[Command Line Interface (CLI)](client-cli.html), a [JSON API](json-api.md)
and an [HTTP API](web-api.md) to perform Walrus operations.
- Aggregators are services that allow download of blobs via HTTP requests.
- Publishers are services used to upload blobs to Walrus.
- A set of storage nodes store encoded stored blobs.
- Aggregators services allow reading blobs via HTTP requests.
- Publishers services are used store blobs to Walrus.
- A set of storage nodes store encoded stored blobs. These nodes form the decentralized
storage infrastructure of Walrus.

Aggregators, Publishers and other services use the client APIs to interact with Walrus. End-users
of services using walrus interact with the store via custom services, aggregators or publishers that
Aggregators, publishers and other services use the client APIs to interact with Walrus. End-users
of services using Walrus interact with the store via custom services, aggregators or publishers that
expose HTTP APIs to avoid the need to run locally a binary client.
1 change: 1 addition & 0 deletions docs/configuration.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Configuration
19 changes: 11 additions & 8 deletions docs/dev-guide.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,20 @@ This guide introduces all the concepts needed to build applications that use Wal
or availability layer. The [overview](./overview.md) provides more background and explains in
more detail how Walrus operates internally.

**Disclaimer about the Walrus developer preview: This release of Walrus \& Walrus Sites is a
developer preview, to showcase the technology and solicit feedback from builders. All storage nodes
and aggregators are operated by MystenLabs, all transactions are executed on the Sui testnet,
and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any
point and possibly with no warning. Do not rely on this the developer preview for any production
purposes, it comes with no availability or persistence guarantees.**

This developer guide describes:

- [Components](components.md) of Walrus of interest to developers that wish to use it for
storage or availability.
- [Operations](dev-operations.md) supported through client binaries, APIs, or Sui operations.
- [The Sui structures](sui-struct.md) Walrus uses to store metadata, and how they can be read
from Sui smart contracts, or through the Sui SDK.
from Sui smart contracts, or through the Sui SDK.


Check failure on line 15 in docs/dev-guide.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Multiple consecutive blank lines

docs/dev-guide.md:15 MD012/no-multiple-blanks Multiple consecutive blank lines [Expected: 1; Actual: 2] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md012.md
## Disclaimer about the Walrus developer preview

**This release of Walrus \& Walrus Sites is a
developer preview, to showcase the technology and solicit feedback from builders. All storage nodes
and aggregators are operated by MystenLabs, all transactions are executed on the Sui testnet,
and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any
point and possibly with no warning. Do not rely on this the developer preview for any production
purposes, it comes with no availability or persistence guarantees.**
28 changes: 16 additions & 12 deletions docs/dev-operations.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,15 +6,15 @@ Walrus may be used to **store a blob**, via the native client APIs or a publishe
number of operations happen both on Sui as well as on storage nodes:

- The client or publisher encodes the blob and derives a *blob ID* that identifies the blob. This
us a `u256` number often encoded as a URL safe base64 string.
is a `u256` often encoded as a URL safe base64 string.
- A transaction is executed on Sui to purchase some storage from the system object, and then to
*register the blob ID* with this storage. Client APIs return the *Sui blob object ID*. The
transactions use SUI to purchase storage and pay for gas.
- Encoded slivers of the blob are distributed to all storage nodes. They each sign a receipt.
- The signed receipts are aggregated and submitted to the Sui blob object to *certify the blob*.
- Signed receipts are aggregated and submitted to the Sui blob object to *certify the blob*.
Certifying a blob emits a Sui event with the blob ID and the period of availability.

A blob can be considered as available on Walrus once the corresponding Sui blob object has been
A blob is considered available on Walrus once the corresponding Sui blob object has been
certified in the final step. The steps involved in a store can be executed by the binary client,
or a publisher that accepts and publishes blobs via HTTP.

Expand All @@ -31,22 +31,26 @@ performing the following steps:
- The blob is reconstructed and checked against the blob ID from the recovered slivers.

The steps involved in the read operation are performed by the binary client, or the aggregator
service that exposes an HTTP interface to read blobs.
service that exposes an HTTP interface to read blobs. Reads are extremely resilient and will
succeed in recovering the blob stored even if up to one-third of storage nodes are
unavailable.

### Certify Availability

One may *certify the availability of a blob* using Sui. This may be done in 3 different ways:
Walrus can be used to **certify the availability of a blob** using Sui. This may be done in 3
different ways:

- A Sui [light-client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client) may be
- A Sui SDK read can be
used to authenticate the certified blob event emitted when the blob ID was certified on Sui.
- A Sui [light-client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client) may be
used to authenticate the blob Sui object corresponding to the blob ID is certified.
- A Sui smart contract can be provided with the blob object on Sui (or a reference to it) to check
- A Sui SDK read may be
used to authenticate the Sui blob object corresponding to the blob ID, and check it is certified.
- A Sui smart contract can read the blob object on Sui (or a reference to it) to check
is is certified.

The underlying protocol of the Sui light client returns digitally signed evidence for emitted events
The underlying protocol of the [Sui light client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client)

Check failure on line 50 in docs/dev-operations.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Line length

docs/dev-operations.md:50:101 MD013/line-length Line length [Expected: 100; Actual: 118] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md
client returns digitally signed evidence for emitted events
or objects, and can be used by off-line or non-interactive applications as a proof of availability
for the blob ID for a certain number of epochs.

Once a blob is certified, the Walrus store will ensure that sufficient slivers will always be
available on storage nodes to be able to recover it within the specified epochs.
Once a blob is certified, Walrus will ensure that sufficient slivers will always be
available on storage nodes to recover it within the specified epochs.
1 change: 1 addition & 0 deletions docs/examples.md
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
# Examples
13 changes: 13 additions & 0 deletions docs/python.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
# Python

The [python examples](https://github.com/MystenLabs/walrus-docs/tree/main/examples/python) folder
contains a number of examples:

- How to [use the HTTP API](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_webapi.py)

Check failure on line 6 in docs/python.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Line length

docs/python.md:6:101 MD013/line-length Line length [Expected: 100; Actual: 119] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md
to store and read a blob.
- How to [use the JSON API](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_jsonapi.py)

Check failure on line 8 in docs/python.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Line length

docs/python.md:8:101 MD013/line-length Line length [Expected: 100; Actual: 120] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md
to store, read, and check the availability of a blob. Checking the certification of a blob
illustrates reading the Blob Sui object that certifies
(see the [Walrus Sui reference](sui-struct.md)).
- How to [read information from the Walrus system object](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_sui_system.py).

Check failure on line 12 in docs/python.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Line length

docs/python.md:12:101 MD013/line-length Line length [Expected: 100; Actual: 154] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md
- How to [track Walrus related Events](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/track_walrus_events.py).

Check failure on line 13 in docs/python.md

View workflow job for this annotation

GitHub Actions / Lint all markdown files

Line length

docs/python.md:13:101 MD013/line-length Line length [Expected: 100; Actual: 131] https://github.com/DavidAnson/markdownlint/blob/v0.34.0/doc/md013.md
34 changes: 20 additions & 14 deletions docs/sui-struct.md
Original file line number Diff line number Diff line change
@@ -1,27 +1,30 @@

# Walrus Sui Reference
# Sui Structures

This section is optional and enables advanced use cases.

You can interact with Walrus purely
through the client CLI, and JSON or HTTP APIs provided, without reading or writing to the Sui chain
directly. However, Sui is used to manage the metadata of Walrus and developers can use information
about the Walrus system, as well as stored blobs using Sui smart contracts.
through the client CLI, and JSON or HTTP APIs provided, without querying or executing transactions
on Sui directly. However, Walrus uses Sui to manage its meta-data and smart contract developers can
read information about the Walrus system, as well as stored blobs, on Sui.

This section provides and overview of how you may use Walrus objects in your Sui smart contracts.

## Blob and Storage Objects

Walrus blobs are represented as Sui `Blob` types. A blob may be registered, indicating that the
storage nodes should expect slivers from a Blob ID to be stored. Then a blob can be certified
Walrus blobs are represented as Sui objects of type `Blob`. A blob is first registered, indicating
that the storage nodes should expect slivers from a Blob ID to be stored. Then a blob is certified
indicating that a sufficient number of slivers have been stored to guarantee the blob's
availability. When a blob is certified its `certified` filed contains the epoch in which it was
availability. When a blob is certified its `certified` field contains the epoch in which it was
certified.

A `Storage` object is always associated with a Blob, reserving enough space for
a long enough period for its storage. A certified blob is available for the period the
A `Storage` object is always associated with a `Blob` object, reserving enough space for
a long enough period for the blob's storage. A certified blob is available for the period the
underlying storage resource guarantees storage.

Concretely, `Blob` and `Storage` objects have the following fields, that can be read through the
Sui SDKs:

```move
/// The blob structure represents a blob that has been registered to with some
/// storage, and then may eventually be certified as being available in the
Expand Down Expand Up @@ -89,18 +92,21 @@ public struct BlobCertified has copy, drop {
blob_id: u256,
end_epoch: u64,
}
```

The `InvalidBlobID` event is emitted when storage nodes detect and incorrectly encoded blob.
Such a blob is guaranteed to be also detected as invalid when a read is attempted.


```move
/// Signals that a BlobID is invalid.
public struct InvalidBlobID has copy, drop {
epoch: u64, // The epoch in which the blob ID is first registered as invalid
blob_id: u256,
}
```

The `InvalidBlobID` event is emitted when storage nodes detect and incorrectly encoded blob.
Such a blob is guaranteed to be also detected as invalid when a read is attempted.

## Walrus System information
## System information

The Walrus system object contains meta-data about the available and used storage, as well as the
price of storage per 100 Kib of storage in MIST. The committee
Expand Down Expand Up @@ -138,7 +144,7 @@ public struct Committee has store {
}
```

A few public functions of the committee allow contracts to read metadata:
A few public functions of the committee allow contracts to read Walrus meta-data:

```move
/// Get epoch. Uses the committee to get the epoch.
Expand Down

0 comments on commit 3c9391b

Please sign in to comment.