-
Notifications
You must be signed in to change notification settings - Fork 60
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Showing
8 changed files
with
82 additions
and
49 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,29 +1,33 @@ | ||
# Components | ||
|
||
From a developer perspective Walrus has some components that are objects and smart contracts on | ||
Sui, and some components that are an independent set of services. As a rule Sui is used to manage | ||
blob and storage node metadata, while off-sui components are used to actually store and read blob | ||
data, which can be large. | ||
From a developer perspective, some Walrus components are objects and smart contracts on | ||
Sui, and some components are Walrus-specific binaries and services. As a rule Sui is used to | ||
manage blob and storage node meta-data, while Walrus-specific services are used to store and | ||
read blob contents, which can be very large. | ||
|
||
Walrus defines a number of objects and smart contracts on Sui: | ||
|
||
- A shared *system object*, records and manages the current committee of storage nodes. | ||
- *Storage resources*, represent empty storage space that may be used to store blobs. | ||
- *Blob resources*, represent blobs being registered and certified as stored. | ||
- Changes to these objects emit *Walrus-related events*. | ||
|
||
The system object ID for Walrus can be found in the Walrus `client_config.yaml` file. You may use | ||
any Sui explorer to look at its content, as well as explore the content of blob objects. You can | ||
find more information in the [quick reference to the Walrus Sui structures](sui-struct.md). | ||
The Walrus system object ID can be found in the Walrus `client_config.yaml` file | ||
(see [Configuration](configuration.md)). You may use | ||
any Sui explorer to look at its content, as well as explore the content of blob objects. | ||
There is more information about these in the | ||
[quick reference to the Walrus Sui structures](sui-struct.md). | ||
|
||
Walrus is also composed of a number of services and binaries: | ||
Walrus is also composed of a number of Walrus-specific services and binaries: | ||
|
||
- A client (binary) can be executed locally and provides a | ||
[Command Line Interface (CLI)](client-cli.html), a [JSON API](json-api.md) | ||
and an [HTTP API](web-api.md) to perform Walrus operations. | ||
- Aggregators are services that allow download of blobs via HTTP requests. | ||
- Publishers are services used to upload blobs to Walrus. | ||
- A set of storage nodes store encoded stored blobs. | ||
- Aggregators services allow reading blobs via HTTP requests. | ||
- Publishers services are used store blobs to Walrus. | ||
- A set of storage nodes store encoded stored blobs. These nodes form the decentralized | ||
storage infrastructure of Walrus. | ||
|
||
Aggregators, Publishers and other services use the client APIs to interact with Walrus. End-users | ||
of services using walrus interact with the store via custom services, aggregators or publishers that | ||
Aggregators, publishers and other services use the client APIs to interact with Walrus. End-users | ||
of services using Walrus interact with the store via custom services, aggregators or publishers that | ||
expose HTTP APIs to avoid the need to run locally a binary client. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Configuration |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -4,17 +4,20 @@ This guide introduces all the concepts needed to build applications that use Wal | |
or availability layer. The [overview](./overview.md) provides more background and explains in | ||
more detail how Walrus operates internally. | ||
|
||
**Disclaimer about the Walrus developer preview: This release of Walrus \& Walrus Sites is a | ||
developer preview, to showcase the technology and solicit feedback from builders. All storage nodes | ||
and aggregators are operated by MystenLabs, all transactions are executed on the Sui testnet, | ||
and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any | ||
point and possibly with no warning. Do not rely on this the developer preview for any production | ||
purposes, it comes with no availability or persistence guarantees.** | ||
|
||
This developer guide describes: | ||
|
||
- [Components](components.md) of Walrus of interest to developers that wish to use it for | ||
storage or availability. | ||
- [Operations](dev-operations.md) supported through client binaries, APIs, or Sui operations. | ||
- [The Sui structures](sui-struct.md) Walrus uses to store metadata, and how they can be read | ||
from Sui smart contracts, or through the Sui SDK. | ||
from Sui smart contracts, or through the Sui SDK. | ||
|
||
|
||
Check failure on line 15 in docs/dev-guide.md GitHub Actions / Lint all markdown filesMultiple consecutive blank lines
|
||
## Disclaimer about the Walrus developer preview | ||
|
||
**This release of Walrus \& Walrus Sites is a | ||
developer preview, to showcase the technology and solicit feedback from builders. All storage nodes | ||
and aggregators are operated by MystenLabs, all transactions are executed on the Sui testnet, | ||
and use testnet SUI which has no value. The state of the store can be, and will be wiped, at any | ||
point and possibly with no warning. Do not rely on this the developer preview for any production | ||
purposes, it comes with no availability or persistence guarantees.** |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -6,15 +6,15 @@ Walrus may be used to **store a blob**, via the native client APIs or a publishe | |
number of operations happen both on Sui as well as on storage nodes: | ||
|
||
- The client or publisher encodes the blob and derives a *blob ID* that identifies the blob. This | ||
us a `u256` number often encoded as a URL safe base64 string. | ||
is a `u256` often encoded as a URL safe base64 string. | ||
- A transaction is executed on Sui to purchase some storage from the system object, and then to | ||
*register the blob ID* with this storage. Client APIs return the *Sui blob object ID*. The | ||
transactions use SUI to purchase storage and pay for gas. | ||
- Encoded slivers of the blob are distributed to all storage nodes. They each sign a receipt. | ||
- The signed receipts are aggregated and submitted to the Sui blob object to *certify the blob*. | ||
- Signed receipts are aggregated and submitted to the Sui blob object to *certify the blob*. | ||
Certifying a blob emits a Sui event with the blob ID and the period of availability. | ||
|
||
A blob can be considered as available on Walrus once the corresponding Sui blob object has been | ||
A blob is considered available on Walrus once the corresponding Sui blob object has been | ||
certified in the final step. The steps involved in a store can be executed by the binary client, | ||
or a publisher that accepts and publishes blobs via HTTP. | ||
|
||
|
@@ -31,22 +31,26 @@ performing the following steps: | |
- The blob is reconstructed and checked against the blob ID from the recovered slivers. | ||
|
||
The steps involved in the read operation are performed by the binary client, or the aggregator | ||
service that exposes an HTTP interface to read blobs. | ||
service that exposes an HTTP interface to read blobs. Reads are extremely resilient and will | ||
succeed in recovering the blob stored even if up to one-third of storage nodes are | ||
unavailable. | ||
|
||
### Certify Availability | ||
|
||
One may *certify the availability of a blob* using Sui. This may be done in 3 different ways: | ||
Walrus can be used to **certify the availability of a blob** using Sui. This may be done in 3 | ||
different ways: | ||
|
||
- A Sui [light-client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client) may be | ||
- A Sui SDK read can be | ||
used to authenticate the certified blob event emitted when the blob ID was certified on Sui. | ||
- A Sui [light-client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client) may be | ||
used to authenticate the blob Sui object corresponding to the blob ID is certified. | ||
- A Sui smart contract can be provided with the blob object on Sui (or a reference to it) to check | ||
- A Sui SDK read may be | ||
used to authenticate the Sui blob object corresponding to the blob ID, and check it is certified. | ||
- A Sui smart contract can read the blob object on Sui (or a reference to it) to check | ||
is is certified. | ||
|
||
The underlying protocol of the Sui light client returns digitally signed evidence for emitted events | ||
The underlying protocol of the [Sui light client](https://github.com/MystenLabs/sui/tree/main/crates/sui-light-client) | ||
Check failure on line 50 in docs/dev-operations.md GitHub Actions / Lint all markdown filesLine length
|
||
client returns digitally signed evidence for emitted events | ||
or objects, and can be used by off-line or non-interactive applications as a proof of availability | ||
for the blob ID for a certain number of epochs. | ||
|
||
Once a blob is certified, the Walrus store will ensure that sufficient slivers will always be | ||
available on storage nodes to be able to recover it within the specified epochs. | ||
Once a blob is certified, Walrus will ensure that sufficient slivers will always be | ||
available on storage nodes to recover it within the specified epochs. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
# Examples |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,13 @@ | ||
# Python | ||
|
||
The [python examples](https://github.com/MystenLabs/walrus-docs/tree/main/examples/python) folder | ||
contains a number of examples: | ||
|
||
- How to [use the HTTP API](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_webapi.py) | ||
Check failure on line 6 in docs/python.md GitHub Actions / Lint all markdown filesLine length
|
||
to store and read a blob. | ||
- How to [use the JSON API](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_jsonapi.py) | ||
Check failure on line 8 in docs/python.md GitHub Actions / Lint all markdown filesLine length
|
||
to store, read, and check the availability of a blob. Checking the certification of a blob | ||
illustrates reading the Blob Sui object that certifies | ||
(see the [Walrus Sui reference](sui-struct.md)). | ||
- How to [read information from the Walrus system object](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/hello_walrus_sui_system.py). | ||
Check failure on line 12 in docs/python.md GitHub Actions / Lint all markdown filesLine length
|
||
- How to [track Walrus related Events](https://github.com/MystenLabs/walrus-docs/blob/main/examples/python/track_walrus_events.py). | ||
Check failure on line 13 in docs/python.md GitHub Actions / Lint all markdown filesLine length
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters