Skip to content

Commit

Permalink
docs: document recent changes to the CLI and publisher (#178)
Browse files Browse the repository at this point in the history
* docs: document recent changes to the CLI and publisher

* update po files
  • Loading branch information
mlegner authored Dec 4, 2024
1 parent c63c65b commit ca2d177
Show file tree
Hide file tree
Showing 4 changed files with 670 additions and 442 deletions.
53 changes: 36 additions & 17 deletions docs/usage/client-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,25 +15,32 @@ their meaning.
## Walrus system information

Information about the Walrus system is available through the `walrus info` command. For example,
`walrus info` gives an overview of the number of storage nodes and shards in the system, the maximum
blob size, and the current cost in (Testnet) WAL for storing blobs:
`walrus info` gives an overview of current system parameters such as the current epoch, the number
of storage nodes and shards in the system, the maximum blob size, and the current cost in (Testnet)
WAL for storing blobs:

```console
$ walrus info

Walrus system information
Current epoch: 2

Epochs and storage duration
Current epoch: 47
Epoch duration: 1day
Blobs can be stored for at most 200 epochs in the future.

Storage nodes
Number of nodes: 25
Number of storage nodes: 30
Number of shards: 1000

Blob size
Maximum blob size: 13.3 GiB (14,273,391,930 B)
Storage unit: 1.00 MiB

Approximate storage prices per epoch
(Conversion rate: 1 WAL = 1,000,000,000 FROST)
Price per encoded storage unit: 100 FROST
Additional price for each write: 2,000 FROST
Price to store metadata: 6,200 FROST
Marginal price per additional 1 MiB (w/o metadata): 500 FROST

Expand All @@ -49,8 +56,9 @@ SUI: `1 WAL = 1 000 000 000 FROST`.
```

Additional information such as encoding parameters and sizes, BFT system information, and
information on the storage nodes and their shard distribution can be viewed with the `--dev`
argument: `walrus info --dev`.
information on the storage nodes in the current and (if already selected) the next committee,
including their node IDs and stake and shard distribution can be viewed with the `--dev` argument:
`walrus info --dev`.

## Storing, querying status, and reading blobs

Expand All @@ -74,11 +82,17 @@ page](./setup.md#testnet-wal-faucet) for further details.
Storing blobs on Walrus can be achieved through the following command:

```sh
walrus store <some file>
walrus store <some file> --epochs <EPOCHS>
```

The store command takes a CLI argument `--epochs <EPOCHS>` (or `-e`) indicating the number of
epochs the blob should be stored for. This defaults to 1 epoch, namely the current one.
The CLI argument `--epochs <EPOCHS>` (or `-e`) indicates the number of epochs the blob should be
stored for. There is an upper limit on the number of epochs a blob can be stored for, which is 200
for the current Testnet deployment.

```admonish warning
The `--epochs` argument is currently optional; if it is not specified, a warning is printed and the
default of 1 epoch is used. However, it will become a mandatory argument in the future.
```

```admonish tip title="Automatic optimizations"
When storing a blob, the client performs a number of automatic optimizations, including the
Expand Down Expand Up @@ -137,15 +151,20 @@ walrus delete --blob-id <BLOB_ID>
Optionally the delete command can be invoked by specifying a `--file <PATH>` option, to derive the
blob ID from a file, or `--object-id <SUI_ID>` to delete the blob in the Sui blob object specified.

The `delete` command reclaims the storage object associated with the deleted blob, which is
re-used to store new blobs. The delete operation provides
flexibility around managing storage costs and re-using storage.
Before deleting a blob, the `walrus delete` command will ask for confirmation unless the `--yes`
(or `-y`) option is specified.

The `delete` command reclaims the storage object associated with the deleted blob, which is re-used
to store new blobs. The delete operation provides flexibility around managing storage costs and
re-using storage.

The delete operation has limited utility for privacy: It only deletes slivers from the current
epoch storage nodes, and subsequent epoch storage nodes, if no other user has uploaded a copy of
the same blob. If another copy of the same blob exists in Walrus the delete operation will not
make the blob unavailable for download, and `walrus read` invocations will download it. Copies of
the public blob may be cached or downloaded by users, and these copies are not deleted.
The delete operation has limited utility for privacy: It only deletes slivers from the current epoch
storage nodes, and subsequent epoch storage nodes, if no other user has uploaded a copy of the same
blob. If another copy of the same blob exists in Walrus, the delete operation will not make the blob
unavailable for download, and `walrus read` invocations will download it. After the deletion is
finished, the CLI checks the updated status of the blob to see if it is still accessible in Walrus
(unless the `--no-status-check` option is specified). However, even if it isn't, copies of the
public blob may be cached or downloaded by users, and these copies are not deleted.

```admonish danger title="Delete reclaims space only"
**All blobs stored in Walrus are public and discoverable by all.** The `delete` command will
Expand Down
46 changes: 35 additions & 11 deletions docs/usage/web-api.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,27 +8,51 @@ the need to run a local client.

## Starting the daemon locally {#local-daemon}

You can run the daemon with the following command, to offer both an aggregator and publisher on
the same address (`127.0.0.1`) and port (`31415`):
You can run a local Walrus daemon through the `walrus` binary. There are three different commands:

- `walrus aggregator` starts an "aggregator" that offers an HTTP interface to read blobs from
Walrus.
- `walrus publisher` starts a "publisher" that offers an HTTP interface to store blobs in Walrus.
- `walrus daemon` offers the combined functionality of an aggregator and publisher on the same
address and port.

The aggregator does not perform any on-chain actions, and only requires specifying the address on
which it listens:

```sh
walrus daemon -b "127.0.0.1:31415"
walrus aggregator --bind-address "127.0.0.1:31415"
```

Or you may run the aggregator and publisher processes separately on different addresses/ports:
The publisher and daemon perform on-chain actions and thus require a Sui Testnet wallet with
sufficient SUI and WAL balances. To enable handling many parallel requests without object
conflicts, they create internal sub-wallets since version 1.4.0, which are funded from the main
wallet. These sub-wallets are persisted in a directory specified with the `--sub-wallets-dir`
argument; any existing directory can be used. If it already contains sub-wallets, they will be
reused.

By default, 8 sub-wallets are created and funded. This can be changed with the `--n-clients`
argument. For simple local testing, 1 or 2 sub-wallets are usually sufficient.

For example, you can run a publisher with a single sub-wallet stored in the Walrus configuration
directory with the following command:

```sh
walrus aggregator -b "127.0.0.1:31415" # run an aggregator to read blobs
walrus publisher -b "127.0.0.1:31416" # run a publisher to store blobs
PUBLISHER_WALLETS_DIR=~/.config/walrus/publisher-wallets
mkdir -p "$PUBLISHER_WALLETS_DIR"
walrus publisher \
--bind-address "127.0.0.1:31416" \
--sub-wallets-dir "$PUBLISHER_WALLETS_DIR" \
--n-clients 1
```

The aggregator provides all read APIs, the publisher all the store APIs, and the daemon provides
both.
Replace `publisher` by `daemon` to run both an aggregator and publisher on the same address and
port.

```admonish warning
While the aggregator does not perform Sui on-chain actions, and therefore consumes no gas, the
publisher does perform actions on-chain and will consume gas. It is therefore important to ensure
only authorized parties may access it, or other measures to manage gas costs.
publisher does perform actions on-chain and will consume both SUI and WAL tokens. It is therefore
important to ensure only authorized parties may access it, or other measures to manage gas costs,
especially in a future Mainnet deployment.
```

## Using a public aggregator or publisher {#public-services}
Expand Down Expand Up @@ -109,7 +133,7 @@ PUBLISHER=https://publisher.walrus-testnet.walrus.space

```admonish tip title="API specification"
Walrus aggregators and publishers expose their API specifications at the path `/v1/api`. You can
view this in the browser, e.g., at <https://aggregator.walrus-testnet.walrus.space/v1/api>
view this in the browser, for example, at <https://aggregator.walrus-testnet.walrus.space/v1/api>.
```

### Store
Expand Down
Loading

0 comments on commit ca2d177

Please sign in to comment.