diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md index 426d96f0..79a9c992 100644 --- a/docs/SUMMARY.md +++ b/docs/SUMMARY.md @@ -31,6 +31,17 @@ # Walrus sites +- [Introduction to Walrus Sites](./walrus-sites/intro.md) +- [Your first Walrus Site](./walrus-sites/tutorial.md) + - [Installing the site builder](./walrus-sites/tutorial-install.md) + - [Publishing a Walrus Site](./walrus-sites/tutorial-publish.md) + - [Bonus: Set a SuiNS name](./walrus-sites/tutorial-suins.md) + - [Advanced configuration](./walrus-sites/tutorial-config.md) +- [Technical overview](./walrus-sites/overview.md) + - [The site builder](./walrus-sites/site-builder.md) + - [The Walrus Sites Portal](./walrus-sites/portal.md) + - [Known restrictions](./walrus-sites/restrictions.md) + --- [Glossary](./glossary.md) diff --git a/docs/assets/suins-asset.png b/docs/assets/suins-asset.png new file mode 100644 index 00000000..bb701dbe Binary files /dev/null and b/docs/assets/suins-asset.png differ diff --git a/docs/walrus-sites/intro.md b/docs/walrus-sites/intro.md new file mode 100644 index 00000000..418c237d --- /dev/null +++ b/docs/walrus-sites/intro.md @@ -0,0 +1,62 @@ +# Introduction to Walrus Sites + +_Walrus Sites_ are "web"-sites that use Sui and Walrus as their underlying technology. They are a +prime example of how Walrus can be used to build new and exciting decentralized applications. Anyone +can build and deploy a Walrus Site, and make it accessible to the World! + +At a high level, the most exciting features include: + +- Publishing a site does not require managing servers or complex configurations; just provide the + source files (produced by your favorite web framework), publish them to Walrus Sites using the + [site builder tool](./site-builder.md), and you are done! +- Sites can be linked to from ordinary Sui objects. This feature enables, for example, creating a + NFT collection in which _every single NFT_ has a _personalized website dedicated to it_. +- Walrus Sites are owned by addresses on Sui, and can be exchanged, shared, and updated thanks to + Sui's flexible programming model. This means, among other things, that Walrus Sites can leverage + the [SuiNS](https://suins.io/) naming system to have human readable names. No more messing around + with DNS! +- Thanks to Walrus' decentralization and extremely high data availability, there is no risk of + having your site wiped for no reason. +- Since they live on Walrus, these sites cannot have a backend in the traditional sense, and can be + therefore considered "static" sites. However, the developer can harness Sui's programmability + to add backend functionality to Walrus Sites! + +## Show me + +To give you a very high level intuition of how Walrus Sites work, let's look at an example: A simple +NFT collection on Sui, that has a frontend dApp to mint the NFTs hosted on Walrus Sites, and in +which _each NFT_ has a _specific, personalized Walrus Site_. + +You can check out the mint page at +[https://flatland.walrus.site/](https://flatland.walrus.site/). This site is serve to your browser +through the Walrus Site _Portal_ [https://walrus.site](https://walrus.site). While the Portal's +operation is explained in a [later section](./portal.md), consider for now that there +can be many Portals (hosted by whomever wants to have their own, and even on `localhost`). Further, +the only function of the Portal is to provide the browser with some code (specifically, a service +worker) that allows it to fetch the Walrus Site from Sui and Walrus. + +If you have a Sui wallet with some Testnet SUI, you can try and "mint a new Flatlander" from the +site. This creates an NFT from the collection, and shows you two links, one to the explorer, and one +to the "Flatlander site". This latter site is a special Walrus Site that exists only for that NFT, +and has special characteristics (the background color, the image...) that are based on the contents +of the NFT. + +The URL to this per-NFT site look something like this +`https://4egmmrw9izzjn0dm2lkd3k0l8phk386z60ub1tpdc1jswbb5dr.walrus.site/`. You'll notice that the +domain remains `walrus.site`, but the subdomain is a long and random-looking string. This string is +actually the [Base36](https://en.wikipedia.org/wiki/Base36) encoding of the object ID of the NFT, +which is +[0xb09b312b...](https://suiscan.xyz/testnet/object/0xb09b312b28049467dd6173b6cebb60ed5fe3046883e248632bf9fb20b7dbdaff). + +In summary: + +- Walrus Sites are served through a Portal; in this case, `https://walrus.site`. There can be many + Portals, and anyone can host one. +- The subdomain on the URL points to a specific object on Sui, that allows the browser to fetch and + render the site resources. This pointer can be: + - A SuiNS name, such as `flatland` in `https://flatland.walrus.site`; or + - the Base36 encoding of a the Sui object ID, such as `0xb09b312b...` in the example above. + +Curious to know how this magic is possible? Read the [technical +overview](./overview.md)! If you just want to get started trying Walrus Sites out, check the +[tutorial](./tutorial.md). diff --git a/docs/walrus-sites/overview.md b/docs/walrus-sites/overview.md new file mode 100644 index 00000000..07dd0c5c --- /dev/null +++ b/docs/walrus-sites/overview.md @@ -0,0 +1 @@ +# Overview diff --git a/docs/walrus-sites/portal.md b/docs/walrus-sites/portal.md new file mode 100644 index 00000000..728c8ca9 --- /dev/null +++ b/docs/walrus-sites/portal.md @@ -0,0 +1 @@ +# The Walrus Sites Portal diff --git a/docs/walrus-sites/restrictions.md b/docs/walrus-sites/restrictions.md new file mode 100644 index 00000000..d3a4f48e --- /dev/null +++ b/docs/walrus-sites/restrictions.md @@ -0,0 +1,51 @@ +# Known restrictions + +Walrus Sites can be used to deploy almost any form of traditional static "Web 2" website build for +modern browsers. There are, however, a number of restrictions that a developer should keep in mind +when creating or porting a website to Walrus Sites. + +## No secret values + +Walrus Sites are fully publicly accessible, as the metadata is stored on Sui and the site content is +stored on Walrus. Therefore, developers _must_ not store secret values within the sites. + +We emphasize again that any such backend-specific operations (storing secret values, authentication, +etc.) are achievable by leveraging the integration with Sui blockchain and the Sui wallet. + +## There is a maximum redirect depth + +The number of consecutive redirects a Walrus Site can perform is capped by the +Portal (see [Portal configuration](./portal.md)). This measure ensures that loading a Walrus Site +does not result in an infinite loading loop. + +Different Portals can set this limit as they desire. The limit for the Portal hosted at +[walrus.site](http://walrus.site) has a maximum redirect depth of 3. + +## Service workers are not available + +**WARNING**: This limitation only applies to Portal based on service workers. A web Portal will not +have this limitation. + +Walrus Sites leverage service workers in the clients' browsers to perform these essential +operations: + +1. reading the site metadata from Sui; +1. fetching the page content from Walrus; and +1. serving the content to the browser. + +Therefore, a site deployed on Walrus Sites cannot use service workers. Installing a service worker +from within a Walrus Site will result in a dysfunctional site and a poor experience for the user. + +## The iOS Sui Wallet Mobile does not work with Walrus Sites + +**WARNING**: This limitation only applies to Portal based on service workers. A web Portal will not +have this limitation. + +Service workers cannot be loaded inside an in-app browser on iOS, because of a limitation of the +WebKit engine. As a consequence, Walrus Sites cannot be used within the [Sui Wallet +Mobile](https://apps.apple.com/us/app/sui-wallet-mobile/id6476572140) app on iOS, and therefore +the Sui wallet can currently not be used on a Walrus Site on iOS. Note, however, that _browsing_ a +Walrus Site is still possible on iOS through any browser. Only the connection to the wallet is +impacted. + +The connection with the Sui Wallet Mobile app works on Android devices. diff --git a/docs/walrus-sites/site-builder.md b/docs/walrus-sites/site-builder.md new file mode 100644 index 00000000..f54a2592 --- /dev/null +++ b/docs/walrus-sites/site-builder.md @@ -0,0 +1 @@ +# The site builder diff --git a/docs/walrus-sites/tutorial-config.md b/docs/walrus-sites/tutorial-config.md new file mode 100644 index 00000000..0132821e --- /dev/null +++ b/docs/walrus-sites/tutorial-config.md @@ -0,0 +1,36 @@ +# Configuring the site builder + +Configuring the `site-builder` tool is straightforward, but care is required to ensure that +everything works correctly. + +The `site-builder` tool requires a configuration file to know which package to use on Sui, which +wallet to use, the gas budget, and other operational details. Most of these are abstracted away +through sensible defaults, so you should not need to touch them. Yet, for completeness, we provide +here the details for all the configuration options. + +## Minimal configuration + +The config file is expected to be in `./builder.yaml`, and it is possible to point elsewhere with +the `--config` flag. For your first run, it should be sufficient to call the `site-builder` with +`--config assets/builder-example.yaml`, which is already configured appropriately. + +## Advanced configuration + +If you want to have more control over the behavior of the site builder, you can customize the +following variables in the config file: + +- `package`: the object ID of the Walrus Sites package on Sui. This must always be specified in the + config, and is already appropriately configured in `assets/example-config.yaml`. +- `portal`: the name of the Portal through which the site will be viewed; this only affects the + output of the CLI, and nothing else (default: `walrus.site`). + All Walrus Sites are accessible through any Portal independent of this setting. +- `general`: these are general options, that can be configured both through the CLI and the config: + - `rpc_url`: The URL of the Sui RPC node to use. If not set, the `site-builder` will infer it from + the wallet. + - `wallet`: Pointer to the Sui wallet to be used. By default, it uses the system-wide wallet (the + one from `sui client addresses`). + - `walrus_binary`: Pointer to the `walrus` binary. By default, this is expected to be run from + `$PATH`. + - `walrus_config`: The configuration for the `walrus` client binary. See the relative + documentation (TODO: link). + - `gas_budget`: The maximum amount of gas to be spent for transactions (default: 500M MIST). diff --git a/docs/walrus-sites/tutorial-install.md b/docs/walrus-sites/tutorial-install.md new file mode 100644 index 00000000..67762be3 --- /dev/null +++ b/docs/walrus-sites/tutorial-install.md @@ -0,0 +1,122 @@ +# Installing the site builder + +We describe here the steps necessary to setup the Walrus Sites' `site-builder` tool, and prepare +your environment for development. + +## Prerequisites + +Before you start, make sure you: + +- Have a recent version of [Rust](https://www.rust-lang.org/tools/install) installed; and +- have the [Sui CLI](https://docs.sui.io/guides/developer/getting-started/sui-install) installed. + +Then, follow these additional setup steps. + +## Get the `walrus` binary and install it + +Download the latest `walrus` binary for your architecture from +`https://storage.googleapis.com/mysten-walrus-binaries/walrus-v0.1.0-a0fb8c9-`, where `` +is your architecture. The available options are listed in this table: + +| OS | CPU | Architecture | +|--------|---------------|-----------------| +| MacOS | Apple Silicon | `macos-arm64` | +| MacOS | Intel 64bit | `macos-x86_64` | +| Ubuntu | Intel 64bit | `ubuntu-x86_64` | + +Then, add it to your `$PATH`. For example, on MacOS you can copy it to +`/Users/myusername/.local/bin/` (check what directories are in your `$PATH` by running `echo +$PATH`). + +Once this is done, you should be able to type `walrus` in your terminal and see: + +``` txt +Walrus client + +Usage: walrus [OPTIONS] + +⋮ +``` + +If, for any reason, you don't want to add `walrus` to `$PATH`, place the binary in your preferred +directory, and remember to configure a pointer to the binary in the `site-builder` config (more on +this [later](./tutorial-config.md)). + +## Point your Sui CLI to testnet, and get some SUI + +Walrus is currently deployed on Sui Testnet. Therefore, you have to ensure that your Sui CLI is +configured accordingly: + +``` txt +sui client envs +╭──────────┬──────────────────────────────────────┬────────╮ +│ alias │ url │ active │ +├──────────┼──────────────────────────────────────┼────────┤ +│ devnet │ https://fullnode.devnet.sui.io:443 │ │ +│ local │ http://127.0.0.1:9000 │ │ +│ testnet │ https://fullnode.testnet.sui.io:443/ │ * │ +│ mainnet │ https://fullnode.mainnet.sui.io:443 │ │ +╰──────────┴──────────────────────────────────────┴────────╯ +``` + +If the `active` network is not `testnet`, switch to `testnet` by running: + +``` sh +sui client switch --envs testnet +``` + +Further, make sure you have at least 2 separate gas coins, with at least 1 SUI each, by running `sui +client gas`. If you don't have enough SUI, you can hit the testnet faucet by running. + +``` sh +sui client faucet --url https://faucet.testnet.sui.io/v1/gas +``` + +After some seconds, running again `sui client gas` should show the newly-minted coins in your +wallet. + +## Clone the Walrus Sites repo, and build the `site-builder` tool + +First clone and enter the Walrus Sites repo from +`https://github.com/MystenLabs/blocksite-poc`). (TODO: change link to public repo when available). + +``` sh +git clone git@github.com:MystenLabs/blocksite-poc.git +cd blocksite-poc +cd site-builder +``` + +Build the release version of the site builder. + +``` sh +cargo build --release +``` + +After the build process completes, it should be possible to run: + +``` sh +./target/release/site-builder +``` + +And output should look like the following: + +``` txt +Usage: site-builder [OPTIONS] + +Commands: + publish Publish a new site on Sui + update Update an existing site + convert Convert an object ID in hex format to the equivalent Base36 + format + sitemap Show the pages composing the Walrus site at the given object ID + help Print this message or the help of the given subcommand(s) + +⋮ +``` + +## Get the latest `walrus` client configuration + +First, +[download](https://storage.googleapis.com/mysten-walrus-binaries/walrus-configs/client_config.yaml) +the `walrus` client config. Then, copy it to `~/.walrus/config.yaml`. This ensures that the +`walrus` binary can connect to the correct Walrus object on Sui. diff --git a/docs/walrus-sites/tutorial-publish.md b/docs/walrus-sites/tutorial-publish.md new file mode 100644 index 00000000..2aae9893 --- /dev/null +++ b/docs/walrus-sites/tutorial-publish.md @@ -0,0 +1,96 @@ +# Publishing a Walrus Site + +Now that everything is installed and configured, you should be able to start publishing +your first Walrus Site! + +## Select the source material for the site + +The `site-builder` works by uploading a directory of files produced by any web framework to Walrus, +and adding the relevant metadata to Sui. This directory should have a file called `index.html` in +its root, which will be the entry point to the Walrus Site. + +For the rest of the tutorial, we will use as an example the simple site contained in +`./examples/snake`. + +## Publish the site + +Since we have placed the `walrus` binary and configuration in their default locations, publishing +the `./examples/snake` site is simple: + +- Ensure that you are in the `site-builder` directory; +- Run the publishing command: + + ``` sh + ./target/release/site-builder --config assets/builder-example.yaml publish ../examples/snake + ``` + +The output should look like the following: + +``` txt +Operations performed: +- created resource /Oi-Regular.ttf with blob ID 2YLU3Usb-WoJAgoNSZUNAFnmyo8cfV8hJYt2YdHL2Hs +- created resource /file.png with blob ID R584P82qm4Dn8LoQMlzkGZS9IAkU0lNZTVlruOsUyOs +- created resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic +- created resource /walrus.png with blob ID SGrrw5NQyFWtqtxzLAQ1tLpcChGc0VNbtFRhfsQPuiM + +Created new site: test site +New site object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e + +Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site +``` + +This output tells you that, for each file in the folder, a new Walrus blob was created, and the +respective blob ID. Further it prints the object ID of the Walrus Site object on Sui (so you can +have a look in the explorer, and use it to set the SuiNS name), and, finally, the URL at which you +can browse the site. + +Note here that we are passing the example config `assets/builder-example.yaml` as the config for the +site builder. The configuration file is necessary to ensure that the `site-builder` knows the +correct Sui package for the Walrus Sites logic. + +More details on the configuration of the `site-builder` can be found under the [advanced +configuration](tutorial-config.md) section. + +## Update the site + +Let's say now you want to update the content of the site, for example by changing the title from +"eat all the blobs!" to "Glob all the Blobs!". + +First, make this edit on in the `../examples/snake/index.html` file. + +Then, you can update the existing site by running the `update` command, and providing the directory +where to find the updated files (still `../example/snake`), and the object ID of the existing site +(`0x5ac988...`): + +``` sh +./target/release/site-builder --config assets/builder-example.yaml update ../examples/snake 0x5ac9888... +``` + +The output this time should be: + +``` txt +Operations performed: + - deleted resource /index.html with blob ID SSzbpPfO2Tqk6xNyF1i-NG9I9CjUjuWnhUATVSs5nic + - created resource /index.html with blob ID LXtY0VdY5kM-3Ph7gLvj8URdz5yiRa5DUy3ZxYqDView + +Updated site at object ID: 0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e + +Browse the resulting site at: https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site +``` + +Compared to the `publish` action, we can see that now the only actions performed were to delete the +old `index.html`, and update it with the newer one. + +Browsing to the provided URL should reflect the change. You've updated the site! + +## Additional commands + +The `site-builder` tool provides two additional utilities: + +- the `convert` command, which converts an object ID in hex format to the equivalent Base36 + format. This command is useful if you have the Sui object ID of a Walrus Site, and want to know + the subdomain where you can browse it. +- the `sitemap` command, which shows the resources that compose the Walrus Site at the given object + ID. + +In general, the `--help` flag is your friend! diff --git a/docs/walrus-sites/tutorial-suins.md b/docs/walrus-sites/tutorial-suins.md new file mode 100644 index 00000000..79f4eca7 --- /dev/null +++ b/docs/walrus-sites/tutorial-suins.md @@ -0,0 +1,61 @@ +# Bonus: Set a SuiNS name + +Browsing a URL like `https://29gjzk8yjl1v7zm2etee1siyzaqfj9jaru5ufs6yyh1yqsgun2.walrus.site` is not +particularly nice. Therefore, Walrus Sites allows to use SuiNS names (this is like DNS for Sui) to +give human-readable names to site. To do so, you simply have to get a SuiNS name you like, and point +it to the object ID of the Walrus Site (as provided by the `publish` or `update` commands). + +Let's do this step by step. + +## Get a SuiNS name + +- Navigate to [https://testnet.suins.io/](https://testnet.suins.io/), and buy a domain name with + your testnet wallet. For example, `walrusgame` (NOTE: this is already taken, choose another you + like!). NOTE: At the moment, you can only select names that are composed of letters `a-z` and + numbers `0-9`, but no special characters (e.g., `-`). +- In the [page](https://testnet.suins.io/account/my-names) listing the domains you own, you should + see the newly-bought name. +- Click the three-dots menu on the top-right corner of the name you want to assign. Choose "View all + info", and copy the `ObjectID`. In our case, this is + `0x6412c4cfbe50e219c2d4d30108d7321d064e15bf64e752307100bff5eb91da38`. + +## Send the SuiNS registration object to the address you use with the Sui CLI + +The steps that follow require that the SuiNS registration object is owned by the address you are +using on the Sui CLI. Therefore, we need to send this registration object from the address you use +in your browser wallet, to the address of your Sui CLI. + +To find the Sui CLI address, execute: + +``` sh +sui client active-address +``` + +Then, from your browser wallet, select the "Assets" tab, and look for the NFT of the SuiNS +registration, which should look as follows: + +![the SuiNS registration inside the wallet](../assets/suins-asset.png) + +Click on it, scroll down to "Send NFT", and send it to the address discovered with the command +above. Now, your Sui CLI address owns the registration NFT, and you can proceed to the next step. + +## Map the SuiNS name to the Walrus Site + +This step associates the name `walrusgame` to the object ID of our Walrus Site. There are possibly +many ways to achieve this, and as the SuiNS UI improves this could be done from the webapp as well. + +Here, we issue an transaction using the Sui CLI that creates this mapping. + +``` sh + sui client call \ + --package 0x22fa05f21b1ad71442491220bb9338f7b7095fe35000ef88d5400d28523bdd93 \ + --module controller \ + --function set_target_address \ + --gas-budget 500000000 \ + --args 0x300369e8909b9a6464da265b9a5a9ab6fe2158a040e84e808628cde7a07ee5a3 \ + --args 0x6412c4cfbe50e219c2d4d30108d7321d064e15bf64e752307100bff5eb91da38 \ + --args "[0x5ac988828a0c9842d91e6d5bdd9552ec9fcdddf11c56bf82dff6d5566685a31e]" \ + --args 0x6 +``` + +If all succeeds, we can now browse [https://walrusgame.walrus.site](https://walrusgame.walrus.site)! diff --git a/docs/walrus-sites/tutorial.md b/docs/walrus-sites/tutorial.md new file mode 100644 index 00000000..de258b72 --- /dev/null +++ b/docs/walrus-sites/tutorial.md @@ -0,0 +1,4 @@ +# Your first Walrus Site + +This tutorial walks you through the steps necessary to publish a Walrus Site. We also provide the +instructions on how to add a SuiNS name to it for convenient browsing.