Skip to content

Commit

Permalink
bump v0.0.9
Browse files Browse the repository at this point in the history
  • Loading branch information
hard-nett committed Dec 28, 2024
1 parent cfa9e7b commit e624141
Show file tree
Hide file tree
Showing 15 changed files with 288 additions and 260 deletions.
53 changes: 7 additions & 46 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,12 +24,12 @@ O-line has a generic base image which can download the required binary at runtim

This generic image provides the O-line scripts and configuration helpers, and nothing else. Set the `BINARY_URL` environment variable to a `.zip`, `.tar` or `.tar.gz` URL, and configure `PROJECT`, `PROJECT_DIR` and `PROJECT_BIN`. Alternatively provide a [Chain Registry](https://github.com/cosmos/chain-registry) `CHAIN_JSON` to configure everything automatically (where data is available).

Image URL: `ghcr.io/terp-network/o-line:v0.0.8-generic`
Image URL: `ghcr.io/terp-network/o-line:v0.0.9-generic`

```yaml
services:
node:
image: ghcr.io/terpnetwork/o-line:v0.0.8-generic
image: ghcr.io/terpnetwork/o-line:v0.0.9-generic
env:
- MONIKER=my-moniker-1
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
Expand All @@ -45,7 +45,7 @@ tagged with the form `$COSMOS_OMNIBUS_VERSION-$PROJECT-$PROJECT_VERSION`.

|Project|Version|Image| |
|---|---|---|---|
|[terp-network](https://github.com/terpnetwork/terp-core)|`v4.2.2`|`terpnetwork/o-line:v0.0.8`|[Example](./terpnetwork)|
|[terp-network](https://github.com/terpnetwork/terp-core)|`v4.2.2`|`terpnetwork/o-line:v0.0.9`|[Example](./terpnetwork)|

## Example configurations

Expand Down Expand Up @@ -89,7 +89,7 @@ TERP_P2P_SEEDS=id@node:26656

<!-- The namespace for each of the supported chains in the cosmos omnibus can be found in the `README` in each project directory. -->

The omnibus images allow some specific variables and shortcuts to configure extra functionality, detailed below.
The o-line images allow some specific variables and shortcuts to configure extra functionality, detailed below.

### Chain configuration

Expand All @@ -108,7 +108,7 @@ Chain config can be sourced from a `chain.json` file [as seen in the Cosmos Chai
Peer information can be provided manually, or obtained automatically from the following sources:

- `CHAIN_JSON` URL with peer information included.
- [Polkachu's live peers](https://www.polkachu.com/live_peers).
<!-- - [Polkachu's live peers](https://www.polkachu.com/live_peers). -->
- Any `ADDRBOOK_URL` where the `config/addrbook.json` file is hosted.

See [Cosmos docs](https://docs.tendermint.com/master/nodes/configuration.html#p2p-settings) for more information.
Expand All @@ -119,7 +119,7 @@ See [Cosmos docs](https://docs.tendermint.com/master/nodes/configuration.html#p2
|`P2P_PERSISTENT_PEERS`|Persistent peers. Can be set by CHAIN_JSON or GENESIS_URL| |`id@node:26656`|
|`ADDRBOOK_URL`|URL to an addrbook.json file| |`https://server-3.itrocket.net/mainnet/terp/addrbook.json`

### Private key backup/restore
<!-- ### Private key backup/restore

On boot, the container can restore a private key from any S3 storage provider. If this is configured and the key doesn't exist in S3 yet, it will be uploaded from the node.

Expand All @@ -136,7 +136,7 @@ The `node_key.json` and `priv_validator_key.json` are both backed up, and can be
|`S3_HOST`|The S3 API host|`https://s3.filebase.com`|`https://s3.us-east-1.amazonaws.com`|
|`STORJ_ACCESS_GRANT`|DCS Storj Access Grant token (replaces `S3_KEY`, `S3_SECRET`, `S3_HOST`| | |
|`KEY_PATH`|Bucket and directory to backup/restore to| |`bucket/nodes/node_1`|
|`KEY_PASSWORD`|An optional password to encrypt your private keys. Shouldn't be optional| | |
|`KEY_PASSWORD`|An optional password to encrypt your private keys. Shouldn't be optional| | | -->

### Statesync

Expand All @@ -156,48 +156,9 @@ Some shortcuts for enabling statesync. Statesync requires 2x nodes with snapshot
|`STATESYNC_TRUST_HEIGHT`|Obtained from `STATESYNC_TRUSTED_NODE`| | |
|`STATESYNC_TRUST_HASH`|Obtained from `STATESYNC_TRUSTED_NODE`| | |

### Snapshot restore

The node `data` directory can be restored from a `.tar`, `.tar.gz` or `.lz4` file stored on a public URL.
The file can be obtained from the following sources:

- Direct URL to the archive file
- Base URL file listing, where the archive matches a given pattern.
- [snapshot.json](#snapshot-backup) generated by [O-Line Snapshot backup](#snapshot-backup) feature.
<!-- - ChainLayer's [Quicksync snapshots](https://quicksync.io/) described by a JSON file.
- Polkachu's [snapshot service](https://www.polkachu.com/tendermint_snapshots), fully automatically. -->

Note that snapshots will be restored in-process, without downloading the snapshot to disk first. This saves disk space but is slower to extract, and could be made configurable in the future.

|Variable|Description|Default|Examples|
|---|---|---|---|
|`DOWNLOAD_SNAPSHOT`|Force bootstrapping from snapshot. If unset the node will only restore a snapshot if the `data` contents are missing| |`1`|
|`SNAPSHOT_URL`|A URL to a `.tar`, `.tar.gz` or `.lz4` file| |`http://135.181.60.250/akash/akashnet-2_2021-06-16.tar`|
|`SNAPSHOT_BASE_URL`|A base URL to a directory containing backup files| |`http://135.181.60.250/akash`|
|`SNAPSHOT_JSON`|A URL to a `snapshot.json` as detailed in [Snapshot backup](#snapshot-backup)| |`https://cosmos-snapshots.s3.filebase.com/akash/pruned/snapshot.json`|
|`SNAPSHOT_FORMAT`|The format of the snapshot file|`tar.gz`|`tar`/`tar.zst`|
|`SNAPSHOT_PATTERN`|The pattern of the file in the `SNAPSHOT_BASE_URL`|`$CHAIN_ID.*$SNAPSHOT_FORMAT`|`foobar.*tar.gz`|
|`SNAPSHOT_DATA_PATH`|The path to the data directory within the archive| |`snapshot/data`|
|`SNAPSHOT_WASM_PATH`|The path to the wasm directory within the archive, if exists outside of data| |`snapshot/wasm`|
|`SNAPSHOT_PRUNING`|Type of snapshot to download, e.g. `archive`, `pruned`, `default`.|`pruned`|`archive`|
|`SNAPSHOT_QUICKSYNC`|A URL to a Quicksync JSON file describing their snapshots. Also see `SNAPSHOT_PRUNING`| |`https://quicksync.io/terra.json`|

### Snapshot backup

O-Line includes a script to automatically snapshot a node and upload the resulting archive to any S3 compatible service like [Filebase](https://filebase.com/).
At a specified time (or day), the script will shut down the tendermint server, create an archive of the `data` directory and upload it.
Snapshots older than a specified time can also be deleted. Finally a JSON metadata file is created listing the current snapshots. The server is then restarted and monitored.

[See an example](_examples/snapshot_backup) of a snapshot node deployment.

|Variable|Description|Default|Examples|
|---|---|---|---|
|`S3_KEY`|S3 access key| | |
|`S3_SECRET`|S3 secret key| | |
|`S3_HOST`|The S3 API host|`https://s3.filebase.com`|`s3.us-east-1.amazonaws.com`|
|`STORJ_ACCESS_GRANT`|DCS Storj Access Grant token (replaces `S3_KEY`, `S3_SECRET`, `S3_HOST`)| | |
|`STORJ_UPLINK_ARGS`|DCS Storj Uplink arguments|`-p 4 --progress=false`|`-p 4 --parallelism-chunk-size 256M --progress=false`|
|`SNAPSHOT_PATH`|The S3 path to upload snapshots to, including the bucket| |`cosmos-snapshots/akash`|
|`SNAPSHOT_PREFIX`|The prefix for the snapshot filename|`$CHAIN_ID`|`snapshot`|
|`SNAPSHOT_TIME`|The time the snapshot will run|`00:00:00`|`09:00:00`|
|`SNAPSHOT_DAY`|The numeric day of the week the snapshot will run (Monday = 1)|`*`|`7`|
Expand Down
9 changes: 4 additions & 5 deletions _examples/load-balanced-rpc-nodes/node_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,13 @@ version: "2.0"

services:
node:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=my-moniker-1
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
# - P2P_POLKACHU=1
# - STATESYNC_POLKACHU=1
- STATESYNC_RPC_SERVERS="<node1>,node2"
expose:
- port: 26657
to:
Expand Down
41 changes: 40 additions & 1 deletion _examples/snapshot_backup/README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,44 @@
# Snapshot backup

The snapshot script will shutdown the node for as long as the archive and upload process takes,
so use a dedicated node for creating snapshots. [See the README](/README.md#snapshot-restore) for the configuration options available.
so use a dedicated node for creating snapshots.



### Snapshot restore

The node `data` directory can be restored from a `.tar`, `.tar.gz` or `.lz4` file stored on a public URL.
The file can be obtained from the following sources:

- Direct URL to the archive file
- Base URL file listing, where the archive matches a given pattern.
- [snapshot.json](#snapshot-backup) generated by [O-Line Snapshot backup](#snapshot-backup) feature.
<!-- - ChainLayer's [Quicksync snapshots](https://quicksync.io/) described by a JSON file.
- Polkachu's [snapshot service](https://www.polkachu.com/tendermint_snapshots), fully automatically. -->

Note that snapshots will be restored in-process, without downloading the snapshot to disk first. This saves disk space but is slower to extract, and could be made configurable in the future.

|Variable|Description|Default|Examples|
|---|---|---|---|
|`DOWNLOAD_SNAPSHOT`|Force bootstrapping from snapshot. If unset the node will only restore a snapshot if the `data` contents are missing| |`1`|
|`SNAPSHOT_URL`|A URL to a `.tar`, `.tar.gz` or `.lz4` file| |`http://135.181.60.250/akash/akashnet-2_2021-06-16.tar`|
|`SNAPSHOT_BASE_URL`|A base URL to a directory containing backup files| |`http://135.181.60.250/akash`|
|`SNAPSHOT_JSON`|A URL to a `snapshot.json` as detailed in [Snapshot backup](#snapshot-backup)| |`https://cosmos-snapshots.s3.filebase.com/akash/pruned/snapshot.json`|
|`SNAPSHOT_FORMAT`|The format of the snapshot file|`tar.gz`|`tar`/`tar.zst`|
|`SNAPSHOT_PATTERN`|The pattern of the file in the `SNAPSHOT_BASE_URL`|`$CHAIN_ID.*$SNAPSHOT_FORMAT`|`foobar.*tar.gz`|
|`SNAPSHOT_DATA_PATH`|The path to the data directory within the archive| |`snapshot/data`|
|`SNAPSHOT_WASM_PATH`|The path to the wasm directory within the archive, if exists outside of data| |`snapshot/wasm`|
|`SNAPSHOT_PRUNING`|Type of snapshot to download, e.g. `archive`, `pruned`, `default`.|`pruned`|`archive`|
<!-- |`SNAPSHOT_QUICKSYNC`|A URL to a Quicksync JSON file describing their snapshots. Also see `SNAPSHOT_PRUNING`| |`https://quicksync.io/terra.json`| -->

### Snapshot backup

O-Line includes a script to automatically snapshot a node and upload the resulting archive to any S3 compatible service like [Filebase](https://filebase.com/).
At a specified time (or day), the script will shut down the tendermint server, create an archive of the `data` directory and upload it.
Snapshots older than a specified time can also be deleted. Finally a JSON metadata file is created listing the current snapshots. The server is then restarted and monitored.

#### Using Caddy

#### Using Rsync

[See an example](_examples/snapshot_backup) of a snapshot node deployment.
11 changes: 4 additions & 7 deletions _examples/snapshot_backup/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,21 +3,18 @@ version: "2.0"

services:
node:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=my-moniker-1
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
# - P2P_POLKACHU=1
# - STATESYNC_POLKACHU=1
- S3_KEY=<s3-key>
- S3_SECRET=<s3-secret>
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
- SNAPSHOT_PATH=<bucket/path>
- SNAPSHOT_TIME=00:00:00
# - SNAPSHOT_DAY=* #(1-7)
- SNAPSHOT_SIZE=214748364800 # 200GB in bytes
- SNAPSHOT_METADATA_URL=https://<bucket>.s3.filebase.com/<path>
expose:
- port: 26657
to:
Expand Down
16 changes: 8 additions & 8 deletions _examples/statesync/snapshot-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,14 +3,14 @@ version: "2.0"

services:
node1:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=public-node-1
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
# - P2P_POLKACHU=1
# - STATESYNC_POLKACHU=1
- PRUNING=nothing
- STATESYNC_SNAPSHOT_INTERVAL=500
expose:
Expand All @@ -26,14 +26,14 @@ services:
# data:
# mount: /root/.akash
node2:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=public-node-2
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
# - P2P_POLKACHU=1
# - STATESYNC_POLKACHU=1
- PRUNING=nothing
- STATESYNC_SNAPSHOT_INTERVAL=500
expose:
Expand Down
6 changes: 3 additions & 3 deletions _examples/statesync/statesync-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,11 +3,11 @@ version: "2.0"

services:
node1:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=private_node_1
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
expose:
Expand Down
54 changes: 21 additions & 33 deletions _examples/validator-and-private-sentries/deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,18 @@ version: "2.0"

services:
validator:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=validator
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- AKASH_P2P_PEX=false
- AKASH_P2P_UNCONDITIONAL_PEER_IDS=<node-1-id>,<node-2-id>...
- AKASH_P2P_ADDR_BOOK_STRICT=false
- AKASH_CONSENSUS_DOUBLE_SIGN_CHECK_HEIGHT=10
- TERPNETWORK_P2P_PEX=false
- TERPNETWORK_P2P_UNCONDITIONAL_PEER_IDS=<node-1-id>,<node-2-id>...
- TERPNETWORK_P2P_ADDR_BOOK_STRICT=false
- TERPNETWORK_CONSENSUS_DOUBLE_SIGN_CHECK_HEIGHT=10
- P2P_PERSISTENT_PEERS=<id@node1:port,id@node2:port>
- STATESYNC_RPC_SERVERS=<node1:80>,<node2:80>
- S3_KEY=<s3-key>
- S3_SECRET=<s3-secret>
- KEY_PASSWORD=<key password>
- KEY_PATH=<bucket/validator>
depends_on:
- service: node1
- service: node2
Expand All @@ -38,22 +34,18 @@ services:
# data:
# mount: /root/.akash
node1:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=private_node_1
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- AKASH_P2P_PEX=true
- AKASH_P2P_PRIVATE_PEER_IDS=<validatorid>
- AKASH_P2P_UNCONDITIONAL_PEER_IDS=<validatorid>
- AKASH_P2P_ADDR_BOOK_STRICT=false
- TERPNETWORK_P2P_PEX=true
- TERPNETWORK_P2P_PRIVATE_PEER_IDS=<validatorid>
- TERPNETWORK_P2P_UNCONDITIONAL_PEER_IDS=<validatorid>
- TERPNETWORK_P2P_ADDR_BOOK_STRICT=false
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
- STATESYNC_SNAPSHOT_INTERVAL=500
- S3_KEY=<s3-key>
- S3_SECRET=<s3-secret>
- KEY_PASSWORD=<key password>
- KEY_PATH=<bucket/node1>
expose:
- port: 26657
as: 80
Expand All @@ -68,22 +60,18 @@ services:
# data:
# mount: /root/.akash
node2:
image: ghcr.io/terpnetwork/o-line:v0.0.8-terpnetwork-v4.2.2
image: ghcr.io/terpnetwork/o-line:v0.0.9-terpnetwork-v4.2.2
env:
- MONIKER=private_node_2
- CHAIN_JSON=https://raw.githubusercontent.com/akash-network/net/main/mainnet/meta.json
- MINIMUM_GAS_PRICES=0.025uakt
- CHAIN_JSON=https://raw.githubusercontent.com/cosmos/chain-registry/refs/heads/master/terpnetwork/chain.json
- MINIMUM_GAS_PRICES=0.05uthiol
- FASTSYNC_VERSION=v0
- AKASH_P2P_PEX=true
- AKASH_P2P_PRIVATE_PEER_IDS=<validatorid>
- AKASH_P2P_UNCONDITIONAL_PEER_IDS=<validatorid>
- AKASH_P2P_ADDR_BOOK_STRICT=false
- TERPNETWORK_P2P_PEX=true
- TERPNETWORK_P2P_PRIVATE_PEER_IDS=<validatorid>
- TERPNETWORK_P2P_UNCONDITIONAL_PEER_IDS=<validatorid>
- TERPNETWORK_P2P_ADDR_BOOK_STRICT=false
- STATESYNC_RPC_SERVERS=<publicnode1:26657,publicnode2:26657>
- STATESYNC_SNAPSHOT_INTERVAL=500
- S3_KEY=<s3-key>
- S3_SECRET=<s3-secret>
- KEY_PASSWORD=<key password>
- KEY_PATH=<bucket/node2>
expose:
- port: 26657
as: 80
Expand Down
2 changes: 0 additions & 2 deletions _examples/validator-and-public-sentries/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
# Validator and Public Sentries

> You can replace Akash with any other chain supported by the cosmos-omnibus project https://github.com/terpnetwork/o-line or is available via meta.json (see https://github.com/cosmos/chain-registry )
Sentry node is the internet facing node which serves the blocks to the validator as well as distributes them from the validator when validator signs them upon its turn in the tendermint consensus.

Validator does not announce its existence to the network (`pex=false` /& `private_peer_ids`) and gets connected only to the sentry nodes.
Expand Down
Loading

0 comments on commit e624141

Please sign in to comment.