Skip to content

Commit

Permalink
penumbra support (#426)
Browse files Browse the repository at this point in the history
  • Loading branch information
agouin committed Jun 19, 2024
1 parent 3c25c3d commit 9d27de9
Show file tree
Hide file tree
Showing 4 changed files with 146 additions and 5 deletions.
10 changes: 7 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
[![License: Apache-2.0](https://img.shields.io/github/license/strangelove-ventures/cosmos-operator.svg?style=flat-square)](https://github.com/strangelove-ventures/cosmos-operator/blob/main/LICENSE)
[![Version](https://img.shields.io/github/tag/strangelove-ventures/cosmos-operator.svg?style=flat-square)](https://github.com/cosmos/strangelove-ventures/cosmos-operator)

Cosmos Operator is a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) for blockchains built with the [Cosmos SDK](https://github.com/cosmos/cosmos-sdk).
Cosmos Operator is a [Kubernetes Operator](https://kubernetes.io/docs/concepts/extend-kubernetes/operator/) primarily for blockchains built with the [Cosmos SDK](https://github.com/cosmos/cosmos-sdk). It also supports [Penumbra](https://github.com/penumbra-zone/penumbra) and other chains which use [CometBFT](https://github.com/cometbft/cometbft) for consensus.

The long-term vision of this operator is to allow you to "configure it and forget it".

Expand Down Expand Up @@ -39,12 +39,16 @@ The CosmosFullNode controller is like a StatefulSet for running Cosmos SDK block
A CosmosFullNode can be configured to run as an RPC node, a validator sentry, or a seed node. All configurations can
be used as persistent peers.

As of this writing, Strangelove has been running CosmosFullNode in production for many months.
As of this writing, Strangelove has been running CosmosFullNode in production for over a year.

## Samples

[Minimal example yaml](./config/samples/cosmos_v1_cosmosfullnode.yaml)

[Full example yaml](./config/samples/cosmos_v1_cosmosfullnode_full.yaml)

[Penumbra example yaml](./config/samples/cosmos_v1_cosmosfullnode_penumbra.yaml)

### Why not a StatefulSet?
Each pod requires different config, such as peer settings in config.toml and mounted node keys. Therefore, a blanket
template as found in StatefulSet did not suffice.
Expand Down Expand Up @@ -75,7 +79,7 @@ See the [best practices guide for CosmosFullNode](./docs/fullnode_best_practices

Disclaimer: Strangelove has not committed to these enhancements and cannot estimate when they will be completed.

- [ ] Scheduled upgrades. Set a halt height and image version. The controller performs a rolling update with the new image version after the committed halt height.
- [x] Scheduled upgrades. Set the upgrade height and image version, optionally setting halt height. The controller performs a rolling update with the new image version after the committed height.
- [x] Support configuration suitable for validator sentries.
- [x] Reliable, persistent peer support.
- [x] Quicker p2p discovery using private peers.
Expand Down
2 changes: 2 additions & 0 deletions api/v1/cosmosfullnode_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -436,6 +436,8 @@ type ChainSpec struct {
Comet CometConfig `json:"config"`

// App configuration applied to app.toml.
// Although optional, it's highly recommended you configure this field.
// +optional
App SDKAppConfig `json:"app"`

// One of trace|debug|info|warn|error|fatal|panic.
Expand Down
5 changes: 3 additions & 2 deletions config/crd/bases/cosmos.strange.love_cosmosfullnodes.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,9 @@ spec:
Use AddrbookScript if the chain has an unconventional file format or address book location.
type: string
app:
description: App configuration applied to app.toml.
description: |-
App configuration applied to app.toml.
Although optional, it's highly recommended you configure this field.
properties:
apiEnableUnsafeCORS:
description: Defines if CORS should be enabled for the API
Expand Down Expand Up @@ -383,7 +385,6 @@ spec:
type: object
type: array
required:
- app
- binary
- chainID
- network
Expand Down
134 changes: 134 additions & 0 deletions config/samples/cosmos_v1_cosmosfullnode_penumbra.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
# Example Penumbra Sentry configuration
# 3 sentries, 1 per worker node
apiVersion: cosmos.strange.love/v1
kind: CosmosFullNode
metadata:
name: penumbra-testnet-sentry
spec:
type: Sentry
replicas: 3
chain:
chainID: penumbra-testnet-deimos-8
network: testnet
versions:
# Genesis version
- height: 0
image: ghcr.io/strangelove-ventures/heighliner/cometbft:v0.37.5
containers:
pd: ghcr.io/strangelove-ventures/heighliner/penumbra:v0.77.2
# Stage upgrades at future block heights by adding the version information here
config:
maxInboundPeers: 20
maxOutboundPeers: 20
overrides: |-
moniker = "strangelove"
fast_sync = true
[consensus]
timeout_commit = "5000ms"
[fastsync]
version = "v0"
seeds: [email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:26656,[email protected]:42656
genesisScript: echo "No need for genesis"
addrbookScript: echo "No need for address book"
skipInvariants: false
binary: cometbft

podTemplate:
affinity:
podAntiAffinity:
# One sentry per node
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchLabels:
app.kubernetes.io/name: penumbra-testnet-sentry
topologyKey: kubernetes.io/hostname
initContainers:
# Initialize the cometbft tmp configuration for the correct version, used for config overlay
- name: chain-init
command:
- cometbft
- init
- --home /home/operator/.tmp
volumeMounts:
- mountPath: /home/operator/.tmp
name: vol-tmp
env:
- name: HOME
value: /home/operator/cosmos

# Initialize the cometbft and pd data from a snapshot if it is not already initialized
- name: pd-init
image: ghcr.io/strangelove-ventures/heighliner/penumbra:v0.77.2
command:
- sh
- -c
- |
HOME=/home/operator/cosmos pd testnet join --archive-url "https://snapshots.penumbra.zone/testnet/pd-migrated-state-76-77.tar.gz" || echo "Already initialized"
volumeMounts:
- mountPath: /home/operator/cosmos
name: vol-chain-home

# If pd-init initialized the data, move the cometbft data to the operator expected location and
# move the pd data also to the root of the vol-chain-home volume so that it is included in snapshots
- name: pd-move
image: busybox:latest
command:
- sh
- -c
- |
HOME=/home/operator/cosmos
NODE0=$HOME/.penumbra/testnet_data/node0
mv $NODE0/cometbft/* $HOME/ || echo "Already moved cometbft data"
mv $NODE0/pd $HOME/ || echo "Already moved pd data"
volumeMounts:
- mountPath: /home/operator/cosmos
name: vol-chain-home
containers:
# pd sidecar container, communicates with cometbft "node" container
- name: pd
command:
- sh
- -c
- |
pd start --home /home/operator/cosmos/pd
ports:
# CometBFT connects to PD on port 26658
- containerPort: 26658
name: abci
protocol: TCP
- containerPort: 8080
name: grpc
protocol: TCP
volumeMounts:
- mountPath: /home/operator/cosmos
name: vol-chain-home
probes:
# Disable liveness and readiness probes for sentries
strategy: None
resources:
requests:
cpu: 4000m
memory: 10Gi
limits:
memory: 64Gi
selfHeal:
heightDriftMitigation:
threshold: 10
pvcAutoScale:
increaseQuantity: 10%
maxSize: 5Ti
usedSpacePercentage: 75
service:
# Create a kubernetes NodePort service for the P2P port on each sentry
maxP2PExternalAddresses: 3
p2pTemplate:
type: NodePort
volumeClaimTemplate:
resources:
requests:
# Initial storage size for the PVC, needs to be enough for snapshot restore
storage: 20Gi
# Storage class for the PVC, will use default storage class if not specified
storageClassName: topolvm-provisioner-thin-striped

0 comments on commit 9d27de9

Please sign in to comment.