feature(charts): hermes chart fixes, bech32 updates, ibc bridge test (#…

…1130)

## Summary
Fixes hermes helm chart

## Background
We want to run hermes via helm chart so we can automate e2e testing.

## Changes
* add 2nd funded celestia account
* use this account's key in hermes
* fix hermes chart values
* add new chart `bridge-test`
* add new justfile `dev/bridgetester.just`
* add `AstriaSequencerHrpPrefix` to evm rollup geth-genesis

## Testing
- `just deploy-cluster`
- `just deploy-ibc-test-infra`
  - deploys ingress controller
  - deploys celestia-local
  - deploys single astria sequencer
    - `./charts/sequencer/values.yaml`
    - `./dev/values/validators/single.yaml`
  - deploys evm rollup
    - `./charts/evm-rollup/values.yaml`
    - `./dev/values/rollup/dev.yaml`
    - `./dev/values/rollup/ibc-bridge-test.yaml`
  - deploy hermes
    - `./charts/hermes/values.yaml`
    - `./dev/values/hermes/local.yml`
- `just deploy-bridge-tester`
  - initContainer handles initializing bridge account on the sequencer
  - container runs scripts
    - `celestia-appd keys add`
    - `celestia-appd tx ibc-transfer transfer`
- checks balance before and after and fails if ibc-transfer did not go
through


## Metrics
- List out metrics added by PR, delete section if none. 

## Breaking Changelist
- Bulleted list of breaking changes, any notes on migration. Delete
section if none.

## Related Issues
Link any issues that are related, prefer full github links.

closes <!-- list any issues closed here -->

.github/workflows/docker-build.yml

@@ -26,7 +26,7 @@ on:
       - "**-v[0-9]+.[0-9]+.[0-9]+-alpha.[0-9]+"
       - "**-v[0-9]+.[0-9]+.[0-9]+-beta.[0-9]+"
       - "**-v[0-9]+.[0-9]+.[0-9]+-rc[0-9]+"
-    
+
   pull_request:
     types:
       - opened
@@ -179,6 +179,52 @@ jobs:
         timeout-minutes: 3
         run: just run-smoke-cli
 
+  ibc-bridge-test:
+    needs: [ run_checker, composer, conductor, sequencer, sequencer-relayer ]
+    if: (github.event_name != 'pull_request' || github.event.pull_request.head.repo.full_name == 'astriaorg/astria') && (github.event_name == 'merge_group' || needs.run_checker.outputs.run_docker == 'true')
+    runs-on: buildjet-8vcpu-ubuntu-2204
+    steps:
+      - uses: actions/checkout@v4
+      - name: Install just
+        uses: taiki-e/install-action@just
+      - name: Install kind
+        uses: helm/kind-action@v1
+        with:
+          install_only: true
+      - name: Log in to GHCR
+        uses: docker/login-action@v2
+        with:
+          registry: ghcr.io
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Setup IBC Bridge Test Environment
+        timeout-minutes: 5
+        run: |
+          TAG=sha-$(git rev-parse --short HEAD)
+          just deploy cluster
+          kubectl create secret generic regcred --from-file=.dockerconfigjson=$HOME/.docker/config.json --type=kubernetes.io/dockerconfigjson
+          echo -e "\n\nDeploying with astria images tagged $TAG"
+          just deploy ibc-test-infra $TAG
+          just build-and-load-bridge-tester-image $TAG
+      - name: Run IBC Bridge Tester
+        timeout-minutes: 2
+        run: |
+          TAG=sha-$(git rev-parse --short HEAD)
+          printlogs() {
+            echo "IBC Transfer Test failed, printing logs..."
+            kubectl describe job bridge-tester-chart -n astria-dev-cluster
+            kubectl logs job/bridge-tester-chart --all-containers -n astria-dev-cluster
+            exit 1
+          }
+          just deploy bridge-tester $TAG
+          # timeout before the gh job times out so we can print logs
+          kubectl wait --for=condition=complete --timeout=90s job/bridge-tester-chart -n astria-dev-cluster || printlogs
+          JOB_STATUS=$(kubectl get job bridge-tester-chart -n astria-dev-cluster -o jsonpath='{.status.succeeded}')
+          if [ "$JOB_STATUS" != "1" ]; then
+            printlogs
+          else
+            echo "IBC Transfer Test passed"
+          fi
 
   docker:
     if: ${{ always() && !cancelled() }}

charts/README.md

@@ -148,7 +148,7 @@ to `local`. You can now deploy the chart with your local image.
 
 ### Dev vs Prod
 
-All of our charts should run against both the lastest code in monorepo AND
+All of our charts should run against both the latest code in monorepo AND
 against the latest release. Sometimes, there are configuration changes between
 releases though. To manage this in various templates you will see the following
 pattern (especially in config maps and genesis files):
@@ -183,8 +183,8 @@ To deploy and run this:
 ## Examining Deployments
 
 [k9s](https://k9scli.io/) is a useful utility for inspecting deployed
-containers, logs and services. Additionally you may interact directly with the
-kubernetes API some helpful commands below.
+containers, logs and services. Additionally, you may interact directly with the
+kubernetes API using some helpful commands below.
 
 ### Helpful commands
 

charts/bridge-test/.helmignore

@@ -0,0 +1,23 @@
+# Patterns to ignore when building packages.
+# This supports shell glob matching, relative path matching, and
+# negation (prefixed with !). Only one pattern per line.
+.DS_Store
+# Common VCS dirs
+.git/
+.gitignore
+.bzr/
+.bzrignore
+.hg/
+.hgignore
+.svn/
+# Common backup files
+*.swp
+*.bak
+*.tmp
+*.orig
+*~
+# Various IDEs
+.project
+.idea/
+*.tmproj
+.vscode/

charts/bridge-test/Chart.yaml

@@ -0,0 +1,28 @@
+apiVersion: v2
+name: bridge-test
+description: A Helm chart for Kubernetes
+
+# A chart can be either an 'application' or a 'library' chart.
+#
+# Application charts are a collection of templates that can be packaged into versioned archives
+# to be deployed.
+#
+# Library charts provide useful utilities or functions for the chart developer. They're included as
+# a dependency of application charts to inject those utilities and functions into the rendering
+# pipeline. Library charts do not define any templates and therefore cannot be deployed.
+type: application
+
+# This is the chart version. This version number should be incremented each time you make changes
+# to the chart and its templates, including the app version.
+# Versions are expected to follow Semantic Versioning (https://semver.org/)
+version: 0.1.0
+
+# This is the version number of the application being deployed. This version number should be
+# incremented each time you make changes to the application. Versions are not expected to
+# follow Semantic Versioning. They should reflect the version the application is using.
+# It is recommended to use it with quotes.
+appVersion: "0.1.0"
+
+maintainers:
+  - name: steezeburger
+    url: astria.org

charts/bridge-test/files/scripts/test-ibc-transfer.sh

@@ -0,0 +1,58 @@
+#!/bin/sh
+
+get_evm_balance() {
+  HEX_NUM=$(curl -X POST "$evm_url" -s -d "{\"jsonrpc\":\"2.0\",\"method\":\"eth_getBalance\",\"params\":[\"$evm_to_address\", \"latest\"],\"id\":1}" -H 'Content-Type: application/json' | jq -r '.result')
+  # strip 0x
+  HEX_NUM=$(echo "$HEX_NUM" | sed 's/^0x//')
+  # capitalize all lowercase letters
+  HEX_NUM=$(echo "$HEX_NUM" | tr '[:lower:]' '[:upper:]')
+  # print as integer
+  echo "ibase=16; $HEX_NUM" | bc
+}
+
+addKeyForCelestiaAccount() {
+  # add key for the celestia dev account using the mnemonic
+  echo "Adding key for the celestia dev account..."
+  echo "$celestia_dev_account_mnemonic" | celestia-appd keys add \
+    "$celestia_dev_account_key_name" \
+    --home "$home_dir" \
+    --keyring-backend="$keyring_backend" \
+    --recover
+}
+
+performIBCTransfer() {
+  # perform ibc transfer
+  echo "Performing IBC transfer..."
+  celestia-appd tx ibc-transfer transfer \
+    transfer \
+    channel-0 \
+    "$bridge_account_address_bech32" \
+    53000utia \
+    --memo="{\"rollupAddress\":\"$evm_to_address\"}" \
+    --chain-id="$celestia_chain_id" \
+    --node="$celestia_node_url" \
+    --from="$celestia_dev_account_address" \
+    --fees=26000utia \
+    --yes \
+    --log_level=debug \
+    --home "$home_dir" \
+    --keyring-backend="$keyring_backend"
+}
+
+initial_balance=$(get_evm_balance)
+
+addKeyForCelestiaAccount
+performIBCTransfer
+
+# FIXME - should probably poll w/ timeout instead of sleeping?
+sleep 30
+
+final_balance=$(get_evm_balance)
+expected_balance=$(echo "$initial_balance + 53000000000000000" | bc)
+if [ "$(echo "$final_balance == $expected_balance" | bc)" -eq 0 ]; then
+  echo "IBC Transfer failed!"
+  echo "Expected balance $expected_balance, got $final_balance"
+  exit 1
+else
+  echo "IBC Transfer successful!"
+fi

charts/bridge-test/templates/_helpers.tpl

@@ -0,0 +1,23 @@
+{{/*
+Namespace to deploy elements into.
+*/}}
+{{- define "bridge-test.namespace" -}}
+{{- default .Release.Namespace .Values.namespaceOverride | trunc 63 | trimSuffix "-" -}}
+{{- end }}
+
+{{/*
+Create a default fully qualified app name.
+We truncate at 63 chars because some Kubernetes name fields are limited to this (by the DNS naming spec).
+*/}}
+{{- define "bridge-test.fullname" -}}
+{{- if .Values.fullnameOverride -}}
+{{- .Values.fullnameOverride | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- $name := default .Chart.Name .Values.nameOverride -}}
+{{- if contains $name .Release.Name -}}
+{{- .Release.Name | trunc 63 | trimSuffix "-" -}}
+{{- else -}}
+{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}}
+{{- end -}}
+{{- end -}}
+{{- end -}}

charts/bridge-test/templates/configmap.yaml

@@ -0,0 +1,26 @@
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: bridge-test-env
+  namespace: {{ include "bridge-test.namespace" . }}
+data:
+  home_dir: "/home/celestia"
+  keyring_backend: "test"
+  bridge_account_address: "{{ .Values.bridgeAccount.address }}"
+  bridge_account_address_bech32: "{{ .Values.bridgeAccount.bech32 }}"
+  evm_to_address: "{{ .Values.evmToAddress }}"
+  evm_url: "{{ .Values.evmURL }}"
+  celestia_chain_id: "{{ .Values.celestiaChainID }}"
+  celestia_node_url: "{{ .Values.celestiaNodeURL }}"
+  celestia_dev_account_address: "{{ .Values.celestiaDevAccount.address }}"
+  celestia_dev_account_mnemonic: "{{ .Values.celestiaDevAccount.mnemonic }}"
+  celestia_dev_account_key_name: "{{ .Values.celestiaDevAccount.name }}"
+---
+apiVersion: v1
+kind: ConfigMap
+metadata:
+  name: bridge-test-scripts
+  namespace: {{ include "bridge-test.namespace" . }}
+data:
+  test-ibc-transfer.sh: |
+    {{- .Files.Get "files/scripts/test-ibc-transfer.sh" | nindent 4 }}

charts/bridge-test/templates/job.yaml

@@ -0,0 +1,41 @@
+apiVersion: batch/v1
+kind: Job
+metadata:
+  name: {{ include "bridge-test.fullname" . }}
+  labels:
+    app: astria-dev-cluster
+  namespace: {{ include "bridge-test.namespace" . }}
+spec:
+  template:
+    metadata:
+      name: {{ include "bridge-test.fullname" . }}
+      labels:
+        app: astria-dev-cluster
+    spec:
+      initContainers:
+        - name: init-bridge-account
+          image: {{ .Values.bridgeTesterUtilityImage }}
+          command: [ "astria-go", "sequencer", "bridge", "init", {{ .Values.evmRollupName }} ]
+          args:
+            - --privkey={{ .Values.bridgeAccount.privkey }}
+            - --sequencer-chain-id={{ .Values.sequencerChainId }}
+            - --sequencer-url={{ .Values.sequencerURL }}
+            - --asset={{ .Values.asset }}
+            - --fee-asset={{ .Values.feeAsset }}
+            - --log-level=debug
+      containers:
+        - name: test-ibc-transfer
+          image: {{ .Values.bridgeTesterUtilityImage }}
+          command: [ "/scripts/test-ibc-transfer.sh" ]
+          volumeMounts:
+            - mountPath: /scripts/
+              name: bridge-test-scripts-volume
+          envFrom:
+            - configMapRef:
+                name: bridge-test-env
+      volumes:
+        - name: bridge-test-scripts-volume
+          configMap:
+            name: bridge-test-scripts
+            defaultMode: 0777
+      restartPolicy: Never

charts/bridge-test/values.yaml

@@ -0,0 +1,38 @@
+replicaCount: 1
+
+# this image is overridden in ci/cd with the image built in ci/cd
+bridgeTesterUtilityImage: ghcr.io/astriaorg/bridge-tester-utility:local
+
+imagePullSecrets: []
+nameOverride: ""
+namespaceOverride: ""
+fullnameOverride: ""
+
+# must match rollup name used in evm rollup chart values
+evmRollupName: "astria"
+# this is a shared dev address
+evmToAddress: "0xaC21B97d35Bf75A7dAb16f35b111a50e78A72F30"
+# evm execution api url
+evmURL: "http://astria-evm-service.astria-dev-cluster.svc.cluster.local:8545"
+
+sequencerURL: "http://node0-sequencer-rpc-service.astria-dev-cluster.svc.cluster.local:26657"
+# must match chain id used in sequencer chart values
+sequencerChainId: "sequencer-test-chain-0"
+asset: "transfer/channel-0/utia"
+feeAsset: "nria"
+
+# sequencer bridge account. is funded during sequencer genesis.
+bridgeAccount:
+  address: "6f85297e587b61b37695a1ac17189b3e907e318e"
+  bech32: "astria1d7zjjljc0dsmxa545xkpwxym86g8uvvwhtezcr"
+  privkey: "6015fbe1c365d3c5ef92dc891db8c5bb26ad454bec2db4762b96e9f8b2430285"
+  pubkey: "b78aa61c65f21e5fe0f31d221819053fa2286dd6eff83fc490e3ee746f144626"
+
+celestiaChainID: "celestia-local-0"
+celestiaNodeURL: "http://celestia-app-service.astria-dev-cluster.svc.cluster.local:26657"
+
+# this account should be funded during celestia genesis
+celestiaDevAccount:
+  name: "dev-account"
+  address: "celestia1m0ksdjl2p5nzhqy3p47fksv52at3ln885xvl96"
+  mnemonic: "enrich avocado local net will avoid dizzy truth column excuse ready lesson"

charts/celestia-local/Chart.yaml

@@ -15,7 +15,7 @@ type: application
 # This is the chart version. This version number should be incremented each time you make changes
 # to the chart and its templates, including the app version.
 # Versions are expected to follow Semantic Versioning (https://semver.org/)
-version: 0.4.0
+version: 0.5.0
 
 # This is the version number of the application being deployed. This version number should be
 # incremented each time you make changes to the application. Versions are not expected to

charts/celestia-local/files/scripts/init-celestia-appd.sh

@@ -29,6 +29,30 @@ celestia-appd gentx \
   --chain-id "$chainid" \
   --home "$home_dir"
 
+# add ibc account
+echo "$ibc_account_mnemonic" | celestia-appd keys add \
+  "$ibc_account_key_name" \
+  --home "$home_dir" \
+  --keyring-backend="$keyring_backend" \
+  --recover
+ibc_account_key=$(celestia-appd keys show "$ibc_account_key_name" -a --keyring-backend="$keyring_backend" --home "$home_dir")
+celestia-appd add-genesis-account \
+  "$ibc_account_key" \
+  --home "$home_dir" \
+  "$coins"
+
+# add account used for development and testing
+echo "$dev_account_mnemonic" | celestia-appd keys add \
+  "$dev_account_key_name" \
+  --home "$home_dir" \
+  --keyring-backend="$keyring_backend" \
+  --recover
+dev_account_key=$(celestia-appd keys show "$dev_account_key_name" -a --keyring-backend="$keyring_backend" --home "$home_dir")
+celestia-appd add-genesis-account \
+  "$dev_account_key" \
+  --home "$home_dir" \
+  "$coins"
+
 celestia-appd collect-gentxs --home "$home_dir"
 
 # Enable transaction indexing

charts/celestia-local/files/scripts/start-celestia-appd.sh

@@ -26,5 +26,6 @@ exec celestia-appd start --home "${home_dir}" \
   --grpc.address "0.0.0.0:$celestia_app_grpc_port" \
   --rpc.laddr "tcp://0.0.0.0:$celestia_app_host_port" \
   --api.enable \
+  --api.enabled-unsafe-cors \
   --grpc.enable \
   --grpc-web.enable

charts/celestia-local/templates/configmap.yaml

@@ -16,6 +16,10 @@ data:
   bridge_rpc_port: "{{ .Values.ports.bridgeRPC }}"
   celestia_app_host_port: "{{ .Values.ports.celestiaAppHostPort }}"
   celestia_app_grpc_port: "{{ .Values.ports.celestiaAppGrpcPort }}"
+  ibc_account_mnemonic: "{{ .Values.ibcAccountMnemonic }}"
+  ibc_account_key_name: "{{ .Values.ibcAccountKeyName }}"
+  dev_account_mnemonic: "{{ .Values.devAccountMnemonic }}"
+  dev_account_key_name: "{{ .Values.devAccountName }}"
 ---
 apiVersion: v1
 kind: ConfigMap

charts/celestia-local/templates/deployment.yaml

@@ -63,6 +63,8 @@ spec:
               name: app-rpc
             - containerPort: {{ .Values.ports.celestiaAppGrpcPort }}
               name: app-grpc
+            - containerPort: {{ .Values.ports.celestiaAppRestPort }}
+              name: app-rest
         - name: celestia-bridge
           command: [ "/scripts/start-bridge.sh" ]
           image: {{ .Values.celestiaNodeImage }}