Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add light hotspot automation #411

Closed
wants to merge 2 commits into from
Closed

Add light hotspot automation #411

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
527c1da
Added light hotspot automation
Mar 28, 2022
81fb2ef
Merged latest changes from light-hotspot
Mar 28, 2022
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 3 additions & 31 deletions .github/workflows/push-to-prod-draft.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -10,34 +10,6 @@ jobs:
fail-fast: false
matrix:
sbc: [raspi, rockpi, rak, pisces, og, sensecap, finestra, controllino]
frequency: [470, 868, 915]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we really want to touch the prod workflows? Won't those be unchanged?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We're removing indoor outdoor variants and all frequencies. So, not needed to keep all of matrix rows.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we had decided that fleet reorganization will be separate ticket. if that matter has been settled, then its good.

variant: [indoor, outdoor]
exclude:
- sbc: rak
- sbc: pisces
- sbc: og
- sbc: sensecap
- sbc: finestra
- sbc: controllino
include:
- sbc: pisces
variant: indoor
frequency: 868
- sbc: rak
variant: indoor
frequency: 868
- sbc: og
variant: indoor
frequency: 868
- sbc: sensecap
variant: indoor
frequency: 868
- sbc: finestra
variant: indoor
frequency: 868
- sbc: controllino
variant: indoor
frequency: 868
steps:
- name: Checkout
uses: actions/checkout@v2
Expand All @@ -62,7 +34,7 @@ jobs:
- name: Generate Docker Compose for RAK and Non-Nebra RasPi
if: matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino'
run: |
python gen_docker_compose.py rpi -o docker-compose.yml
python gen_docker_compose.py rpi -o docker-compose.yml
cp balena/balena.yml.${{ matrix.sbc }} balena.yml
- name: Generate Docker Compose for Pisces
if: matrix.sbc == 'pisces'
Expand All @@ -74,13 +46,13 @@ jobs:
if: matrix.sbc == 'rockpi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN}}
balena_command: "deploy nebraltd/helium-${{ matrix.variant }}-${{ matrix.frequency }}${{ env.ROCKPI }} --logs --debug --nocache --build --draft"
balena_command: "deploy nebraltd/helium${{ env.ROCKPI }} --logs --debug --nocache --build --draft"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you remind me where env.ROCKPI is being set and what values it takes on?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

helium-miner-software/.github/workflows/push-to-prod-draft.yml

Line 28 in 81fb2ef

ROCKPI="-rockpi"

- name: Balena Deploy (Nebra RasPi)
uses: nebraltd/[email protected]
if: matrix.sbc == 'raspi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_1}}
balena_command: "deploy nebraltd/helium-${{ matrix.variant }}-${{ matrix.frequency }} --logs --debug --nocache --build --draft"
balena_command: "deploy nebraltd/helium-light-hotspot --logs --debug --nocache --build --draft"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be helium-light-test? I'm currently using helium-light-hotspot to test this flow which is different.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was for production, not testnet.

- name: Balena Deploy (RAK and Non-Nebra RasPi)
uses: nebraltd/[email protected]
if: ( matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino' ) && success()
Expand Down
34 changes: 3 additions & 31 deletions .github/workflows/push-to-prod.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,34 +13,6 @@ jobs:
fail-fast: false
matrix:
sbc: [raspi, rockpi, rak, pisces, og, sensecap, finestra, controllino]
frequency: [470, 868, 915]
variant: [indoor, outdoor]
exclude:
- sbc: rak
- sbc: pisces
- sbc: og
- sbc: sensecap
- sbc: finestra
- sbc: controllino
include:
- sbc: pisces
variant: indoor
frequency: 868
- sbc: rak
variant: indoor
frequency: 868
- sbc: og
variant: indoor
frequency: 868
- sbc: sensecap
variant: indoor
frequency: 868
- sbc: finestra
variant: indoor
frequency: 868
- sbc: controllino
variant: indoor
frequency: 868
steps:
- name: Checkout
uses: actions/checkout@v2
Expand All @@ -63,7 +35,7 @@ jobs:
- name: Generate Docker Compose for RAK and Non-Nebra RasPi
if: matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino'
run: |
python gen_docker_compose.py rpi -o docker-compose.yml
python gen_docker_compose.py rpi -o docker-compose.yml
cp balena/balena.yml.${{ matrix.sbc }} balena.yml
- name: Generate Docker Compose for Pisces
if: matrix.sbc == 'pisces'
Expand All @@ -75,13 +47,13 @@ jobs:
if: matrix.sbc == 'rockpi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN}}
balena_command: "deploy nebraltd/helium-${{ matrix.variant }}-${{ matrix.frequency }}${{ env.ROCKPI }} --logs --debug --nocache --build"
balena_command: "deploy nebraltd/helium${{ env.ROCKPI }} --logs --debug --nocache --build"
- name: Balena Deploy (Nebra RasPi)
uses: nebraltd/[email protected]
if: matrix.sbc == 'raspi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_1}}
balena_command: "deploy nebraltd/helium-${{ matrix.variant }}-${{ matrix.frequency }} --logs --debug --nocache --build"
balena_command: "deploy nebraltd/helium-light-hotspot --logs --debug --nocache --build"
- name: Balena Deploy (RAK and Non-Nebra RasPi)
uses: nebraltd/[email protected]
if: ( matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino' ) && success()
Expand Down
10 changes: 5 additions & 5 deletions .github/workflows/push-to-testnet-draft.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,19 +41,19 @@ jobs:
if: matrix.sbc == 'rockpi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN}}
balena_command: "deploy nebraltd/helium-testnet${{ env.ROCKPI }} --logs --debug --nocache --build --draft"
balena_command: "deploy nebraltd/helium-light-testnet${{ env.ROCKPI }} --logs --debug --nocache --build --draft"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about consolidating all of these into a single alena_command: "deploy nebraltd/helium-light-testnet-${{ matrix.sbc }} --logs --debug --nocache --build --draft"?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why do we need more than one testnet for light? Probably overkill just for testing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also it should be helium-light-test fleet that already exists

Take a look at the light-hotspot-software repo

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about consolidating all of these into a single alena_command: "deploy nebraltd/helium-light-testnet-${{ matrix.sbc }} --logs --debug --nocache --build --draft"?

That could be possible. Let me think about our reasons to keep it separate.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It was my idea. We'll need to have separate fleets for every device type. I thought it has to mimic the same environment for testing. Also this is a one time setup. Otherwise, we'll be chasing different things for each device on testnet. That could need more effort than this one. At least this was my humble opinion.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also it should be helium-light-test fleet that already exists

Take a look at the light-hotspot-software repo

OK. I can use helium-light-test. But AFAIK we had abondoned light-sotspot-software and using light-hotspot branch of helium-miner-software?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes we are, but essentially we are migrating from that older repo to this one to make it easier for us to merge this to master branch when it goes live.

So it needs to be done in a way which can easily be merged with master further down the line.

Also, we aren't going to move devices to new fleets for light. We will just update the existing fleets with the newer software, when the time comes

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK.

- name: Balena Deploy (Nebra RasPi)
uses: nebraltd/[email protected]
if: matrix.sbc == 'raspi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_1}}
balena_command: "deploy nebraltd/helium-testnet --logs --debug --nocache --build --draft"
balena_command: "deploy nebraltd/helium-light-testnet --logs --debug --nocache --build --draft"
- name: Balena Deploy (RAK and other non-Nebra RasPi)
uses: nebraltd/[email protected]
if: success() && (matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino' || matrix.sbc == 'pisces')
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_2}}
balena_command: "deploy nebraltd/helium-testnet-${{ matrix.sbc }} --logs --debug --nocache --build --draft"
balena_command: "deploy nebraltd/helium-light-testnet-${{ matrix.sbc }} --logs --debug --nocache --build --draft"
- name: Push updated docker compose file (ROCK Pi)
if: matrix.sbc == 'rockpi' && success()
uses: test-room-7/action-update-file@v1
Expand All @@ -68,13 +68,13 @@ jobs:
with:
branch: master
file-path: device-compose-files/docker-compose-rpi.yml
commit-msg: Update Raspberry Pi docker compose output
commit-msg: Update Raspberry Pi docker compose output
github-token: ${{ secrets.MR_BUMP }}
- name: Push updated docker compose file (Pisces)
if: matrix.sbc == 'pisces' && success()
uses: test-room-7/action-update-file@v1
with:
branch: master
file-path: device-compose-files/docker-compose-pisces.yml
commit-msg: Update Pisces P100 docker compose output
commit-msg: Update Pisces P100 docker compose output
github-token: ${{ secrets.MR_BUMP }}
18 changes: 9 additions & 9 deletions .github/workflows/push-to-testnet.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ on:
push:
# Only run workflow for pushes to specific branches
branches:
- master
- light-hotspot
paths-ignore:
- 'device-compose-files/**'

Expand Down Expand Up @@ -46,40 +46,40 @@ jobs:
if: matrix.sbc == 'rockpi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN}}
balena_command: "deploy nebraltd/helium-testnet${{ env.ROCKPI }} --logs --debug --nocache --build"
balena_command: "deploy nebraltd/helium-light-testnet${{ env.ROCKPI }} --logs --debug --nocache --build"
- name: Balena Deploy (Nebra RasPi)
uses: nebraltd/[email protected]
if: matrix.sbc == 'raspi' && success()
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_1}}
balena_command: "deploy nebraltd/helium-testnet --logs --debug --nocache --build"
balena_command: "deploy nebraltd/helium-light-testnet --logs --debug --nocache --build"
- name: Balena Deploy (RAK and other non-Nebra RasPi)
uses: nebraltd/[email protected]
if: success() && (matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino' || matrix.sbc == 'pisces')
with:
balena_api_token: ${{secrets.BALENA_API_TOKEN_2}}
balena_command: "deploy nebraltd/helium-testnet-${{ matrix.sbc }} --logs --debug --nocache --build"
balena_command: "deploy nebraltd/helium-light-testnet-${{ matrix.sbc }} --logs --debug --nocache --build"
- name: Push updated docker compose file (ROCK Pi)
if: matrix.sbc == 'rockpi' && success()
uses: test-room-7/action-update-file@v1
with:
branch: master
branch: light-hotspot
file-path: device-compose-files/docker-compose-rockpi.yml
commit-msg: Update ROCK Pi docker compose output
github-token: ${{ secrets.MR_BUMP }}
- name: Push updated docker compose file (Nebra RasPi, RAK and other RasPi based vendors)
if: (matrix.sbc == 'raspi' || matrix.sbc == 'rak' || matrix.sbc == 'og' || matrix.sbc == 'sensecap' || matrix.sbc == 'finestra' || matrix.sbc == 'controllino') && success()
uses: test-room-7/action-update-file@v1
with:
branch: master
branch: light-hotspot
file-path: device-compose-files/docker-compose-rpi.yml
commit-msg: Update Raspberry Pi docker compose output
commit-msg: Update Raspberry Pi docker compose output
github-token: ${{ secrets.MR_BUMP }}
- name: Push updated docker compose file (Pisces)
if: matrix.sbc == 'pisces' && success()
uses: test-room-7/action-update-file@v1
with:
branch: master
branch: light-hotspot
file-path: device-compose-files/docker-compose-pisces.yml
commit-msg: Update Pisces P100 docker compose output
commit-msg: Update Pisces P100 docker compose output
github-token: ${{ secrets.MR_BUMP }}
34 changes: 34 additions & 0 deletions .github/workflows/update-gatewayrs.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
name: Update Gateway-rs (Testnet)
on:
repository_dispatch:
types: [gatewayrs-update]
jobs:
gatewayrs-update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
ref: ${{ github.ref }}
- run: |
LATEST_GATEWAYRS=${{ github.event.client_payload.tag }}
LONG_SHA=${{ github.event.client_payload.sha }}
SHORT_SHA=$( echo ${LONG_SHA:0:7} )
echo "LATEST_GATEWAYRS=$LATEST_GATEWAYRS" >> $GITHUB_ENV
echo "LONG_SHA=$LONG_SHA" >> $GITHUB_ENV

echo "We're not on the latest Helium Gateway-rs release. Updating to $LATEST_GATEWAYRS with SHA $SHORT_SHA."
sed -i -E "s/FIRMWARE_VERSION=.*/FIRMWARE_VERSION=$LATEST_GATEWAYRS/g" settings.ini
sed -i -E "s/GATEWAYRS_VERSION=.*/GATEWAYRS_VERSION=$SHORT_SHA/g" settings.ini
- name: Create Pull Request
uses: peter-evans/create-pull-request@v3
with:
commit-message: Update gateway-rs to latest public gateway-rs release ${{ env.LATEST_GATEWAYRS }}
branch: gatewayrs-bump/${{ env.LATEST_GATEWAYRS }}
delete-branch: true
base: master
title: "release(testnet): Update gateway-rs to latest public gateway-rs release ${{ env.LATEST_GATEWAYRS }}"
body: |
Update gateway-rs to latest public gateway-rs release ${{ env.LATEST_GATEWAYRS }}

Ref NebraLtd/hm-gatewayrs@${{ env.LONG_SHA }}
draft: false
34 changes: 0 additions & 34 deletions .github/workflows/update-miner.yml
View file
Open in desktop

This file was deleted.

4 changes: 2 additions & 2 deletions settings.ini
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
[versions]
FIRMWARE_VERSION=2022.03.07.0-2
FIRMWARE_VERSION=v1.0.0-alpha.23
DIAGNOSTICS_VERSION=fb63618
CONFIG_VERSION=e5ac58c
PKTFWD_VERSION=b95cc46
GATEWAY_VERSION=409c974
GATEWAYRS_VERSION=d3b1c78

[quectel_modem]
MODEM_SERIAL0=/dev/ttyUSB0
Expand Down
30 changes: 5 additions & 25 deletions templates/docker-compose.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,40 +5,35 @@ services:
gateway-config:
image: nebraltd/hm-config:{{CONFIG_VERSION}}
depends_on:
- helium-miner
- gateway-rs
- diagnostics
- dbus-session
environment:
- FIRMWARE_VERSION={{FIRMWARE_VERSION}}
- FIRMWARE_SHORT_HASH={{ENV.FIRMWARE_SHORT_HASH}}
- DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket
- DBUS_SESSION_BUS_ADDRESS=unix:path=/session/dbus/session_bus_socket
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System bus is still required I believe

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above. This was a thing I'd like to discuss while the PR is open.

privileged: true
network_mode: host
cap_add:
- NET_ADMIN
volumes:
- miner-storage:/var/data
- dbus:/session/dbus
labels:
io.balena.features.sysfs: 1
io.balena.features.kernel-modules: 1
io.balena.features.dbus: 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

System bus still needed I think

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍

stop_signal: SIGINT

packet-forwarder:
image: nebraltd/hm-pktfwd:{{PKTFWD_VERSION}}
depends_on:
- helium-miner
- gateway-rs
restart: always
privileged: true
volumes:
- pktfwdr:/var/pktfwd
environment:
- FIRMWARE_VERSION={{FIRMWARE_VERSION}}

helium-miner:
image: nebraltd/hm-gatewayrs:{{GATEWAY_VERSION}}
gateway-rs:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

See comment about why this can't be changed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It shouldn't be a problem, but I'm not sure. I've removed the dependency, but will make sure packet-forwarder doesn't fail.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we will have to keep helium-miner. it will need changes at too many places. Not important right now.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah it would need changing in packet forwarder and in hm-pyhelper and not worth the effort.

Also having it called helium-miner is still technically correct

image: nebraltd/hm-gatewayrs:{{GATEWAYRS_VERSION}}
expose:
- "1680"
- "4467"
Expand All @@ -48,20 +43,17 @@ services:
- {{I2C_DEVICE}}:{{I2C_DEVICE}}
restart: on-failure
environment:
- DBUS_SYSTEM_BUS_ADDRESS=unix:path=/session/dbus/session_bus_socket
- FIRMWARE_VERSION={{FIRMWARE_VERSION}}
- I2C_DEVICE={{I2C_DEVICE}}

diagnostics:
image: nebraltd/hm-diag:{{DIAGNOSTICS_VERSION}}
depends_on:
- helium-miner
- dbus-session
- gateway-rs
environment:
- FIRMWARE_VERSION={{FIRMWARE_VERSION}}
- FIRMWARE_SHORT_HASH={{ENV.FIRMWARE_SHORT_HASH}}
- DIAGNOSTICS_VERSION={{DIAGNOSTICS_VERSION}}
- DBUS_SYSTEM_BUS_ADDRESS=unix:path=/host/run/dbus/system_bus_socket
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is still required

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wasn't sure about that. @pritamghanghas What do you think?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

system bus is still needed.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need it for Bluetooth / LTE / WiFi

volumes:
- pktfwdr:/var/pktfwd
- miner-storage:/var/data
Expand All @@ -79,19 +71,7 @@ services:
labels:
io.balena.features.sysfs: 1
io.balena.features.procfs: 1
io.balena.features.dbus: 1
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Still needed

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍


dbus-session:
image: balenablocks/dbus:rpi-0.0.2
restart: always
volumes:
- dbus:/session/dbus
environment:
- DBUS_ADDRESS=unix:path=/session/dbus/session_bus_socket
- FIRMWARE_VERSION={{FIRMWARE_VERSION}}
- FIRMWARE_SHORT_HASH={{ENV.FIRMWARE_SHORT_HASH}}

volumes:
miner-storage:
pktfwdr:
dbus: