-
-
Notifications
You must be signed in to change notification settings - Fork 27
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Refactor build process #59
Merged
Merged
Changes from all commits
Commits
Show all changes
16 commits
Select commit
Hold shift + click to select a range
19dc602
refactor: repackage everything
marvinmarnold 1091867
refactor: redo entire app build
marvinmarnold d297992
refactor: reset_lgw shared by all concentrators
marvinmarnold c88a9f3
feat: monitor lora_pkt_fwd state
marvinmarnold d0d11aa
refactor(build): use images with Python
marvinmarnold 4ff56f9
fix: injected conf for sx1302
marvinmarnold 1547f8b
fix: shellcheck and linting
marvinmarnold da5d989
fix: miscellaneous errors
marvinmarnold b9d7038
build: bump packet_forwarder dep to master branch
marvinmarnold eeecb51
build: bump sx1302_hal dep to master branch
marvinmarnold 5563ec6
docs: removed TODO about sleep
marvinmarnold 8b540fb
refactor: more granular start logic
marvinmarnold e7eb5ce
Merge remote-tracking branch 'origin/master' into marvinmarnold/total…
marvinmarnold 8a4362d
fix: linting
marvinmarnold c1852ac
fix: tweak sleep when starting
marvinmarnold f948c6f
docs: add envvars to README
marvinmarnold File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,114 +1,76 @@ | ||
| # Packet Forwarder Docker File | ||
| # (C) Nebra Ltd 2019 | ||
| # (C) Nebra Ltd 2021 | ||
| # Licensed under the MIT License. | ||
|
|
||
| #################################################################################################### | ||
| ################################## Stage: builder ################################################## | ||
| ########################### Stage: PktFwd Python App Builder ####################################### | ||
| FROM balenalib/raspberry-pi-debian-python:buster-build-20210705 as pktfwd-builder | ||
|
|
||
| FROM balenalib/raspberry-pi-debian:buster-build as builder | ||
| # Variables used internally to this stage | ||
| ENV INPUT_DIR=/opt/input | ||
|
|
||
| # Move to correct working directory | ||
| WORKDIR /opt/iotloragateway/dev | ||
| # Variables to be referenced pktfwd-runner stage | ||
| ENV OUTPUT_DIR=/opt/output/pktfwd-dependencies | ||
|
|
||
| # Copy python dependencies for `pip install` later | ||
| WORKDIR "$INPUT_DIR" | ||
|
|
||
| # Copy python dependencies for `pip3 install` later | ||
| COPY requirements.txt requirements.txt | ||
|
|
||
| # This will be the path that venv uses for installation below | ||
| ENV PATH="/opt/iotloragateway/dev/venv/bin:$PATH" | ||
|
|
||
| # Install build tools | ||
| # hadolint ignore=DL3008 | ||
| RUN apt-get update && \ | ||
| apt-get -y install --no-install-recommends \ | ||
| automake \ | ||
| libtool \ | ||
| autoconf \ | ||
| git \ | ||
| ca-certificates \ | ||
| pkg-config \ | ||
| build-essential \ | ||
| python3 \ | ||
| python3-pip \ | ||
| python3-venv && \ | ||
| # Because the PATH is already updated above, this command creates a new venv AND activates it | ||
| python3 -m venv /opt/iotloragateway/dev/venv && \ | ||
| # Given venv is active, this `pip` refers to the python3 variant | ||
| pip install --no-cache-dir -r requirements.txt | ||
|
|
||
| # Copy the buildfiles and sx1302 concentrator fixes | ||
| COPY buildfiles buildfiles | ||
| COPY sx1302fixes sx1302fixes | ||
|
|
||
| # Clone the lora gateway and packet forwarder repos | ||
| RUN git clone https://github.com/NebraLtd/lora_gateway.git | ||
| RUN git clone https://github.com/NebraLtd/packet_forwarder.git | ||
|
|
||
| # Create folder needed by packetforwarder compiler | ||
| RUN mkdir -p /opt/iotloragateway/packetforwarder | ||
|
|
||
| # Compile for sx1301 concentrator on all the necessary SPI buses | ||
| RUN ./buildfiles/compileSX1301.sh spidev0.0 | ||
| RUN ./buildfiles/compileSX1301.sh spidev0.1 | ||
| RUN ./buildfiles/compileSX1301.sh spidev1.0 | ||
| RUN ./buildfiles/compileSX1301.sh spidev1.1 | ||
| RUN ./buildfiles/compileSX1301.sh spidev1.2 | ||
| RUN ./buildfiles/compileSX1301.sh spidev2.0 | ||
| RUN ./buildfiles/compileSX1301.sh spidev2.1 | ||
| RUN ./buildfiles/compileSX1301.sh spidev32766.0 | ||
|
|
||
| # Compile for sx1302 concentrator | ||
| RUN ./buildfiles/compileSX1302.sh | ||
|
|
||
| # No need to cleanup the builder | ||
| RUN pip3 install --target="$OUTPUT_DIR" --no-cache-dir -r requirements.txt | ||
|
|
||
| #################################################################################################### | ||
| ################################### Stage: runner ################################################## | ||
| ################################################################################################### | ||
| ################################## Stage: runner ################################################## | ||
| FROM balenalib/raspberry-pi-debian-python:buster-run-20210705 as pktfwd-runner | ||
|
|
||
| FROM balenalib/raspberry-pi-debian:buster-run as runner | ||
| ENV ROOT_DIR=/opt | ||
|
|
||
| # Start in sx1301 directory | ||
| WORKDIR /opt/iotloragateway/packet_forwarder/sx1301 | ||
| # Copy from: Locations of build assets within images of earlier stages | ||
| ENV SX1301_PACKET_FORWARDER_OUTPUT_DIR=/opt/output | ||
| ENV SX1302_HAL_OUTPUT_DIR=/opt/output | ||
| ENV PKTFWD_BUILDER_OUTPUT_DIR=/opt/output/pktfwd-dependencies | ||
|
|
||
| # Install python3-venv and python3-rpi.gpio | ||
| # hadolint ignore=DL3008 | ||
| RUN apt-get update && \ | ||
| apt-get -y install \ | ||
| python3-venv \ | ||
| python3-rpi.gpio && \ | ||
| apt-get autoremove -y && \ | ||
| apt-get clean && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
| # Copy to: Locations build assets from earlier stages/source are copied into | ||
| ENV SX1301_DIR="$ROOT_DIR/sx1301" | ||
| ENV SX1302_DIR="$ROOT_DIR/sx1302" | ||
| ENV PYTHON_APP_DIR="$ROOT_DIR/pktfwd" | ||
| ENV PYTHON_DEPENDENCIES_DIR="$ROOT_DIR/pktfwd-dependencies" | ||
| ENV SX1301_RESET_LGW_FILEPATH="$SX1301_DIR/reset_lgw.sh" | ||
|
|
||
| # Copy sx1301 packetforwader from builder | ||
| COPY --from=builder /opt/iotloragateway/packetforwarder . | ||
| # Variables required for pktfwd python app | ||
| ENV SX1301_REGION_CONFIGS_DIR="$PYTHON_APP_DIR/config/lora_templates_sx1301" | ||
| ENV SX1302_REGION_CONFIGS_DIR="$PYTHON_APP_DIR/config/lora_templates_sx1302" | ||
| ENV SX1302_LORA_PKT_FWD_FILEPATH="$SX1302_DIR/lora_pkt_fwd" | ||
| ENV UTIL_CHIP_ID_FILEPATH="$SX1302_DIR/chip_id" | ||
| # The sx1302_hal concentrator script requires reset_lgw to be in this location | ||
| ENV RESET_LGW_FILEPATH="$SX1302_DIR/reset_lgw.sh" | ||
|
|
||
| # Copy sx1301 regional config templates | ||
| COPY lora_templates_sx1301 lora_templates_sx1301/ | ||
| WORKDIR "$ROOT_DIR" | ||
|
|
||
| # Use EU config as initial default | ||
| COPY lora_templates_sx1301/local_conf.json local_conf.json | ||
| COPY lora_templates_sx1301/EU-global_conf.json global_conf.json | ||
| # Copy python app | ||
| COPY pktfwd/ "$PYTHON_APP_DIR" | ||
|
|
||
| # Move to sx1302 directory | ||
| WORKDIR /opt/iotloragateway/packet_forwarder/sx1302 | ||
| # Copy reset script | ||
| COPY reset_lgw.sh "$RESET_LGW_FILEPATH" | ||
|
|
||
| # Copy sx1302 hal from builder | ||
| COPY --from=builder /opt/iotloragateway/dev/sx1302_hal-2.1.0 . | ||
| # Copy sx1301 lora_pkt_fwd_SPI_BUS | ||
| # hadolint ignore=DL3022 | ||
| COPY --from=nebraltd/packet_forwarder:e8f24fe37ba555e5ad1ddf8eed26d0136f30f8de "$SX1301_PACKET_FORWARDER_OUTPUT_DIR" "$SX1301_DIR" | ||
|
|
||
| # Copy sx1302 regional config templates | ||
| COPY lora_templates_sx1302 lora_templates_sx1302/ | ||
| # Copy sx1302 chip_id, reset_lgw, and lora_pkt_fwd | ||
| # hadolint ignore=DL3022 | ||
| COPY --from=nebraltd/sx1302_hal:69811057222f6f9cf8929ebfdb7fc6e36cc2618d "$SX1302_HAL_OUTPUT_DIR" "$SX1302_DIR" | ||
|
|
||
| # Use EU config as initial default | ||
| COPY lora_templates_sx1302/local_conf.json packet_forwarder/local_conf.json | ||
| COPY lora_templates_sx1302/EU-global_conf.json packet_forwarder/global_conf.json | ||
| # Copy pktfwd python app dependencies | ||
| COPY --from=pktfwd-builder "$PKTFWD_BUILDER_OUTPUT_DIR" "$PYTHON_DEPENDENCIES_DIR" | ||
|
|
||
| # Move to main packet forwarder directory and copy source code | ||
| WORKDIR /opt/iotloragateway/packet_forwarder | ||
| COPY files/* . | ||
| # Cleanup | ||
| RUN apt-get autoremove -y && \ | ||
| apt-get clean && \ | ||
| rm -rf /var/lib/apt/lists/* | ||
|
|
||
| # Copy venv from builder and update PATH to activate it | ||
| COPY --from=builder /opt/iotloragateway/dev/venv /opt/iotloragateway/dev/venv | ||
| ENV PATH="/opt/iotloragateway/dev/venv/bin:$PATH" | ||
| # Add python dependencies to PYTHONPATH | ||
| ENV PYTHONPATH="${PYTHONPATH}:${PYTHON_DEPENDENCIES_DIR}" | ||
|
|
||
| # Run run_pkt script | ||
| ENTRYPOINT ["sh", "/opt/iotloragateway/packet_forwarder/run_pkt.sh"] | ||
| # Run pktfwd/__main__.py | ||
| ENTRYPOINT ["python3", "pktfwd"] | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file was deleted.
Oops, something went wrong.
This file was deleted.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
During a recent engineering call, we decided not to use venv moving forward. The overall feeling is that Docker is already portable so venv is an unnecessary level of encapsulation. One advantage of the venv is that the python version is also copied.