Updated testing documentation

Signed-off-by: Jose Noguera <[email protected]>

README.md

@@ -11,6 +11,7 @@ Generic Health Check Framework for systemd on [rpm-ostree](https://coreos.github
     - [Wanted Checks](#wanted-checks)
   + [Configuration](#configuration)
 * [How does it work](#how-does-it-work)
+* [Development](#development)
 
 ## Installation
 Greenboot is comprised of two packages:
@@ -123,3 +124,6 @@ At the moment, it is possible to customize the following parameters via environm
     - `greenboot-grub2-set-success.service` is run. It unsets `boot_counter` GRUB env var and sets `boot_success` GRUB env var to 1.
     - `greenboot-task-runner.service` launches `/usr/libexec/greenboot/greenboot green`, which runs the scripts in `green.d` folder, scripts that are meant to be run after a successful update.
     - `greenboot-status.service` is run, creating the MOTD with a success message.
+
+## Development
+Please refer to [development/README.md](https://github.com/fedora-iot/greenboot/blob/main/development/README.md) file.

development/README.md

@@ -1,137 +1,53 @@
-# Testing
-
-## Fedora IoT 30
-
-In order to be able to easily check the motd output, the preferred way of testing is over ssh.
-
-### Preparation
-```bash
-# ssh into your machine
-
-# Ensure the following rpms are installed:
-# greenboot greenboot-status
-# otherwise install them:
-build=00876215 && \
-    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-0.7-1.fc30.noarch.rpm --output greenboot-0.7-1.fc30.noarch.rpm && \
-    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-grub2-0.7-1.fc30.noarch.rpm --output greenboot-grub2-0.7-1.fc30.noarch.rpm && \
-    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-reboot-0.7-1.fc30.noarch.rpm --output greenboot-reboot-0.7-1.fc30.noarch.rpm && \
-    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm --output greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm && \
-    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-status-0.7-1.fc30.noarch.rpm --output greenboot-status-0.7-1.fc30.noarch.rpm && \
-sudo rpm-ostree override replace \
-    greenboot-0.7-1.fc30.noarch.rpm \
-    greenboot-grub2-0.7-1.fc30.noarch.rpm \
-    greenboot-reboot-0.7-1.fc30.noarch.rpm \
-    --remove=greenboot-ostree-grub2-0.6-1.fc30.noarch \
-    --install=greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm \
-    --install=greenboot-status-0.7-1.fc30.noarch.rpm && \
-sudo systemctl reboot
-
-# Enabling services, but let's hold off on enabling redboot-auto-reboot.service for a bit
-# so we don't get into reboot-looping through all our boot attempts (i.e. until boot_counter reaches 0)
-sudo systemctl enable \
-    greenboot-task-runner \
-    greenboot-healthcheck \
-    greenboot-rpm-ostree-grub2-check-fallback \
-    greenboot-grub2-set-counter \
-    greenboot-grub2-set-success \
-    greenboot-status \
-    redboot-task-runner && \
-sudo systemctl reboot
-
+# How to develop Greenboot
+
+## How to test your changes
+### Container 
+Ideal for quick development and making sure GitHub tests are going to pass but before creating a pull request, [try it out first in a VM with Fedora IoT and/or RHEL For Edge](#fedora-iot-or-rfe)
+
+Comment the line `ENTRYPOINT [ "/bin/bash", "launch_all_tests.sh" ]` in Dockerfile.
+
+Add this to a `docker-compose.yml` file:
+
+``` yaml
+version: "3.8"
+services: 
+  greenboot:
+    privileged: true
+    build: .
+    container_name: greenboot
+    image: greenboot
+    volumes:
+      - /run/systemd/journal:/run/systemd/journal
+    working_dir: /testing
+    command: sleep infinity
 ```
 
-### Testing
-
-```bash
-# Test success to complete checks
-#
-
-# When logging in via ssh, you should see the motd for a green boot status:
-#
-# Boot Status is GREEN - Health Check SUCCESS
-
-# Check all our logs
-sudo journalctl -b -0 \
-    -u boot-complete.target \
-    -u greenboot-task-runner \
-    -u greenboot-healthcheck \
-    -u greenboot-rpm-ostree-grub2-check-fallback \
-    -u greenboot-grub2-set-counter \
-    -u greenboot-grub2-set-success \
-    -u greenboot-status \
-    -u redboot-task-runner \
-    -u redboot-auto-reboot \
-    -u redboot.target
 
-# check grubenv variables
-sudo grub2-editenv list
+Run this script
+``` bash
+#!/bin/bash
+set -e
 
-# the service that sets boot_success to 1 before reboot should be active:
-sudo systemctl is-active greenboot-grub2-set-success.service
+# Bring down the previous environment in case there was one
+docker-compose down
 
+docker-compose up --build -d
+docker exec -it greenboot /bin/bash
+docker-compose down
+```
 
-# Test failure to complete checks
-#
-
-# Install sanely failing health check unit to test red boot status behavior
-curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/00858207-greenboot-failing-unit/greenboot-failing-unit-1.0-1.fc30.noarch.rpm --output greenboot-failing-unit-1.0-1.fc30.noarch.rpm && \
-sudo rpm-ostree install greenboot-failing-unit-1.0-1.fc30.noarch.rpm && \
-sudo systemctl reboot
-
-# If greenboot-reboot is disabled, you should see the following when logging in:
-#
-# Boot Status is RED - Health Check FAILURE!
-# Script '10_failing_check.sh' FAILURE (exit code '1')
-
-# Check all our journal logs again
-sudo journalctl -b -0 \
-    -u boot-complete.target \
-    -u greenboot-task-runner \
-    -u greenboot-healthcheck \
-    -u greenboot-rpm-ostree-grub2-check-fallback \
-    -u greenboot-grub2-set-counter \
-    -u greenboot-grub2-set-success \
-    -u greenboot-status \
-    -u redboot-task-runner \
-    -u redboot-auto-reboot \
-    -u redboot.target
-
-# grubenv should contain:
-# boot_counter=2
-sudo grub2-editenv list
-
-# the service to set boot_success to 1 before reboot should be inactive (dead):
-sudo systemctl status greenboot-grub2-set-success.service
-
-# Let's enable the system to try all the remaining boot attempts at this failing deployment,
-# before finally booting back into the rollback deployment automatically.
-sudo systemctl enable redboot-auto-reboot && \
-sudo systemctl reboot
-
-# After the remaining boot attempts have been tried, log in again and you should see:
-#
-# Boot Status is GREEN - Health Check SUCCESS
-# FALLBACK BOOT DETECTED! Default rpm-ostree deployment has been rolled back.
-# Health check logs from previous boot:
-# Script '10_failing_check.sh' FAILURE (exit code '1')
+### Fedora IoT or RFE:
+As `/usr` is mounted as a read-only directory:
 
-# Check all our journal logs again
-sudo journalctl -b -0 \
-    -u boot-complete.target \
-    -u greenboot-task-runner \
-    -u greenboot-healthcheck \
-    -u greenboot-rpm-ostree-grub2-check-fallback \
-    -u greenboot-grub2-set-counter \
-    -u greenboot-grub2-set-success \
-    -u greenboot-status \
-    -u redboot-task-runner \
-    -u redboot-auto-reboot \
-    -u redboot.target
+- If you want to test health checks, place them under `/etc/greenboot/check/required.d` or `/etc/greenboot/check/wanted.d`. I personally would go for `wanted.d` first to make sure that you don't end up in a full loop of reboots in case your script goes wrong.
 
-# grubenv
-sudo grub2-editenv list
+- If you’re editing any of the services under `/usr/lib` or `/usr/libexec`, you have two options:
+    - place the updated file under `/etc/systemd/system/{same_name_as_in_usr_lib} `and systemd will take this file with more priority.
+    - Advanced version: unlock the deployment and add your files in the proper `/usr` directory.
 
-# the service to set boot_success to 1 before reboot should be active again:
-sudo systemctl status greenboot-grub2-set-success.service
+## Unit testing
+Health checks, when possible, have been unit-tested with [BATS](https://bats-core.readthedocs.io/en/stable/writing-tests.html). These tests are placed under the [`tests` folder](https://github.com/fedora-iot/greenboot/blob/main/tests).
 
-```
+Every file with the `.bats` extension will be tested.
+For declaring/using common environment variables, use [`common.bash`](https://github.com/fedora-iot/greenboot/blob/main/tests/common.bash). You can source them in your BATS file.
+If you need to add a file, please use [`tests/testing_files` folder](https://github.com/fedora-iot/greenboot/blob/main/tests/testing_files).

development/legacy/README.md

@@ -0,0 +1,137 @@
+# Testing
+
+## Fedora IoT 30
+
+In order to be able to easily check the motd output, the preferred way of testing is over ssh.
+
+### Preparation
+```bash
+# ssh into your machine
+
+# Ensure the following rpms are installed:
+# greenboot greenboot-status
+# otherwise install them:
+build=00876215 && \
+    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-0.7-1.fc30.noarch.rpm --output greenboot-0.7-1.fc30.noarch.rpm && \
+    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-grub2-0.7-1.fc30.noarch.rpm --output greenboot-grub2-0.7-1.fc30.noarch.rpm && \
+    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-reboot-0.7-1.fc30.noarch.rpm --output greenboot-reboot-0.7-1.fc30.noarch.rpm && \
+    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm --output greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm && \
+    curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/$build-greenboot/greenboot-status-0.7-1.fc30.noarch.rpm --output greenboot-status-0.7-1.fc30.noarch.rpm && \
+sudo rpm-ostree override replace \
+    greenboot-0.7-1.fc30.noarch.rpm \
+    greenboot-grub2-0.7-1.fc30.noarch.rpm \
+    greenboot-reboot-0.7-1.fc30.noarch.rpm \
+    --remove=greenboot-ostree-grub2-0.6-1.fc30.noarch \
+    --install=greenboot-rpm-ostree-grub2-0.7-1.fc30.noarch.rpm \
+    --install=greenboot-status-0.7-1.fc30.noarch.rpm && \
+sudo systemctl reboot
+
+# Enabling services, but let's hold off on enabling redboot-auto-reboot.service for a bit
+# so we don't get into reboot-looping through all our boot attempts (i.e. until boot_counter reaches 0)
+sudo systemctl enable \
+    greenboot-task-runner \
+    greenboot-healthcheck \
+    greenboot-rpm-ostree-grub2-check-fallback \
+    greenboot-grub2-set-counter \
+    greenboot-grub2-set-success \
+    greenboot-status \
+    redboot-task-runner && \
+sudo systemctl reboot
+
+```
+
+### Testing
+
+```bash
+# Test success to complete checks
+#
+
+# When logging in via ssh, you should see the motd for a green boot status:
+#
+# Boot Status is GREEN - Health Check SUCCESS
+
+# Check all our logs
+sudo journalctl -b -0 \
+    -u boot-complete.target \
+    -u greenboot-task-runner \
+    -u greenboot-healthcheck \
+    -u greenboot-rpm-ostree-grub2-check-fallback \
+    -u greenboot-grub2-set-counter \
+    -u greenboot-grub2-set-success \
+    -u greenboot-status \
+    -u redboot-task-runner \
+    -u redboot-auto-reboot \
+    -u redboot.target
+
+# check grubenv variables
+sudo grub2-editenv list
+
+# the service that sets boot_success to 1 before reboot should be active:
+sudo systemctl is-active greenboot-grub2-set-success.service
+
+
+# Test failure to complete checks
+#
+
+# Install sanely failing health check unit to test red boot status behavior
+curl https://copr-be.cloud.fedoraproject.org/results/lorbus/greenboot/fedora-30-x86_64/00858207-greenboot-failing-unit/greenboot-failing-unit-1.0-1.fc30.noarch.rpm --output greenboot-failing-unit-1.0-1.fc30.noarch.rpm && \
+sudo rpm-ostree install greenboot-failing-unit-1.0-1.fc30.noarch.rpm && \
+sudo systemctl reboot
+
+# If greenboot-reboot is disabled, you should see the following when logging in:
+#
+# Boot Status is RED - Health Check FAILURE!
+# Script '10_failing_check.sh' FAILURE (exit code '1')
+
+# Check all our journal logs again
+sudo journalctl -b -0 \
+    -u boot-complete.target \
+    -u greenboot-task-runner \
+    -u greenboot-healthcheck \
+    -u greenboot-rpm-ostree-grub2-check-fallback \
+    -u greenboot-grub2-set-counter \
+    -u greenboot-grub2-set-success \
+    -u greenboot-status \
+    -u redboot-task-runner \
+    -u redboot-auto-reboot \
+    -u redboot.target
+
+# grubenv should contain:
+# boot_counter=2
+sudo grub2-editenv list
+
+# the service to set boot_success to 1 before reboot should be inactive (dead):
+sudo systemctl status greenboot-grub2-set-success.service
+
+# Let's enable the system to try all the remaining boot attempts at this failing deployment,
+# before finally booting back into the rollback deployment automatically.
+sudo systemctl enable redboot-auto-reboot && \
+sudo systemctl reboot
+
+# After the remaining boot attempts have been tried, log in again and you should see:
+#
+# Boot Status is GREEN - Health Check SUCCESS
+# FALLBACK BOOT DETECTED! Default rpm-ostree deployment has been rolled back.
+# Health check logs from previous boot:
+# Script '10_failing_check.sh' FAILURE (exit code '1')
+
+# Check all our journal logs again
+sudo journalctl -b -0 \
+    -u boot-complete.target \
+    -u greenboot-task-runner \
+    -u greenboot-healthcheck \
+    -u greenboot-rpm-ostree-grub2-check-fallback \
+    -u greenboot-grub2-set-counter \
+    -u greenboot-grub2-set-success \
+    -u greenboot-status \
+    -u redboot-task-runner \
+    -u redboot-auto-reboot \
+    -u redboot.target
+
+# grubenv
+sudo grub2-editenv list
+
+# the service to set boot_success to 1 before reboot should be active again:
+sudo systemctl status greenboot-grub2-set-success.service
+
+```