diff --git a/doc/_quarto.yml b/doc/_quarto.yml index 60082153..fb69e66a 100644 --- a/doc/_quarto.yml +++ b/doc/_quarto.yml @@ -70,6 +70,8 @@ website: - title: NixOS file: ./nixos/introduction.md - section: Tutorials + contents: + - ./nixos/tutorials/archiver-appliance.md - section: User Guides - section: Explanations - section: References diff --git a/doc/nixos/tutorials/archiver-appliance.md b/doc/nixos/tutorials/archiver-appliance.md new file mode 100644 index 00000000..23ee34b4 --- /dev/null +++ b/doc/nixos/tutorials/archiver-appliance.md @@ -0,0 +1,338 @@ +# Creating an Archiver Appliance instance + +In this tutorial, +we're going to see how to create a virtual machine that runs Archiver Appliance, +under the NixOS Linux distribution. + +You will need: + +- A virtual machine, +- and the [NixOS ISO file]. + Select the "Graphical ISO image." + + [NixOS ISO file]: https://nixos.org/download#download-nixos + +## Installing NixOS + +First things first, +create your virtual machine, +and select the ISO image that you downloaded. + +Then, start the virtual machine. + +From the booted virtual machine, +you can follow the graphical installation process, +and reboot once finished. + +You can select any desktop environment, +or no desktop. +This tutorial only uses the command-line. + +## Making your configuration a flake + +The installation process has created the `/etc/nixos` directory in your VM. +This directory describes the complete configuration of your machine. + +EPNix is a "Nix flake", +which a way of managing Nix projects. +Using Nix flakes also enables you to use Nix code outside of your repository, +in a controlled manner. + +To be able to import EPNix into you NixOS configuration, +you first need to make your NixOS configuration into a Nix flake. + +As root, go into `/etc/nixos` in your virtual machine. +Create a `flake.nix` file under it, +by running `nano flake.nix`. +Fill the file with these lines: + +``` {.nix filename="flake.nix" code-line-numbers="true"} +{ + description = "Configuration for running Archiver Appliance in a VM"; + + inputs.nixpkgs.url = "github:NixOS/nixpkgs/nixos-23.05"; + inputs.epnix.url = "github:epics-extensions/EPNix"; + + outputs = { self, nixpkgs, epnix }: { + nixosConfigurations.nixos = nixpkgs.lib.nixosSystem { + modules = [ + epnix.nixosModules.nixos + + ./configuration.nix + ]; + }; + }; +} +``` + +```{=html} + + +``` +Save and quit by typing ``{=html}Ctrl-x``{=html}, ``{=html}y``{=html}, and ``{=html}Enter``{=html}, +and run `nixos-rebuild test` to test your changes. + +Some explanations: + +You can see in the `flake.nix` file that the flake has 2 inputs: +`nixpkgs` and `epnix`, +lines 4--5. + +Having the `nixpkgs` input enables you to use code from [Nixpkgs]. +This is what enables you to use all those NixOS options, +and every package installed on your machine now. +With the current configuration, +we are only using code from Nixpkgs. + +Having the `epnix` input is what's going to enable you to use [packages from EPNix], +such as Archiver Appliance. +It also enables you to use [EPNix' extra NixOS options], +such as the options configuring Tomcat, the systemd service, the `archappl` user and group, MariaDB, and so on. + + [Nixpkgs]: https://github.com/NixOS/nixpkgs + [packages from EPNix]: ../../pkgs/packages.md + [EPNix' extra NixOS options]: ../options.md + +## Configuring Archiver Appliance + +Now for the fun part, +actually using those EPNix options to install and configure Archiver Appliance, +and all its dependencies. + +Create and edit the file `archiver-appliance.nix` under `/etc/nixos`. +For now, here are the contents: + +``` {.nix filename="archiver-appliance.nix"} +{ + services.archiver-appliance.enable = true; +} +``` + +In your `flake.nix`, +import the newly created file by adding `./archiver-appliance.nix`, +under `./configuration.nix`: + +``` diff + modules = [ + epnix.nixosModules.nixos + + ./configuration.nix ++ ./archiver-appliance.nix + ]; +``` + +If you try to test your changes by running `nixos-rebuild test`, +you will see a helpful error message: + +::: sourceCode +``` sourcecode +error: The option `services.archiver-appliance.stores.lts.location' + is used but not defined. +(use '--show-trace' to show detailed location information) +``` +::: + +This tells you that the `services.archiver-appliance.stores.lts.location` is mandatory, +but we didn't set any value. + +To figure out what this option is about, +you can examine the [options reference]. + +The options reference gives a description for this option: + +> Backing directory containing the LTS. + +and an example: + +``` nix +"/data/lts" +``` + +It tells us that you need to choose where the Long Term Store (LTS) is. +See the "Architecture" section of the [Archiver Appliance Details] page for what the various stores are. + +Because this is a test VM, +let's configure the LTS to a test location, +like `/tmp/lts`. +You will also need to configure the location of the Medium Term Store (MTS). + +Here's how to change `archiver-appliance.nix`: + +``` {.nix filename="archiver-appliance.nix"} +{ + services.archiver-appliance.enable = true; + services.archiver-appliance.stores.lts.location = "/tmp/lts"; + services.archiver-appliance.stores.mts.location = "/tmp/mts"; +} +``` + +If you don't want to repeat yourself, +you can also change it like so: + +``` {.nix filename="archiver-appliance.nix"} +{ + services.archiver-appliance = { + enable = true; + stores.lts.location = "/tmp/lts"; + stores.mts.location = "/tmp/mts"; + }; +} +``` + +And now, +`nixos-rebuild test` should succeed: + +::: sourceCode +``` sourcecode +building the system configuration... +activating the configuration... +setting up /etc... +reloading user units for admin... +setting up tmpfiles +reloading the following units: dbus.service +the following new units were started: arch-lts-ArchiverStore.mount, + arch-mts-ArchiverStore.mount, arch-sts-ArchiverStore.mount, + mysql.service, tomcat.service +``` +::: + +From the message, +we can guess it started the Tomcat server running Archiver Appliance, +the MySQL (in fact, MariaDB) server, +and some mounted some partitions. +Fantastic! + +You can run the `systemctl` command to see if any systemd unit failed. + +In the default configuration, +Archiver Appliance and Tomcat are configured to output logs to journald. +You can see those logs by running: + +``` bash +journalctl -xeu tomcat.service +``` + +::: callout-note +Here are some details on what was done by EPNix' `services.archiver-appliance` NixOS module: + +- Creation of the Linux user and group `archappl` +- Installation and configuration of MariaDB: + - Creation of the `archappl` user, + with UNIX socket authentication + - Creation of the Archiver Appliance database + - Creation of the [various tables] in that database + - Giving access rights to this database for the `archappl` user +- Installation and configuration of Tomcat: + - Installation of the WAR files of Archiver Appliance + - Installation of the MariaDB connector and its dependencies + - Configuring the MariaDB connector to authenticate to the database + - Logging configuration to `journald` +- Configuring mounts so that: + - `/arch/lts` and `/arch/mts` are bind mounts to the configured locations, + with some added security options, + such as `nodev` and `noexec` + - Mounting `/arch/sts` as a new `tmpfs` +::: + +Tomcat runs by default under port 8080, +and NixOS has a firewall enabled by default. + +Change your `archiver-appliance.nix`: + +``` {.nix filename="archiver-appliance.nix"} +{ + services.archiver-appliance = { + enable = true; + stores.lts.location = "/tmp/lts"; + stores.mts.location = "/tmp/mts"; + + # New option: + openFirewall = true; + }; +} +``` + +and run `nixos-rebuild test`. +It will restart `firewall.service`, +but configured to allow incoming connection on port 8080. + +Check the IP address of your VM with `ip a`, +and open a browser to `http://:8080/mgmt/ui/index.html`. + +You have now configured Archiver Appliance on NixOS. + + [options reference]: ../options.md + [Archiver Appliance Details]: https://slacmshankar.github.io/epicsarchiver_docs/details.html + [various tables]: https://github.com/slacmshankar/epicsarchiverap/blob/master/src/main/org/epics/archiverappliance/config/persistence/archappl_mysql.sql + +## Next steps + +This VM configuration has some problems: + +- It stores the LTS and MTS in memory +- The size of the Short Term Store (STS) isn't configured +- Both "management" and "retrieval" URLs are accessible without authentication + +The following sections are some pointers to fix these issues. + +### Configuring partitions + +If you want to change the location of the LST or MTS, +you can change the value of the corresponding options: + +- `services.archiver-appliance.stores.lts.location` +- `services.archiver-appliance.stores.mts.location` + +But these values won't mean much if the configured directories are not backed by the appropriate hardware. + +As an example given by the [Archiver Appliance Details] page, +section "Architecture", +we can have the LTS backed by a NAS or SAN, +and the MTS backed by SSD or SAS storage. + +The way to do that if to configure the `fileSystems` NixOS option. +See the [File Systems NixOS documentation] for more information. + + [Archiver Appliance Details]: https://slacmshankar.github.io/epicsarchiver_docs/details.html + [File Systems NixOS documentation]: https://nixos.org/manual/nixos/stable/#ch-file-systems + +### Size of the short term store + +To configure the size of the short term store, +use the `services.archiver-appliance.stores.sts.size` option. + +For example: + +``` {.nix filename="archiver-appliance.nix"} +{ + services.archiver-appliance = { + enable = true; + stores.lts.location = "/tmp/lts"; + stores.mts.location = "/tmp/mts"; + + openFirewall = true; + + # New option: + stores.sts.size = "20g"; + }; +} +``` + +See the [`sts.size` option] in the reference for a more in-depth description. + + [`sts.size` option]: ../options.md#services.archiver-appliance.stores.sts.size + +### Restricting access + +Allowing access to `mgmt` URLs to anyone can be dangerous, +because it allows anyone to delete and archive PVs. + +To restrict access, +you can close the firewall and put an nginx server in front. + +You can configure the nginx server to disallow access to the URLs you want. +You can also configure nginx to require authentication. + +```{=html} + +```