lib: add collectFiles and collectFilesAt

[Misterio77]

This is a lib function that recursively reads a directory, and returns them in the format expected by files/symlinks. This makes it possible to, for example, link all mods from a modpack, and still include additional mods.

[Misterio77]

An example on how it works:

nix-repl> lib.collectFilesAt modpack "config"
{
  "config/appliedenergistics2/common.json" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/appliedenergistics2/common.json";
  "config/bclib/main.json" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/bclib/main.json";
  "config/bclib/server.json" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/bclib/server.json";
  "config/byg/biomepedia.json5" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/byg/biomepedia.json5";
  "config/byg/client/biomepedia_inventory.json5" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/byg/client/biomepedia_inventory.json5";
  "config/charm.toml" = "/nix/store/ihsw8ifdmv7xjd3v1c7zqbfbbqkqvzar-packwiz-pack-0.2.18//config/charm.toml";
    # ...
}

{
  services.minecraft-servers.servers.cool-modpack = {
    enable = true;
    package = pkgs.fabricServers.fabric-1_18_2-0_14_9;
    symlinks = collectFilesAt modpack "mods" // {
      "mods/FabricProxy-lite.jar" = pkgs.fetchurl rec {
        pname = "FabricProxy-Lite";
        version = "1.1.6";
        url = "https://cdn.modrinth.com/data/8dI2tmqs/versions/v${version}/${pname}-${version}.jar";
        hash = "sha256-U+nXvILXlYdx0vgomVDkKxj0dGCtw60qW22EK4FhAJk=";
      };
    };
    files = collectFilesAt modpack "config" // {
      "config/server-specific.conf".value = {
        example = "foo-bar";
      };
    };
  };
}

[Infinidoge]

Would it be good to have collectFiles <path>, with collectFilesAt <path> <subdir> being something like (in pseudo-code) addPrefixToAttrKeys subdir (collectFiles "${path}/${subdir}")?

If files / symlinks let you do something like

files = {
  config = {
    "file1" = ...;
    "file2" = ...;
  };
};

to create

config/file1
config/file2

Then collectFiles would be quite useful, but as it stands the system doesn't do that that, so I'm not really sure what cases where you'd need collectFiles on its own, unless you want to do addPrefix "config" (collectFiles ./config) instead of collectFilesAt . "config"

[Misterio77]

Hey! Sorry for taking so long to respond.

Would it be good to have collectFiles , with collectFilesAt being something like (in pseudo-code) addPrefixToAttrKeys subdir (collectFiles "${path}/${subdir}")?

This is a very good idea and looks cleaner, will def get that in.

If files / symlinks let you do something like

This sounds super nice and actually kinda solves some concerns with composability, I think.

---

Sadly my current design for this PR has a major footgun.

We currently cleanup files when shutting down the server, but removing files/symlinks entries from your nixos config will NOT remove the files (as the service isn't stopped with the previous generation's stop script)! This isn't a major concern when using a single derivation for a big-ish directory, for example:

  services.minecraft-servers.servers.cool-modpack = {
    enable = true;
    package = pkgs.fabricServers.fabric-1_18_2-0_14_9;
    symlinks."mods" =  "${modpack}/mods"
};

Removing mods from modpack will propagate these changes to your server. Removing the entry altogether won't propagate (but you're likely always using some mods).

Once the entry is a single file, things get messy:

  services.minecraft-servers.servers.cool-modpack = {
    enable = true;
    package = pkgs.fabricServers.fabric-1_18_2-0_14_9;
    symlinks = collectFilesAt modpack "mods" // {
      "mods/FabricProxy-lite.jar" = pkgs.fetchurl rec {
        pname = "FabricProxy-Lite";
        version = "1.1.6";
        url = "https://cdn.modrinth.com/data/8dI2tmqs/versions/v${version}/${pname}-${version}.jar";
        hash = "sha256-U+nXvILXlYdx0vgomVDkKxj0dGCtw60qW22EK4FhAJk=";
      };
    };

As mentioned, the stop script will remove what's currently defined, not what was previously there. We'll have to special case this somehow. A few solutions I thought of:

• Keep track of managed files. Maybe a hidden file on the server root dir?
• Clean up more agressively. Always delete symlinks to nix store, or try and detect what is being managed some other way.

[Infinidoge]

I'm pretty sure it runs the stop script with the old configuration given stopIfChanged is set, making that a trivial issue to fix by just setting that.

[Misterio77]

SMH, for every common problem, there is already an elegant solution haha, had no idea about that. Will look into it.

[Misterio77]

I think stopIfChanged is pretty limited (e.g. it won't work if we prefer to reload instead), so I decided to implement a file-attrs based solution (#116).

[Misterio77]

Should be good to go.