biosdevname

[CMCDragonkai]

Hey @cstrahan

So what is the biosdevname used for? Is it related to this issue? hashicorp/vagrant#2614

Why does Vagrant need biosdevname, and is biosdevname not part of the standard packages?

[CMCDragonkai]

Also that whois thing still hasn't propagated. Any ideas? Apparently it got reverted. Where's your new PR?

[CMCDragonkai]

One more question, I used your mkpasswd patch, so it now finishes installing, but ssh doesn't start on startup even though I used services.openssh.enable. So that packer cannot finish the provisioning process. Do you know why sshd isn't launched upon rebooting?

Apparently it does start, but not accessible from Packer.

[cstrahan]

@CMCDragonkai: So what is the biosdevname used for? Is it related to this issue? hashicorp/vagrant#2614

Yeah. So, biosdevname isn't strictly necessary (my Vagrant guest plugin can fallback to other methods), but here's the idea:

I use biosdevname as a sort of supercharged ifconfig -a. I can get the hardware MAC address (where ifconfig might display a spoofed address), the name of each device, and the order of the devices based on their hardware configuration (e.g. the PCI slots and such). I then use this information from the vagrant guest support to know exactly what adapter configuration (from the Vagrant file) corresponds to which interface name.

So, for example, if you had these lines your Vagrantfile:

config.vm.network "private_network", type: "dhcp"        # Adapter #1
config.vm.network "private_network", ip: "192.168.50.4"  # Adapter #2
config.vm.network "public_network"                       # Adapter #3

Given the configuration above, the Vagrant guest plugin is invoked with a list of "networks", each with a given adapter number as seen above. What the plugin isn't given is the device name, which is what I actually need to put in the configuration file. Remember that you always have one more interface than what you have in your Vagrantfile - that interface is the NAT, which you don't want to configure. So I jump through these hoops to get the right names for each adapter number, and thus avoid accidentally configuring the NAT.

(To be extra explicit, the order that the kernel discovers interfaces isn't deterministic. For instance, if you have a server, and you see that you have an eth0 and eth1 interface, and you reboot, you're not guaranteed that eth0 and eth1 name the same hardware interfaces before and after rebooting - they can actually swap. This is why the predictable network naming scheme is really great: because the names are based on the physical port and slot numbers, you're guaranteed that a device named enp5s0 before reboot will point to the same hardware device after reboot.)

If biosdevname isn't found, I fallback to just running ifconfig -a and cross my fingers and hope that the kernel discovered each device in the same order that Vagrant added them, or, put another way, that the order that each interface shows up in ifconfig -a corresponds to directly to the order of the networks in the Vagrantfile.

While all of above might sound convincing, I actually discovered most of this stuff over the past week or so. So, if anyone sees any holes in my understanding, and can explain why some of these gyrations aren't necessary, I'd gladly simplify the code in the Vagrant plugin (and I always like to learn when I'm wrong).

You can check out the PR that added the above support: hashicorp/vagrant#3830

Also that whois thing still hasn't propagated. Any ideas? Apparently it got reverted. Where's your new PR?

Yeah, that got reverted because it made use of fetchFromGitHub, which hasn't yet landed in release-14.04.

I'll submit a compatible PR shortly.

One more question, I used your mkpasswd patch, so it now finishes installing, but ssh doesn't start on startup even though I used services.openssh.enable. So that packer cannot finish the provisioning process. Do you know why sshd isn't launched upon rebooting?

Apparently it does start, but not accessible from Packer.

Hmmm... I'm not sure why that would be, but I can keep it in the back of my mind. If you put your Packer definition online, I'll take a look and see if anything stands out.

[CMCDragonkai]

Oh right, so biosdevname is intended to be used with the Vagrant plugin you have. Is that plugin automatically ran, or do I have to configure it somewhere?

[cstrahan]

It will automatically run. You'll need to wait for the v1.6.3 release though (or manually install HEAD). It will write /etc/nixos/vagrant-{network,hostname}.nix, which are expected to be imported into the user's main configuration.nix. It's then up to the user to call nixos-rebuild switch (most likely from a provision script). I made it manual because I don't like the idea of automatically rebuilding a user's machine without any warning, and it's trivial enough to add a line to one's provision script.

[CMCDragonkai]

So that's what the empty vagrant-network and vagrant-hostname is for. A question, why does Nixos need this special treatment and not other OSes?

[cstrahan]

So that's what the empty vagrant-network and vagrant-hostname is for.

Yep!

A question, why does Nixos need this special treatment and not other OSes?

What special treatment are you referring to? I'm asking genuinely (not being snarky).

[CMCDragonkai]

Do other OSes need biosdevname as well? Because I was doing ctrl+f on the vagrant plugin, and I saw the conditional logic regarding biosdevname vs ifconfig -a, I was wondering if this just specific to nixos.

[cstrahan]

Other OSes need similar support to be as bullet proof. If you look at the other Vagrant guest OS plugins, most are pretty simple - they usually just prefix the adapter number with "eth" and get something like eth0, eth1, etc. That's already a problem for OSes like Red Hat and NixOS, because both use predictable network names (like the enp0s3 NIC in the VM I'm SSHd into at the moment) - and if you look at the Red Hat plugin, you'll see that they use biosdevname too for exactly this reason.

The simple "prefix the adapter number with eth rule" is also a bad assumption anyway, because eth0 and eth1 could swap after reboot. It might not happen much in practice, but it is possible. My code is little hairy because it tries to gracefully fallback in various ways, whereas the Red Hat plugin just assumes that biosdevname is always available.

So the simple plugins are simple because they don't support OSes with non-ethN style interface names, don't protect against the nondeterminism in how the Linux kernel detects/names interfaces, or because they make an assumption or two about what packages are installed. This isn't a failing on NixOS's part; it's universally tricky to (correctly) answer the question "what's the name of the NIC in the Nth slot?"

[CMCDragonkai]

Hey have you added biosdevname to the official Nix packages? That way we don't need to patch it in during our builds. It would be great if there was a way to autocreate the vagrant-hostname and vagrant-networks during the vagrant provisioning phase instead of the packer phase though.

[CMCDragonkai]

I'm getting a build error with regards to biosdevname:

patching file Makefile.am
patch unexpectedly ends in middle of line
Hunk #1 FAILED at 8.
1 out of 1 hunk FAILED -- saving rejects to file Makefile.am.rej

I tried using dos2unix and unix2dos but I don't think this is the makefile patch that you have in your current repository, its coming from somewhere else. But yes, biosdevname is not installable on my NixOS currently.

[cstrahan]

@CMCDragonkai

Is this with the package that I added upstream?

Can you share the Makefile.am.rej?

Here's something to watch out for: if you copied the patch file from the browser, tabs will probably be replaced with spaces, which will cause patch to fail because the actual whitespace doesn't match the expected.

[CMCDragonkai]

Hmm... that could be the source of the problem. But why is biosdevname not in the official channels? Also my box is working without biosdevname right now, so I'm semi happy. I deleted the Makefile thing and biosdevname so I don't have atm.

[cstrahan]

I think the package is just waiting for someone to backport, or waiting to be part of the next release.

[CMCDragonkai]

So it's already part of the master?

[cstrahan]

Yep!

[CMCDragonkai]

Should I put up an issue asking for backporting? Do you have a link to the specific commit for the biosdevname package in master branch?

[cstrahan]

That might not be a bad idea; here's the PR: NixOS/nixpkgs#2632

[CMCDragonkai]

If biosdevname becomes available, do we still need to create vagrant-hostname and vagrant-networks? I guess so since they are somewhat orthogonal.

Is there a way to declaratively create those files, rather than just making those files and expecting something else to fill them out later?

[CMCDragonkai]

It got backported! Still wondering about my above question though.

[CMCDragonkai]

Wow after studying MAC addresses, hexadecimals, IPV6, biosdevname, ethernet NICs and your pull request, I finally get it.

Ok so the reason we have those empty vagrant-hostname/vagrant-networks.nix files is so that Nix won't throw an error when we are importing those.

However the vagrant PR is only available in 1.6.3, which is now available.

Thanks, I'll try to use it and report back with any errors.

[cstrahan]

Ok so the reason we have those empty vagrant-hostname/vagrant-networks.nix files is so that Nix won't throw an error when we are importing those.

Yep! I might add a comment to the empty files to clarify - a sort of "this page left intentionally blank".

[CMCDragonkai]

Something's weird. I've done everything, but when I go and checkout what's inside vagrant-networks or vagrant-hostname it just contains what I wrote previously, and nothing else?

Is the biosdevname kicking in during the provisioning phase? Because it doesn't seem to be working.

[CMCDragonkai]

I did notice you changed some stuff especially here:

https://github.com/cstrahan/nix-packer/blob/master/nixos/configuration.nix#L18-L20

I was just putting in biosdevname as a package that needs to be installed. Any ideas?

[cstrahan]

Something's weird. I've done everything, but when I go and checkout what's inside vagrant-networks or vagrant-hostname it just contains what I wrote previously, and nothing else?

Is the biosdevname kicking in during the provisioning phase? Because it doesn't seem to be working.

The files should be written whenever you restart the box, e.g. vagrant halt; vagrant up. You'll still need to nixos-rebuild switch, perhaps using a one-liner provision script.

I did notice you changed some stuff especially here:

  environment.systemPackages =
    [ ]
    ++ lib.optionals (hasAttr "biosdevname" pkgs) [pkgs.biosdevname]; # Vagrant plugin support.

I just made the biosdevname package optional, in case the user has an old Nixpkgs version that lacks the package (like with NixOS 13.10).

[CMCDragonkai]

I've tried doing that, vagrant halt and vagrant up, and also nixos-rebuild switch. But nothing. My vagrant version is 1.6.3.

Here's my files:

vagrant-hostname.nix

{ config, pkgs, ... }: {}

# This file is intentionally blank, it will be overwritten by Vagrant

vagrant-network.nix

{ config, pkgs, ... }: {}

# This file is intentionally blank, it will be overwritten by Vagrant

Here's the import in my configuration.nix (which works of course)

    # Import hardware detection, this is auto generated, it will specify filesystem configuration
    imports = [
        ./hardware-configuration.nix
        ./vagrant-hostname.nix
        ./vagrant-network.nix
    ];

Bringing in the biosdevname:

    # Default system packages to be installed
    # biosdevname is required for vagrant to deterministically acquire the network adapters
    environment.systemPackages = with pkgs; [
        biosdevname
        openssl
        gnupg
    ];

In my provisioning script in Packer I have this line (which works as well):

nixos-rebuild switch --upgrade

My vagrant file is just this:

Vagrant.configure("2") do |config|

    # This Vagrantfile is only used for testing purposes. It is not intended for production.

    config.vm.box = "node"
    config.vm.box_url = "./node.box"

end

So nothing special.

The biosdevname is installed, and the files inside /etc/nixos/vagrant-network.nix and /etc/nixos/vagrant-hostname.nix is still the same as I wrote it.

Perhaps there's a permission issue? Maybe the overwriting is failing because of permissions? In that case, here's my users portion of my configuration.nix:

    users = {
        extraGroups = [
            {
                name = "vagrant";
            }
            {
                name = "vboxsf"; # Virtual Box group
            }
        ];
        extraUsers = {
            root = {
                openssh.authorizedKeys.keyFiles = [
                    ./vagrant.pub
                ];
            };
            vagrant = {
                description = "Vagrant User";
                password = "vagrant";
                group = "vagrant";
                extraGroups = [ "users" "vboxsf" "wheel" ];
                home = "/home/vagrant";
                createHome = true;
                useDefaultShell = true;
                openssh.authorizedKeys.keyFiles = [
                    ./vagrant.pub
                ];
            };
        };
    };

[cstrahan]

@CMCDragonkai

In the example Vagrantfile you gave, you're not specifying a hostname or any networks - perhaps Vagrant doesn't run those two "capabilities" (in Vagrant terms) when neither are specified. Would you mind setting those and seeing if that makes a difference?

[CMCDragonkai]

Hey I set this config.vm.network "private_network", type: "dhcp" in my vagrant file.

Then I checked the vagrant-hostname.nix and vagrant-network.nix.

I found this inside my vagrant-network.nix:

{ config, pkgs, ... }:
{
  networking.interfaces = [
    {
      name         = "enp0s8";
      prefixLength = 24;
    }
  ];
}

However vagrant-hostname.nix was still the same and nothing got changed. So it looks like biosdevname is working, but I'm guessing something else is meant to be triggering vagrant-hostname.nix?

[cstrahan]

@CMCDragonkai

From the NixOS manual:

networking.hostName

The name of the machine. Leave it empty if you want to obtain it from a DHCP server (if using DHCP).

Default: "nixos"

Declared by:
<nixpkgs/nixos/modules/tasks/network-interfaces.nix>

It should work if you put config.vm.hostname = "" in your Vangrantfile - let me know if it doesn't.