elitak / nixos-infect Goto Github PK

Gets to /nix/var/nix/profiles/system/bin/switch-to-configuration boot in the infect function, then fails due to grub errors, appears to be trying to install to a i386 system when system is clearly x86_64

Since Digital Ocean seems to be a special case, i think the default option should be something else

Out of all providers I have tested Digital Ocean seems to be the only one who needs the special network configuration. For all of the other providers it seemed to be better to use the normal routine.
But since nixos-infect defaults to digitalocean, i always have to specify PROVIDER=something for all other providers. This would be unnecessary if nixos-infect wouldn't default to digitalocean.

I've skimmed through the script and it seems to me that it's preserving keys located in ~/.ssh/authorized_keys. I have my key in there for root but when I install nixOS with this script I cannot ssh back into it once the script is complete and the machine has performed a reboot.
I'm currently running this script as root user. Is there any additional config I need to add in order for this to work?

I recently got this error:

Nov 05 15:46:50 stage-1-init: [fsck.ext4 (1) -- /mnt-root/] fsck.ext4 -a /dev/vda1
Nov 05 15:46:50 stage-1-init: /dev/vda1 has unsupported feature(s): metadata_csum
Nov 05 15:46:50 stage-1-init: e2fsck: Get a newer version of e2fsck!
Nov 05 15:46:50 stage-1-init: fsck on /dev/vda1 failed.
Nov 05 15:46:50 stage-1-init: An error occurred in stage 1 of the boot process, which must mount the
Nov 05 15:46:50 stage-1-init: root filesystem on `/mnt-root' and then start stage 2.  Press one
Nov 05 15:46:50 stage-1-init: of the following keys:
Nov 05 15:46:50 stage-1-init: r) to reboot immediately
Nov 05 15:46:50 stage-1-init: *) to ignore the error and continue

Also the DNS server wasn't set right.

Hi, I'm trying to install nixos on https://networkpresence.com.au and it's close but the network doesn't seem to set up after reboot. Any tips about what I should look for in the base OS, and how to modify the script?

So far:

1. Starting from Ubuntu 20.04, set up ssh for root
2. Change hostname using hostnamectl set-hostname ... as Network Presence uses fqdn as hostname and this breaks the script
3. Grab nixos-infect, set NIX_CHANNEL=nixos-22.05, bash -x nixos-infect. Script runs and machine reboots.
4. Machine IP cannot be pinged and ssh timeouts
5. On VNC I can watch the boot sequence fly past, nothing notable - all OK's, gets to user management and throws up a login prompt on the console. Of course there's no credentials to log in with and it's not reachable with ssh.

Or is it the hostname changing step that makes the machine inaccessible?

It would be nice to support an optional argument to the script, an API key.

With the API key, it should be possible to attach an ISO with NixOS to the server and at least use the store on that ISO as cache, or install whatever version is on that ISO.

For Hetzner Cloud, this API https://docs.hetzner.cloud/#resources-server-actions-post-13 is relevant.

Attaching an ISO like that would make it possible to read and write directly over SSD and local network, potentially making the installation very fast.

I tried on a VPS 2016 SSD 3 from OVH, each of these :

• Debian 8
• Debian 9
• Ubuntu 16.04

And each time I run the script, then installation + reboot are going well. But after that, I can't connect to the VPS anymore. I accessed it by KVM to debug and it seems that the VPS just get disconnected from network.

networking.interfaces.<name?>.ipv4.addresses option in /etc/nixos/networking.nix will trigger an error and won't let initial build to finish. According to https://nixos.org/nixos/options.html#ip4 it should be networking.interfaces.<name?>.ipv4. I manually edited that file by removing ".addresses" and was able to complete the build.

DigitalOcean, Ubuntu16.04.

• #61
  Yeah, i know that relying on external tool in just bash script is a bad idea. Any change in it's source code can lead to unworkable nixos-infect. But it seems to do good work. Just try it
• I've experienced connection issues, which are probably firewall related. By default firewall seems blocking all incoming connections.
• add support for external configuration. If specified path in option, copy file or folder content to /etc/nixos/specified

In such scenario we have files generated by nixos-generate-config, files that enable networking on port 22 and pointing to custom configuration if path specified in option.

First of all i want to ask you if you agree with me. This project have a lot potential, but sometimes feels like glueing some parts. If nixos-generate-config is not enough to fulfill some providers (like digital ocean) then maybe we can create clean default script which usually would work out of the box on most providers/devices. If custom provider is specified use another method or (better) script to handle it?

What are your experiences in that topic? Are maintainers open to accept such contribution even in different branch (as we lose current compatibility progress)? It would be nice to easily support additional hosting providers without mess in the code.

I ssh'd into my freshly created Azure Ubuntu 18.04 VM, run this script with the default options as in the Vultr section and observed that it rendered the system unbootable. I'm not sure what would cause this, though through some googling it seems to also happen to people upgrading ubuntu to 19.10, so possibly a mismatch between grub versions or something like that?

I'm using this because nixops has dropped Azure support so I want to just use the VM standalone.

Parsing ip output is error prone and provides lot of unnecessary duplication. All the data that is necessary to configure should be (and in fact is) provided by means of droplet metadata, exposed via convenient REST API.

eth0_ip4s=$(ip address show dev "$eth0_name" | grep 'inet ' | sed -r 's|.*inet ([0-9.]+)/([0-9]+).*|{ address="\1"; prefixLength=\2; }|')
eth0_ip6s=$(ip address show dev "$eth0_name" | grep 'inet6 ' | sed -r 's|.*inet6 ([0-9a-f:]+)/([0-9]+).*|{ address="\1"; prefixLength=\2; }|' || '')

eth0_ip4s=$(curl http://169.254.169.254/metadata/v1/interfaces/public/0/ipv4/address)
eth0_ip6s=$(curl http://169.254.169.254/metadata/v1/interfaces/public/0/ipv6/address)

Existence and lack of specific keys gives one great inspection abilities with minimal effort.

eth1_name=$(ip address show | grep '^3:' | awk -F': ' '{print $2}')||true
if [ -n "$eth1_name" ];then

if curl --fail http://169.254.169.254/metadata/v1/interfaces/public/1 ; then

The interfaces endpoint contains precise information on amount and intention of interfaces:

# curl http://169.254.169.254/metadata/v1/interfaces/
public/
private/
# curl http://169.254.169.254/metadata/v1/interfaces/private/
0/
# curl http://169.254.169.254/metadata/v1/interfaces/private/0/ipv4/address
10.135.53.177

ubuntu 20.04 22.04 23.03 FAIL
debian 11 SUCCESS

If you create a digital ocean VM with 2 interfaces (one public and one private), the configuration by nixos-infect is as follows:

interfaces = {
  eth0 = {
    ip4 = [
      { address="1.2.3.4"; prefixLength=18; }
      { address="10.15.0.8"; prefixLength=16; }
    ];
  };
  # eth1 is for private networking or something?
  eth1.useDHCP = false;
};

It should instead be:

interfaces = {
  eth0 = {
    ip4 = [
      { address="1.2.3.4"; prefixLength=18; }
    ];
  };
  eth1 = {
    ip4 = [
      { address="10.15.0.8"; prefixLength=16; }
    ];
  };
};

When /root/.ssh/authorized_keys is an empty file the infect script will look no further and not install any ssh keys for root in the new NixOS.

It requires export doNetConf=y, so script should be modified to support ssdnodes PROVIDER. Without network configuration it wasn't working.

With little modify (remove PROVIDER):

#cloud-config
write_files:
- path: /etc/nixos/host.nix
  permissions: '0644'
  content: |
    {pkgs, ...}:
    {
      environment.systemPackages = with pkgs; [ vim ];
    }
runcmd:
  - curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIXOS_IMPORT=./host.nix NIX_CHANNEL=nixos-21.05 bash 2>&1 | tee /tmp/infect.log

Source: https://nixos.wiki/wiki/Install_NixOS_on_Scaleway_X86_Virtual_Cloud_Server

Tested and work on instances:

• DEV-1S
• DEV-1M

Also, it must work properly with:

• DEV-1L
• DEV-1XL

Tested with images:

• Ubuntu Bionic - work
• Ubuntu Focal Fossa - doesn't work
• rockylinux 8 - doesn't work

Now hangs at "Welcome to Grub2"

only obvious error

+/nix/var/nix/profiles/system/bin/switch-to-configuration boot
perl: warning: Setting locale failed.
perl: warning: Please check that your locale settings:
    LANGUAGE = (unset),
    LC_ALL = (unset),
    LANG = "en_GB.UTF-8"
    are supported and installed on your system.
perl: warning: Falling back to the standard locale ("C").

Ubuntu 20.04-slim worked
Ubuntu 22.04-slim worked

Using image Ubuntu Server 22.04 LTS
Boot message:

It seemed a bit crazy to me, but actually it is part of the cloudinit standard to put scripts inside /root/authorized_keys to prevent direct root login for example and return a message to the user who tried.

A typical cloud init generated authrized_keys can look like that:

no-port-forwarding,no-agent-forwarding,no-X11-forwarding,command="echo 'Please login as the user \"ubuntu\" rather than the user \"root\".';echo;sleep 10" ssh-ed25519 AAAAdeadbeefxxxxxxxxxx.......

Since nixos-infect copies all of that into users.users.root.openssh.authorizedKeys.keys, the nixos installation fails. I'm not sure what would be the best way to modify the current regex to capture that case. I would appreciate some help with the grep magic.

The script seems to work fine in the sense that it installs nixos, and reboot, and nixos starts fine.

I can't contact the server though, so the network is down, or possibly ssh is not running.

when I try to run it

useradd: user 'nixbld10' already exists
+ true
+ curl -L https://nixos.org/nix/install
+ /bin/bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  4046  100  4046    0     0  31609      0 --:--:-- --:--:-- --:--:-- 31609
downloading Nix 2.7.0 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.7.0/nix-2.7.0-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.YEF6MN1DW3'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 27.4M  100 27.4M    0     0   175M      0 --:--:-- --:--:-- --:--:--  175M
Note: a multi-user installation is possible. See https://nixos.org/manual/nix/stable/installation/installing-binary.html#multi-user-installation
warning: installing Nix as root is not supported by this script!
performing a single-user installation of Nix...
copying Nix to /nix/store...

error: profile '/nix/var/nix/profiles/default' is incompatible with 'nix-env'; please use 'nix profile' instead
/tmp/nix-binary-tarball-unpack.YEF6MN1DW3/unpack/nix-2.7.0-x86_64-linux/install: unable to install Nix into your default profile

I've tried to convert the command to nix profile

nix profile install \
    --profile /nix/var/nix/profiles/system \
    github:nixos/nixpkgs/nixos-20.09#system

but this won't work. Not sure how to retreive the system attribute.

What advantages/disadvantages does this script have over nixos-in-place?
Why was it created?

After running nix-infect on a clean Hetzner Cloud VPS, the system comes back after reboot and works fine, except ipv6.

It seems the ipv4 address is provided through DHCP, but ipv6 is static. It would be nice if there is an option to only generate an ipv6 network configuration.

Complete log:

stian@extravm:~$   curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-20.09 bash -x
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0+ set -e -o pipefail
100  9291  100  9291    0     0  38712      0 --:--:-- --:--:-- --:--:-- 38712
+ '[' '' = digitalocean ']'
+ prepareEnv
+ for grubdev in /dev/vda /dev/sda
+ [[ -e /dev/vda ]]
+ break
++ awk '{print $1;}'
++ grep 'on / type'
++ mount
+ rootfsdev=/dev/vda1
+ export USER=root
+ USER=root
+ export HOME=/root
+ HOME=/root
+ mkdir -p -m 0755 /nix
mkdir: cannot create directory ‘/nix’: Permission denied

stian@extravm:~$ sudo su
root@extravm:/home/stian#   curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-20.09 bash -x
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0+ set -e -o pipefail
100  9291  100  9291    0     0   181k      0 --:--:-- --:--:-- --:--:--  181k
+ '[' '' = digitalocean ']'
+ prepareEnv
+ for grubdev in /dev/vda /dev/sda
+ [[ -e /dev/vda ]]
+ break
++ awk '{print $1;}'
++ grep 'on / type'
++ mount
+ rootfsdev=/dev/vda1
+ export USER=root
+ USER=root
+ export HOME=/root
+ HOME=/root
+ mkdir -p -m 0755 /nix
+ makeSwap
++ mktemp /tmp/nixos-infect.XXXXX.swp
+ swapFile=/tmp/nixos-infect.2h1uC.swp
+ dd if=/dev/zero of=/tmp/nixos-infect.2h1uC.swp bs=1M count=1024
1024+0 records in
1024+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.28505 s, 836 MB/s
+ chmod 0600 /tmp/nixos-infect.2h1uC.swp
+ mkswap /tmp/nixos-infect.2h1uC.swp
Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes)
no label, UUID=c45c4811-eaf4-475a-8417-fb96c966389d
+ swapon -v /tmp/nixos-infect.2h1uC.swp
swapon: /tmp/nixos-infect.2h1uC.swp: found signature [pagesize=4096, signature=swap]
swapon: /tmp/nixos-infect.2h1uC.swp: pagesize=4096, swapsize=1073741824, devsize=1073741824
swapon /tmp/nixos-infect.2h1uC.swp
+ checkEnv
+ which dnf
+ which bzcat
/bin/bzcat
+ which xzcat
/usr/bin/xzcat
+ which curl
/usr/bin/curl
++ whoami
+ [[ root == \r\o\o\t ]]
+ req curl
+ type curl
+ req bzcat
+ type bzcat
+ req xzcat
+ type xzcat
+ req groupadd
+ type groupadd
+ req useradd
+ type useradd
+ req ip
+ type ip
+ req awk
+ type awk
+ req cut
+ type cut
+ makeConf
+ [[ -e /etc/nixos/configuration.nix ]]
+ mkdir -p /etc/nixos
+ local 'IFS=
'
+ for trypath in /root/.ssh/authorized_keys $HOME/.ssh/authorized_keys
+ [[ -r /root/.ssh/authorized_keys ]]
+ for trypath in /root/.ssh/authorized_keys $HOME/.ssh/authorized_keys
+ [[ -r /root/.ssh/authorized_keys ]]
+ local network_import=
+ [[ -n '' ]]
+ cat
++ hostname
++ read -r line
++ echo -n '
    "" '
++ read -r line
+ cat
+ [[ -n '' ]]
+ true
+ infect
+ groupadd nixbld -g 30000
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 1' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld1
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 2' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld2
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 3' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld3
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 4' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld4
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 5' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld5
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 6' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld6
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 7' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld7
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 8' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld8
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 9' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld9
+ for i in {1..10}
++ which nologin
+ useradd -c 'Nix build user 10' -d /var/empty -g nixbld -G nixbld -M -N -r -s /usr/sbin/nologin nixbld10
+ /bin/bash
+ curl -L https://nixos.org/nix/install
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0
100  2699  100  2699    0     0  13841      0 --:--:-- --:--:-- --:--:-- 13841
downloading Nix 2.3.10 binary tarball for x86_64-linux from 'https://releases.nixos.org/nix/nix-2.3.10/nix-2.3.10-x86_64-linux.tar.xz' to '/tmp/nix-binary-tarball-unpack.L9vjWSOaVs'...
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 16.4M  100 16.4M    0     0  62.8M      0 --:--:-- --:--:-- --:--:-- 62.8M
Note: a multi-user installation is possible. See https://nixos.org/nix/manual/#sect-multi-user-installation
warning: installing Nix as root is not supported by this script!
performing a single-user installation of Nix...
copying Nix to /nix/store......................................
installing 'nix-2.3.10'
building '/nix/store/3988ni0kzan1yl3n9a67vk470flgnqi2-user-environment.drv'...
created 6 symlinks in user environment
unpacking channels...
created 1 symlinks in user environment
modifying /root/.profile...

Installation finished!  To ensure that the necessary environment
variables are set, either log in again, or type

  . /root/.nix-profile/etc/profile.d/nix.sh

in your shell.
+ source /root/.nix-profile/etc/profile.d/nix.sh
++ '[' -n /root ']'
++ '[' -n root ']'
++ NIX_LINK=/root/.nix-profile
++ export NIX_PATH=/root/.nix-defexpr/channels
++ NIX_PATH=/root/.nix-defexpr/channels
++ export 'NIX_PROFILES=/nix/var/nix/profiles/default /root/.nix-profile'
++ NIX_PROFILES='/nix/var/nix/profiles/default /root/.nix-profile'
++ '[' -e /etc/ssl/certs/ca-certificates.crt ']'
++ export NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
++ NIX_SSL_CERT_FILE=/etc/ssl/certs/ca-certificates.crt
++ '[' -n '' ']'
++ export PATH=/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
++ PATH=/root/.nix-profile/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
++ unset NIX_LINK
+ [[ -z nixos-20.09 ]]
+ nix-channel --remove nixpkgs
uninstalling 'nixpkgs-21.05pre283043.f0efbe21f9a'
building '/nix/store/xsk9wjy0dpp7w7d3r3bkzxzaklqdzq6q-user-environment.drv'...
created 0 symlinks in user environment
+ nix-channel --add https://nixos.org/channels/nixos-20.09 nixos
+ nix-channel --update
unpacking channels...
created 1 symlinks in user environment
+ export NIXOS_CONFIG=/etc/nixos/configuration.nix
+ NIXOS_CONFIG=/etc/nixos/configuration.nix
+ nix-env --set -I nixpkgs=/root/.nix-defexpr/channels/nixos -f '<nixpkgs/nixos>' -p /nix/var/nix/profiles/system -A system
error: The option value `networking.hostName' in `/etc/nixos/configuration.nix' is not of type `string matching the pattern ^$|^[[:alnum:]]([[:alnum:]_-]{0,61}[[:alnum:]])?$'.
(use '--show-trace' to show detailed location information)

hostname:

root@extravm:/home/stian# hostname
extravm.stianlagstad.no

Should I change the hostname? Thanks!

Machine info:

Operating System | Ubuntu 20.04 x64
-- | --
IPv6 Address | 1
Disk Space | 7 GB
Bandwidth | 1 TB
Memory | 512 MB
Swap | 256 MB
Virtualization Type | (KVM)

There seems to be a permission error trying to create the per-user directory for non root users.

mkdir: cannot create directory ‘/nix/var/nix/gcroots/per-user/fred’: Permission denied
stat: cannot stat '/nix/var/nix/gcroots/per-user/fred': No such file or directory
WARNING: bad ownership on /nix/var/nix/gcroots/per-user/fred

Any help would be greatly appreciated. Thanks. I followed https://chris-martin.org/2016/nixos-on-digitalocean to install it.

Thanks

Hey,

just used this tool on a VM on Hetzner for which I didn't configured an IPv4 address. The generated networking.nix is invalid and can't be applied because there is an empty /32 IPv4 route.

Generated networking.nix:

{ lib, ... }: {
  # This file was populated at runtime with the networking
  # details gathered from the active system.
  networking = {
    nameservers = [ "8.8.8.8"
 ];
    defaultGateway = "";
    defaultGateway6 = {
      address = "fe80::1";
      interface = "eth0";
    };
    dhcpcd.enable = false;
    usePredictableInterfaceNames = lib.mkForce false;
    interfaces = {
      eth0 = {
        ipv4.addresses = [
          { address="100.64.194.249"; prefixLength=32; }
        ];
        ipv6.addresses = [
          { address="2a01:4f8:c0c:65c2::1"; prefixLength=64; }
{ address="fe80::9400:2ff:fe27:2e0b"; prefixLength=64; }
        ];
        ipv4.routes = [ { address = ""; prefixLength = 32; } ];
        ipv6.routes = [ { address = "fe80::1"; prefixLength = 128; } ];
      };
      
    };
  };
  services.udev.extraRules = ''
    ATTR{address}=="96:00:02:27:2e:0b", NAME="eth0"
    
  '';
}

This is what the interfaces look like on a fresh IPv6 only machine booted into their stock Ubuntu:

# ip a
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000
    link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
    inet 127.0.0.1/8 scope host lo
       valid_lft forever preferred_lft forever
    inet6 ::1/128 scope host 
       valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc fq_codel state UP group default qlen 1000
    link/ether 96:00:02:27:4a:20 brd ff:ff:ff:ff:ff:ff
    altname enp0s3
    altname ens3
    inet 100.66.8.195/32 metric 100 scope global dynamic eth0
       valid_lft 86280sec preferred_lft 86280sec
    inet6 2a01:4f9:c012:859b::1/64 scope global 
       valid_lft forever preferred_lft forever
    inet6 fe80::9400:2ff:fe27:4a20/64 scope link 
       valid_lft forever preferred_lft forever

# ip -4 r
169.254.169.254 via 172.31.1.1 dev eth0 proto dhcp src 100.66.8.195 metric 100 
172.31.1.1 dev eth0 proto dhcp scope link src 100.66.8.195 metric 100 

# ip -6 r
::1 dev lo proto kernel metric 256 pref medium
2a01:4f9:c012:859b::/64 dev eth0 proto kernel metric 256 pref medium
fe80::/64 dev eth0 proto kernel metric 256 pref medium
default via fe80::1 dev eth0 proto static metric 1024 onlink pref medium

Happy to provide more information and help debug this if needed.

Best,
Felix

nevermind i figured it out.

old system

debian 11

boot error

Booting the kernel .

<<<NixOS Stage 1 >>>

loading module virtio_balloon...
loading module virtio_console...
loading module virtio_rng...
loading module nvme...
loading module btrfs...
loading module dm_mod...
running udev...
Starting version 250.4
starting device mapper and LVM...
Scanning for Btrfs filesystems
mounting /dev/vda2 on /...
state 2 init script (/mnt-root//nix/store/zr9pd87311q94dp3n103r6chx3dk64sw-nixos-system-debian-22.05.3935.b3a8f7ed267/init) not found

An error occurred in stage 1 of the boot process, which must mount the root filesystem on '/mnt-root' and then start stage 2. Press one of the following keys:

    r) to reboot immediately
    *) to ignore the error and continue

I think that if you mkdir /nixosroot and run the install on that and then reboot with root bind-mounted to /nixosroot, everything will work smoothly.

Today I have been trying to infect Ubuntu 20.04 images from Contabo VPS.

When running it I had to tweak two things. The first was that by default, Contabo assigns a hostname such as vmi12345678.contaboserver.net. This is not allowed in NixOS as it is a fully qualified name, and it is not possible to have one. I just did the change of hostname and it worked. I am not sure about what consequences it has but to be honest I don't care enough to take a look to it.

The other issue I saw was that when trying to do the step mv -v /boot /boot.bak was that the device was mounted, thus I got the error

mv: cannot move '/boot' to '/boot.bak': Device or resource busy.

I just had commented the line and successfully run it. But before adding a PR selecting the provider as OK I was wondering some things.

1. Shouldn't all the steps related to the copy of /boot only be done if the system is EFI?
2. In case that it has to also be done it is safe to do so? (To put as a notice in the PR)

Thanks

Hello,

I just installed nixos using nixosinfect but I have a problem: now when I start my vps it asks me to login but my old credentials are no longer working and I don't know the new ones. I tried to change the password using systemd.debug-shell but I can't get to tty9 using VNC.
How should I do?

Thank you very much,
Thomas

That is awesome to see, should be added to README!

More specifically, when run on a machine with non-EFI boot where /boot is a mount point, you get the following error:

mv: cannot move '/boot' to '/boot.bak': Device or resource busy

I ran into this error on a contabo VPS but I suspect the kind of setup that causes it is not super rare.

I get this error while trying to use with a digital ocean vps with ubuntu image. any help is appreciated.

/nix/store/r7bnab4vmgz3ds4z0rbrfjhm20j3n1lk-grub-2.x-2015-11-16/sbin/grub-install: warning: this GPT partition label contains no BIOS Boot Partition; embedding won't be possible.
/nix/store/r7bnab4vmgz3ds4z0rbrfjhm20j3n1lk-grub-2.x-2015-11-16/sbin/grub-install: error: embedding is not possible, but this is required for cross-disk install.
/nix/store/w345fzhsr2ckzhazmxr3lc906glnxqiy-install-grub.pl: installation of GRUB on /dev/vda failed

I just tried running the script and got an error about isX86_64 that causes it to fail on my DreamHost Ubuntu 22.04 installation.

It appears adding () to the function definition at https://github.com/elitak/nixos-infect/blame/c364d49d9d39d3c85348c05b2e6467a3e8da92ca/nixos-infect#L177 lets me move forward.

Hello! Would you be interested in re-homing this project under the nix-community umbrella? Thank you for nixos-infect!

Relevant parts of nixos-infect run on osuosl research lab openstack cluster:

++ mktemp /tmp/nixos-infect.XXXXX.swp

• swapFile=/tmp/nixos-infect.qAMel.swp
• dd if=/dev/zero of=/tmp/nixos-infect.qAMel.swp bs=1M count=1024
  1024+0 records in
  1024+0 records out
  1073741824 bytes (1.1 GB, 1.0 GiB) copied, 1.38329 s, 776 MB/s
• chmod 0600 /tmp/nixos-infect.qAMel.swp
• mkswap /tmp/nixos-infect.qAMel.swp
  Setting up swapspace version 1, size = 1024 MiB (1073737728 bytes)
  no label, UUID=4a61f628-c714-43ed-8baf-046844d4e328
• swapon -v /tmp/nixos-infect.qAMel.swp
  swapon: /tmp/nixos-infect.qAMel.swp: found signature [pagesize=4096, signature=swap]
  swapon: /tmp/nixos-infect.qAMel.swp: pagesize=4096, swapsize=1073741824, devsize=1073741824
  swapon /tmp/nixos-infect.qAMel.swp
  swapon: /tmp/nixos-infect.qAMel.swp: swapon failed: Invalid argument

Underlying tested distro:

root@ci-runner ~]# cat /etc/os-release
NAME=Fedora
VERSION="30 (Thirty)"
ID=fedora
VERSION_ID=30
VERSION_CODENAME=""
PLATFORM_ID="platform:f30"
PRETTY_NAME="Fedora 30 (Thirty)"

The script completes and succeeds when I comment out the makeSwap and removeSwap, hence it's not mandatory for the infection to succeed.

Should we make those optional or at least failing gracefully in?
Thanks.

I just tried the infect script on a hetzner cloud VM. When i set PROVIDER to something else than digitalocean, the installation just works fine and also afterwards dns resolution works without problems.
Maye the readme should be changed to use PROVIDER=hetznercloud instead of digitalocean and the nameserver hack can be removed.

I like to use SSH certificate authorities to authorize all my computers at once. The regex in nixos-infect strips off the "cert-authority" prefix, making authentication impossible:

Here's a sample authorized_keys line for this sort of authentication:

cert-authority ssh-ed25519 00000000000000000000000000000000000000000000000000000000000000000000 2020-11-23

After writing all of this, I noticed #52. I'm happy to work in some of the changes from that PR too, but the changes there feel excessive (and I wouldn't want to pull in the shell-script changes).

Frankly, documentation on this script is a bit of a mess.

There's haphazard attempts at showing how to get hosted platforms working with the script,
but two of the three are the same incantation but with slightly different logging params.

There's no note of how one should log in after the install runs to completion.

The nix-channel in use is now out of date.

There's nothing fundamentally unfixable though.
I am working on a fork whereby the README would be restructured.

Changes I am considering for the README:

• DigitalOcean becomes the default platform.
• Other platforms are mentioned "lumped together".
• Update the suggested channel to 20.09 (soon to be 21.03, I suppose - but that's an easy fix)
• Generic section about deployment notes
  This gives us an area to talk about the various "gotchas", but so far it'd contain:
  • Root user password vs authentication key (which has been brought up before as an issue, but was not documented)
  • grub bootloader issues (I personally needed to set boot.loader.grub.devices to nodev before nixos-rebuild switch would work)
• A table containing hosts/os/notes (which allows us to add per-provider notes without bloating the wording of the README)

Running nixos-infect on an Ampere instance on Oracle Cloud (Ubuntu 22.04), I get this error:

building '/nix/store/xa5k8n6zy9za0canz5b0ffhnm1nayi2f-linux-5.15.77-modules-shrunk.drv'...
kernel version is 5.15.77
root module: virtio_net
  builtin dependency: virtio_net
root module: virtio_pci
  builtin dependency: virtio_pci
root module: virtio_mmio
  builtin dependency: virtio_mmio
root module: virtio_blk
  builtin dependency: virtio_blk
root module: virtio_scsi
  copying dependency: /nix/store/nql345d4xxv4kcgg9qv0g8z65346jv9a-linux-5.15.77-modules/lib/modules/5.15.77/kernel/drivers/scsi/virtio_scsi.ko.xz
root module: 9p
  builtin dependency: 9p
root module: 9pnet_virtio
  builtin dependency: 9pnet_virtio
root module: ata_piix
  copying dependency: /nix/store/nql345d4xxv4kcgg9qv0g8z65346jv9a-linux-5.15.77-modules/lib/modules/5.15.77/kernel/drivers/ata/ata_piix.ko.xz
root module: uhci_hcd
  builtin dependency: ehci_pci
  copying dependency: /nix/store/nql345d4xxv4kcgg9qv0g8z65346jv9a-linux-5.15.77-modules/lib/modules/5.15.77/kernel/drivers/usb/host/uhci-hcd.ko.xz
root module: vmw_pvscsi
modprobe: FATAL: Module vmw_pvscsi not found in directory /nix/store/nql345d4xxv4kcgg9qv0g8z65346jv9a-linux-5.15.77-modules/lib/modules/5.15.77
error: builder for '/nix/store/xa5k8n6zy9za0canz5b0ffhnm1nayi2f-linux-5.15.77-modules-shrunk.drv' failed with exit code 1
error: 1 dependencies of derivation '/nix/store/ijx2aa13r24wrln9ycavdnvyc9vry0kc-stage-1-init.sh.drv' failed to build
error: 1 dependencies of derivation '/nix/store/j0iq3cspwck28qav6r015rdwi1651j9i-initrd-linux-5.15.77.drv' failed to build
building '/nix/store/w8kr4nakaxdc37z2lrcglxgwn05jwjza-localhost-hosts.drv'...
error: 1 dependencies of derivation '/nix/store/wbz9y2ldjyynsaplqp60d6ir8z0sh8b9-nixos-system-djmuk2-22.05.4033.ebf65554b18.drv' failed to build

Initially I thought it was caused by #115 but I got the same error using an older version of nixos-infect, and also when manually removing the reference to vmw_pvscsi from the latest version.
This is caused by the mentioned issue - I was still using the hardware configuration generated by the first run.

I tried nixos-unstable, nixos-22.05 and nixos-22.11 channels, and get the same error one all three.

I'm not 100% sure this is an nixos-infect issue, but reporting in case it helps someone else.

After nixos-infect completed from an Ubuntu 18.03 Digital Ocean droplet, the server's DNS was broken. I had to update the nameserver in networking.nix:

nameservers = [
      "8.8.8.8"
    ];

This sets /etc/resolv.conf, so I had to update the entry there too.

First of all thanks for this very useful script.

It was my experience that running the command as documented in the README on a DigitalOcean host:

curl https://raw.githubusercontent.com/elitak/nixos-infect/master/nixos-infect | NIX_CHANNEL=nixos-20.09 bash -x

... puts the system into a state with no networking enabled, which kept me puzzled for quite a while. I gather that this is how the script is intended to operate when PROVIDER is unset but think it could be documented a bit more clearly that PROVIDER is required for Digital Ocean (and not just when running through cloud-init).

Thank you for publishing nixos-infect! It is working well on my digitalocean droplets and I am happy to learn about that NIXOS_LUSTRATE mechanism.

I forked this repo and went to make sure I had a no-warranty clause, and I noticed it didn't have a license. If you are willing, could you please add one? Thanks a lot.

Recently I was running into issues where nixos infect generated a config that rendered the system unbootable.
An installation via official nixos minimal ISO installer worked.

So why don't we just use the installer's way of generating the config?

To make nixos-generate-config available in a non-nixos system, one can use the following derivation:

let
  pkgs = import <nixpkgs> {};
  machine = import "${pkgs.path}/nixos" {
    configuration = { ... }: {
      imports = [ "${pkgs.path}/nixos/modules/installer/tools/tools.nix" ];
    };
  };
in
(builtins.filter (p: p.name == "nixos-generate-config") machine.config.environment.systemPackages)

I did not test this yet on a non-nixos system.
Any opinions?

On the always free tier machine
Image: Oracle-Linux-7.9-aarch64-2021.10.20-0
Shape: VM.Standard.A1.Flex

Execution of the "nixos-infect" script is successful
But even right after the successful completion of the script

$ nixos-rebuils switch
leads to the following error:
building Nix...
building the system configuration...
updating GRUB 2 menu...
activating the configuration...
setting up /etc...
reloading user units for root...
setting up tmpfiles
warning: the following units failed: dev-sda2.swap

× dev-sda2.swap - Swap Partition
     Loaded: loaded (/run/systemd/generator.late/dev-sda2.swap; generated)
     Active: failed (Result: exit-code) since Sun 2021-12-12 13:03:32 UTC; 32ms ago
       What: /dev/sda2
       Docs: man:systemd-gpt-auto-generator(8)
         IP: 0B in, 0B out
        CPU: 4ms

Dec 12 13:03:32 instance-20211212-1554 systemd[1]: Activating swap Swap Partition...
Dec 12 13:03:32 instance-20211212-1554 swapon[1242]: swapon: /dev/sda2: swap format pagesize does not match. (Use --fixpgsz to reinitialize it.)
Dec 12 13:03:32 instance-20211212-1554 swapon[1242]: swapon: /dev/sda2: swapon failed: Invalid argument
Dec 12 13:03:32 instance-20211212-1554 systemd[1]: dev-sda2.swap: Swap process exited, code=exited, status=255/EXCEPTION
Dec 12 13:03:32 instance-20211212-1554 systemd[1]: dev-sda2.swap: Failed with result 'exit-code'.
Dec 12 13:03:32 instance-20211212-1554 systemd[1]: Failed to activate swap Swap Partition.
warning: error(s) occurred while switching to the new configuration

After a bit of studying I've attempted to recretate (?) swap file

$ swapon -af /dev/sda2
swapon: /dev/sda2: swap format pagesize does not match.
swapon: /dev/sda2: reinitializing the swap.
mkswap: /dev/sda2: warning: wiping old swap signature.
Setting up swapspace version 1, size = 8 GiB (8589930496 bytes)
no label, UUID=a4f9827d-d978-4837-ab36-31d09c83ad7d

$ nixos-rebuild switch
building Nix...
building the system configuration...
updating GRUB 2 menu...
activating the configuration...
setting up /etc...
reloading user units for root...
setting up tmpfiles

and all seems well

I don't know enough to understand whether this is something that can be addressed via script modification, or something that is inevitable.
In any case note on this experience could help anyone who might encounter similar issue

P.S. I have a lot of gratitude to the authors and supporters of this script, it is marvelous!

Important things to keep in mind:

• As of now, you can't reattach a volume as a boot volume once you format it! Once you detach and format it, you can only create a new instance with that volume. If you do try to attach it, it will show "attaching" for a few minutes, then detach automatically.
• RHEL-based ARM distros have an old kernel, which has a different page size, meaning btrfs volumes which work there won't work on new kernels. Also, you can't pivot_root into Nix on their RHEL-based image, and chroot doesn't work in any case as it breaks Nix sandboxing which can't be disabled. Furthermore, their RHEL-based images use a complicated partition scheme with 3 partitions and LVM. Basically, use Ubuntu.
• Don't forget to set partition types and labels properly
• If your system fails in initrd, open your VM's cloud console to find out what the issue is
• You will need to use multiple VMs. Make sure you create all of them in the same AD, or they won't be able to interact with each other! They also must all have the same architecture as the VM you will end up using with the btrfs volume.

With that in mind, rough operation order:

1. Create a vm1 with Ubuntu
2. Create a second vm2 (Ubuntu as well)
3. Terminate vm2, keep its boot volume
4. Attach vm2 boot volume to vm1
5. Resize the vm2 efi partition by moving the ext partition further and recreating the efi partition (optional, the default of 100mb is just barely enough), btrfs-convert the vm2 root partition. Dont forget to label the new partitions properly (UEFI and cloud-root or something like that)
6. Chroot into vm2 boot volume, patch fstab to change ext4 into btrfs, reinstall grub (don't mistake the drive!). Now you are done with vm1, detach vm2 boot drive from it, if you don't think you'll need more troubleshooting just terminate it alongside its boot volume.
7. Create vm3 from vm2's boot drive. You should now have Ubuntu with btrfs - this will be your final instance.
8. Boot vm3, run nixos-infect. I had some config file issues which I didn't really feel like investigating, and by the time you reach this step you must be considerably tired too - so instead of running it as is, just copy the functions into your shell and run them one by one, in particular, check the generated drive configutation before running the actual infect script, and make sure the bootloader was successfully installed before rebooting.
9. Hopefully, nixos should now be installed on btrfs! I recommend setting a grub configuration limit, since you can't choose an old generation at boot either way, and your efi partition is fairly small

There's some leeway for errors (for example, the VM booted up even when I didn't mark the EFI partition type as EFI) but it still took me around 15 hours of trial and error to end up with this order

I'm using flakes to configure my NixOS machines. More or less as explained here.

The point is that, if I add the flake configuration before infection, and then infect it, the new system will be configured using the /etc/nixo/configuration.nix file generated in the infection script, instead of using the /etc/nixos/flake.nix that I already put there before.

Is there a way to preconfigure the system before infection so that, after infected, it's directly booted to the system flake?

The wiki says that I have to run nixos-infecton a debian or ubuntu but the README says Hetzner cloud works out of the box. When creating a server provide the following script as "User data"
But I cannot find a user data field when adding a server.
Is the README outdated is it me that is to stupid to find that user data 😅 ?