DEV Community

Ray Harris
Ray Harris

Posted on

Day 4: (Re)Installing NixOS on my Macbook Pro

#nixos #dualbooting #linux

If you're following from previous notes, you'll know that I'm starting fresh. I've got a totally fresh install of macOS Ventura and I'm ready to rock and roll. I'm combining a couple different guides to get my setup working and I'm noting everything I do here. Spoiler: I got it working this time! Let's get started!

Basic Setup

To clarify, my Macbook is an APFS T2 Intel Macbook Pro 2019 15 inch. If you follow this with another device, there may be some differences in steps required. But at least this section is easy:

  1. Plug in power
  2. Plug in Ethernet via USB-C adapter (optional)
  3. Plug in Get 4GB* USB for install media (via USB-C adapter)

*I actually used a 16gb drive this time, but I did use a 4gb in the past and I think it was good enough.

Create installation media

I downloaded the minimal nixos-t2-release and used USBImager to write the resulting ISO to my usb drive. Balena Etcher is a good choice as well.

Create new partitions on Macbook Pro SSD

This can be done many ways, but I decided to use the macOS Disk Utility because it seems like the safest way to avoid messing up my APFS partition. Here is the desired endstate:



Total Physical SSD Storage: 500GB
0. EFI Partition (<1 GB; Not shown in Disk Utility)
1. APFS Partition (250 GB; macOS has volumes within this)
2. ExFAT Partition (30 GB; CrossData - sharing between OSes)
3. Ext4 Partition (120 GB; NixOS Root)
4. Ext4 Partition (80 GB; NixOS Home)
5. Swap Partition (20 GB; Swap - Memory Overflow for NixOS)
Enter fullscreen mode Exit fullscreen mode

I am giving a lot of space to NixOS root because this is where I understand packages and previous generations will be stored. I also witnessed online that people who complain about running out of space in their root partition are mocked by people who claim to be allocating 100+ GB to it. It seems to me that this is a bit of a mental shift from past operating systems. Since all my programs will be nix packages, they can be nicely separated from my personal files and data. Apple has done a pretty neat thing with APFS. macOS's portion of the drive is so tidy because all the operational divisions are handled with dynamically sized volumes within the APFS partition. Meanwhile, Microsoft will screw up your EFI partition, so you need a second EFI to mitigate that if you want to triple boot. I don't really understand all that, but I'm just going to use one for now.

Here's how to create the partitions with macOS GUI Disk Utility. The trick is to click the right items in the right order.

  1. View > Show all devices
  2. Click the top level APPLE SSD AP0512M Media
  3. Click Partition
  4. Hit the +; choose Add Partition not volume
  5. Set the size for the partition
  6. Name the partition
  7. Select the ExFat format
  8. Hit Apply & Confirm

Plus button first, then size, then name, then format.

Do this for each partition. I did it 4 times to create my CrossData partition and 3 partitions for NixOS (NixOS Root, NixOS Home, and Swap). Sometimes there may be an error. I don't know the consequences of that, but I just respond by deleting the new partition and creating it again. One time it took 3 tries, but usually it works the first time.

Looks like this for me now:

Image description

Extract the Macbook Devices Firmware

We need to get the firmware from macOS somewhere linux can access it. The t2linux site provides a handy script called firmware.sh which has a macOS portion and a Linux portion. The macOS portion does 2 things:

  1. Zips the firmware files up into firmware.tar.gz on the EFI partition
  2. Copies the firmware.sh script into the EFI partition as well

Download the script, cd into its directory, and run it.

Grant execution permission with chmod first.



chmod +x firmware.sh
Enter fullscreen mode Exit fullscreen mode

Execute it



./firmware.sh
Enter fullscreen mode Exit fullscreen mode

If you want to do it manually, you can follow the steps on this guide. Those steps include the easy part (copying the files over) as well as the hard part (packaging them up appropriately). Looked a little intimidating for me, so I stuck to running the firmware.sh script and crossing my fingers.

Prepare for dual booting

I'm not sure why, but I still couldn't boot into another device without rEFInd. Here's what I had to do to get my boot-up process working.

Boot into recovery mode by holding CMD+R

  • Use Startup Security Utility to set secure boot to No Security and allowed boot media to Allow booting. Apple docs on this here.
  • Use Terminal to disable system integrity protection (SIP) by running csrutil disable. Apple docs here.

Reboot to normal macOS and download rEFInd from the A binary zip file sourceforge link from rodsbooks website.

From its directory, run the install script



./refind-install
Enter fullscreen mode Exit fullscreen mode

To make life easier, I also set an nvram parameter so that I wouldn't have to hold option ⌥ to boot to startup manager.



sudo nvram manufacturing-enter-picker=true
Enter fullscreen mode Exit fullscreen mode

Boot from USB

Time to get down to business. Restart the mac and we should see two boot options: Macintosh HD and EFI Boot. Since I have rEFInd installed, it doesn't matter what I choose at this point, but if you don't, you need to pick EFI Boot.

When rEFInd loads, what I saw was a list of 4 boot options.

  1. macOS from Preboot (ssd)
  2. Boot EFI\boot\refind_x64.efi from EFIBOOT (usb)
  3. Boot Fallback boot loader from EFIBOOT (usb)
  4. Boot Legacy OS from whole disk volume (usb)

Now I don't fully understand these options, but I know to boot from my install media, I need to choose #3. Some text comes up, I get brief GUI boot option and let the timer countdown to 0 to automatically choose the first option. Some green and white matrix code flies by and next thing I know I'm at the NixOS Live CD command line.

Finish preparing the drive

Welcome to the command line. We should see a blinking underscore after [nixos@nixos:~]$. We have a few steps to take before actually installing NixOS. First things first, let's make the font size bigger. 



setfont ter-v32n
Enter fullscreen mode Exit fullscreen mode

Wow!!! The keyboard works!!! We're on the right track! Next let's take a look at the partitions we prepared earlier



lsblk -o NAME,FSTYPE,SIZE,MOUNTPOINT
Enter fullscreen mode Exit fullscreen mode

This gives me a bunch of stuff, take a look:



NAME        FSTYPE    SIZE     MOUNTPOINT
loop0       squashfs  800M     /nix/.ro-store
sda         iso9660   14.7G
├─sda1      iso9660   839M     /iso
└─sda2      vfat      3M
nvme0n1               465.9G
├─nvme0n1p1 vfat      300M
├─nvme0n1p2 apfs      232.8G
├─nvme0n1p3 exfat     18.6G
├─nvme0n1p4 exfat     74.5G
├─nvme0n1p5 exfat     111.8G
└─nvme0n1p6 exfat     27.9G
Enter fullscreen mode Exit fullscreen mode

Looks familiar, right? Now we need to format the two NixOS partitions as Ext4. We'll keep the CrossData partition as exfat and we'll format the swap partition differently.

First, let's format the swap partition and label it swap



sudo mkswap -L swap /dev/nvme0n1p3
Enter fullscreen mode Exit fullscreen mode

Next, let's format the nixos partition. Graciously, it gave me a confirmation dialog to confirm. That's good because this command wipes the contents of the partition. I don't know what will happen if you wipe the APFS partition. Can't be very good.



sudo mkfs.ext4 -L nixos /dev/nvme0n1p5
Enter fullscreen mode Exit fullscreen mode

Finally, the partition for the home directory



sudo mkfs.ext4 -L home /dev/nvme0n1p4
Enter fullscreen mode Exit fullscreen mode

Let's check our work



lsblk -o NAME,FSTYPE,SIZE,LABEL,MOUNTPOINT
Enter fullscreen mode Exit fullscreen mode

Same command as before, but now with the LABEL column.



NAME        FSTYPE    LABEL                         SIZE     MOUNTPOINT
loop0       squashfs  802.6M                                 /nix/.ro-store
sda         iso9660   nixos-minimal-23.05-x86_64    14.7G
├─sda1      iso9660   nixos-minimal-23.05-x86_64     839M    /iso
└─sda2      vfat      EFIBOOT                          3M
nvme0n1                                            465.9G
├─nvme0n1p1 vfat      EFI                            300M
├─nvme0n1p2 apfs                                   232.8G
├─nvme0n1p3 swap      swap                          18.6G
├─nvme0n1p4 ext4      home                          74.5G
├─nvme0n1p5 ext4      nixos                        111.8G
└─nvme0n1p6 exfat     CrossData                     27.9G
Enter fullscreen mode Exit fullscreen mode

Looking good! Before we continue mounting, let's follow this complex process for connecting to the internet.

Loading Wi-Fi firmware to installer

With an ethernet-USBC adapter, you'll automatically be able to connect online, but these steps will still help prepare for regular usage later on, so I suggest following them.

First, let's run that firmware.sh script from before. Remember the first time we ran it, it just copied the script file into the EFI partition along with a the zipped firmware files. Thanks to its power of conditional statements, it will do something else when run on Linux. Run the script with this command:



./mnt/boot/firmware.sh
Enter fullscreen mode Exit fullscreen mode

Hopefully that works for you first try. When I did it, it failed and I ended up going in and reading the script so that I could pick out parts to execute manually. I think my mistake was mounting my EFI partition before running this script, which also mounts it to another location. Since we saved that step for AFTER this one, you should be fine.

Once you've got your firmware installed, you can connect online! Use wpa_supplicant and add your network. The same instructions as the NixOS manual.



systemctl start wpa_supplicant
Enter fullscreen mode Exit fullscreen mode

Enter the wpa_cli with



wpa_cli
Enter fullscreen mode Exit fullscreen mode

A few more commands...



add_network
Enter fullscreen mode Exit fullscreen mode

If this returns something other than 0, use that number as the network id for the following commands.



set_network 0 ssid "myhomenetwork"
Enter fullscreen mode Exit fullscreen mode 


set_network 0 psk "mypassword"
Enter fullscreen mode Exit fullscreen mode 


set_network 0 key_mgmt WPA-PSK
Enter fullscreen mode Exit fullscreen mode 


enable_network 0
Enter fullscreen mode Exit fullscreen mode

If that worked, then you'll see some messages including the word CONNECTED. Wonderful. On a Macbook!

Mounting Partitions

Now we mount the partitions. Consider that right now we are running on the USB. In order to access the Macbook SSD partitions, we need to make them available at some location. Mounting is the process that takes a partition or device and makes it available at a specified location. So we'll be creating /mnt as a working directory from where we can access these formatted partitions.

Furthermore, the drive mount statuses are what the NixOS installer will look at when generating the important hardware-configuration.nix file.

The order of operations here doesn't matter AFAIK. But we need to mount the root partition (nixos), the home partition (home), and the boot partition (EFI). We'll activate the swap partition because the nixos manual says to do so, even though we have 16GB of RAM in this MBP which really should be enough.

First let's run that lsblk command again



lsblk -o NAME,FSTYPE,SIZE,LABEL,MOUNTPOINT
Enter fullscreen mode Exit fullscreen mode

None of your new partitions should be mounted at this point. If they are, unmount them. I needed to unmount my boot partition at this point so I ran sudo umount /dev/nvme0n1p1 then did my lsblk command again to confirm.

If you don't want to keep typing sudo, you can run sudo -i to identify yourself as the super user. WARNING: Make sure you are using your partition names. They might not be the same as my examples below. Ex: My nvme0n1p5 might be your nvme0n1p4.

Mount the root partition



mount /dev/nvme0n1p5 /mnt
Enter fullscreen mode Exit fullscreen mode

Create the home directory



mkdir /mnt/home
Enter fullscreen mode Exit fullscreen mode

Mount the home partition



mount /dev/nvme0n1p4 /mnt/home
Enter fullscreen mode Exit fullscreen mode

Create a directory to mount the boot/EFI partition



mkdir -p /mnt/boot/efi
Enter fullscreen mode Exit fullscreen mode

Mount the EFI partition



mount /dev/nvme0n1p1 /mnt/boot/efi
Enter fullscreen mode Exit fullscreen mode

Activate the swap



swapon /dev/nvme0n1p3
Enter fullscreen mode Exit fullscreen mode

Preparing Configuration.nix

Remember how NixOS is a declaratively defined operating system? Everything we're about to install will be declared in configuration.nix. And since we have an internet connection, you could probably just download a valid config file right now and not do anything manually. In fact, here's my configuration.nix that I ended up with after finishing this guide. But if you've come so far with me doing things the hard way, why not continue? Here's the minimal config I want to implement right now:

  1. Add general t2 mac hardware support
  2. Add wifi drivers
  3. Store the home directory contents on my home partition
  4. Anything else to get the machine to work
  5. Install a desktop environment

First things first, let's generate our starter configuration.nix file. There's a command for that:



nixos-generate-config --root /mnt
Enter fullscreen mode Exit fullscreen mode

This actually generates two files. We'll only focus on configuration.nix for now, as the other one, hardware-configuration.nix shouldn't need to be modified. You'll notice in a moment that the main config file imports the hardware config file. Pretty neat. Apparently there's a lot of room to get creative with organizing config. For now we'll try to keep it simple.

Now let's edit it with Nano, the teeny tiny command line text editor. If you haven't already, cd back to root. To do that, I ran cd ../.. and then pwd to confirm that I was in /. Really, just try running the next command and navigate with ls, cd, and pwd as needed. It's good to be comfortable with that kind of navigation!



nano /mnt/etc/nixos/configuration.nix
Enter fullscreen mode Exit fullscreen mode

Up pops a bunch of cyan text. Mostly everything is commented out. It's nice that the installer gives us some guidance with this starter config rather than just an empty file.

The bottom of the screen should show the available controls. Arrow keys to navigate, CTRL+O to save, CTRL+X to exit.

Let's work top to bottom.

First up is an import that we need to add. The top of your file probably looks like this:



imports =
  [ # Include the results of the hardware scan.
    ./hardware-configuration.nix
  ];
Enter fullscreen mode Exit fullscreen mode

We need to add a line following the t2linux instructions so that it looks like this:



imports =
  [ # Include the results of the hardware scan.
    ./hardware-configuration.nix
    "${builtins.fetchGit { url = "https://github.com/kekrby/nixos-hardware.git"; }}/apple/t2"
  ];
Enter fullscreen mode Exit fullscreen mode

I really don't like that the line turns red. But this is what the instructions say and if it doesn't work I'll come back and edit this part of my post. I don't know if the indentation matters, but I'm keeping things aligned and using spaces not tabs.

To the boot.loader section, add one line:



boot.loader.efi.efiSysMountPoint = "/boot/efi";
Enter fullscreen mode Exit fullscreen mode

Next, networking.hostname. It's commented out right now, which is fine, but how do you want your device to be identified on the network? I'm going to have two nixos devices (laptop and desktop) so I'm going to uncomment this and change the name. So my line now looks like this:



networking.hostname = "nixos-mbp"
Enter fullscreen mode Exit fullscreen mode

Continuing down the list, we need to choose a network solution: networking.wireless or networking.networkmanager. Uncomment networking.networkmanager.enable = true; and move on! Next up, timezone. I think the spelling counts here, so maybe skip it if you don't know yours. I'm setting mine to time.timeZone = "America/Los_Angeles";

I'm skipping the network proxy. In the internationalization section, I'm uncommenting only the first line:



i18n.defaultLocale = "en_US.UTF-8";
Enter fullscreen mode Exit fullscreen mode

Next there's something about X11 windowing system. This is required if you want to use a desktop environment. I certainly do. If you go online you can find arguments about which is best. If you look long enough, you might find two people that agree. What's great about NixOS is that if we want to change what we have, we just come in here to the configuration and change it.

So I am uncommenting the line



services.xserver.enable = true
Enter fullscreen mode Exit fullscreen mode

and adding a few more to enable a Plasma desktop environment 



# Enable the Plasma Desktop Environment.
services.xserver.displayManager.sddm.enable = true;
services.xserver.desktopManager.plasma5.enable = true;
Enter fullscreen mode Exit fullscreen mode

If you want GNOME instead of Plasma, change sddm to gdm and plasma5 to gnome within those 2 lines.

Next there is some config about how the keyboard should work. I am using a US keyboard and don't need nothin' fancy. Uncommenting the first line does it for me:



services.xserver.layout = "us";
Enter fullscreen mode Exit fullscreen mode

Skipping the part about CUPS and sound. Yes, sound 🥲. I think I read somewhere that sound is borked on Macbooks, so I will leave that for another day. There's a line to enable touchpad support. I'm not sure if this is the right way to do it, but I'm uncommenting that line as well



services.xserver.libinput.enable = true;
Enter fullscreen mode Exit fullscreen mode

Then we have a user account. Apparently accounts are managed in config! Awesome! I wonder how passwords work though -- don't want those in config!! Here's what mine looks like



users.users.ray = {
    isNormalUser = true;
    description = "Ray";
    extraGroups = [ "networkmanager" "wheel" ];
    packages = with pkgs; [
      firefox
    ];
  };
Enter fullscreen mode Exit fullscreen mode

I deleted thunderbird, but why not keep firefox. Below this, I added another important line:



# Allow unfree packages
nixpkgs.config.allowUnfree = true;
Enter fullscreen mode Exit fullscreen mode

Remember, everything is a package. So if you want to use something unfree like VSCode, you need to allow it.

The T2Linux guide instructs to add this, so let's add it:



# Firmware for Broadcom Wi-Fi and Bluetooth
hardware.firmware = [
  (pkgs.stdenvNoCC.mkDerivation {
    name = "brcm-firmware";

    buildCommand = ''
      dir="$out/lib/firmware"
      mkdir -p "$dir"
      cp -r ${./firmware}/* "$dir"
    '';
  })
];
Enter fullscreen mode Exit fullscreen mode

*Note: I have modified one line in the t2linux guide by removing /files so that our firmware will be stored at /etc/nixos/firmware/bcrm.

This is another spot where the syntax highlighting is no good. Nano thinks that /* is the beginning of a multi-line comment, so the following lines are all blue. Nix doesn't support multi-line comments as far as I can tell, and the expression is still valid. Maybe Nix does support it and it doesn't take effect because of the double apostrophe? Dunno! But this works, let's move on.

Last config change is to add the git package. Since we're relying on the fetchGit expression to get the hardware firmware, we need to make sure git is available. You can simply add git as a new line under your created user, but I'm going to add it as a system package instead of a user package. AFAIK this just makes it available to everyone instead of just one user. So lower down you'll see



# List packages installed in system profile. To search, run:
# $ nix search wget
#   environment.systemPackages = with pkgs; [
#     vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
#     wget
# ];
Enter fullscreen mode Exit fullscreen mode

We will add git very simply so that it looks like this;



# List packages installed in system profile. To search, run:
# $ nix search wget
  environment.systemPackages = with pkgs; [
#   vim # Do not forget to add an editor to edit configuration.nix! The Nano editor is also installed by default.
    # wget
    git
  ];
Enter fullscreen mode Exit fullscreen mode

When I first installed, I didn't realize I needed to add git. When I booted into my machine for the first time, I had to use an external keyboard to comment out the fetchGit hardware line near the top, add git, rebuild, then go an uncomment the fetchGit hardware line and rebuild & reboot again.

This is what the final configuration.nix and hardware-configuration.nix should look like.

Lastly before installing, we also need to copy the Broadcom wireless firmware into the nixos partition. First create the appropriate directory:



mkdir -p /mnt/etc/nixos/firmware/brcm
Enter fullscreen mode Exit fullscreen mode

Then issue the copy pasta command:



cp -r /lib/firmware/brcm/* /mnt/etc/nixos/firmware/brcm/
Enter fullscreen mode Exit fullscreen mode

Install NixOS

Run



nixos-install
Enter fullscreen mode Exit fullscreen mode

Another awesome thing about NixOS: if something is wrong, it's gonna halt the installation and tell you right away. You get to fix it before having the chance to screw up your system. I had a couple typos in my configuration.nix when I got to this point. Once it gets going, you'll hear the airplanes take off (the laptop fan). Lots of white text scrolling across the screen. I don't really read it. Looks like a lot of building '/nix/store/....

Eventually it will prompt you for a new root password and report Installation finished!! Woot!

Boot NixOS on Macbook Pro

We're still running the installer, so let's reboot with this command:



reboot
Enter fullscreen mode Exit fullscreen mode

If everything went right, Macbook's startup manager will run and show 3 options: EFI Boot (Internal), Macintosh HD (Internal), and EFI BOOT (external).

NixOS should be running now, with access to internet and ability to log in to a user named root with the password you set during installation. Once logged in, you can set a password for your user with a command like this



passwd ray
Enter fullscreen mode Exit fullscreen mode

It'll ask you to type and retype the desired password. Now I don't know how passwords work yet in NixOS, but from what I can tell it's not a fully solved problem. Perhaps we'll explore that together another day.

Booting macOS and NixOS selectively

I don't know if you'll encounter these, but I had to do a few extra things before I could get into NixOS on the Mac SSD.

From my installation media, I had to run an additional command to install the bootloader because my original configuration.nix had the boot partition mounted at /boot instead of /boot/efi. To do that, I followed some steps here. Specifically I ran nixos-enter from my installation media and then once that gave me the > text entry prompt, I ran



NIXOS_INSTALL_BOOTLOADER=1 /nix/var/nix/profiles/system/bin/switch-to-configuration boot
Enter fullscreen mode Exit fullscreen mode

After that. I could only boot NixOS, not macOS. So I reset the nvram by booting with CMD+OPTION+P+R, disabled SIP with csrutil disable after booting in recovery mode CMD+R, and finally reinstalled rEFInd via macOS.

So now I choose my boot partition successfully with rEFInd. I'd like to use the Mac's native startup manager instead but that'll take some figuring out.

Congrats!

We got our Macbook Pro dual booting macOS and NixOS. We got our keyboard, touchbar, trackpad, and wireless networks working. Woo!! This post is labeled "day 4" but this took more than 1 day to figure out, I promise. I'm just glad to have this stable starting point from which I can continue to improve my OS. Maybe we'll reach the limits of NixOS eventually, but for now, I'm optimistic.

To Do

The more I learn, the less I know. Here are some things I want to get back to:

  1. Getting a stable & desirable boot management configuration set up
  2. NixOS keeps trying to have 2 wired connections but there's only one... Would like to fix that.
  3. Getting audio to work
  4. If I leave my laptop open and running for a long time and come back, I can't get the screen to turn back on. Want to fix that
  5. Setting up a smooth integration between my configuration.nix and a github repository

And of course, installing on my desktop, potentially getting my two environments in sync, and setting up a development environment for some software project or another.

Thank you!

If you went through all this, I would appreciate your feedback. Anything on what you liked or would have liked to have seen would be great! I can update this post with improvements.

Top comments (1)

Collapse
 
vovs03 profile image
Vladimir Pavlychev •

Ray, thanx a lot for great series of articles about your experience dive into NixOS. That’s perfect! Good luck in this way!!!

Read next

moseeh_52 profile image

Understanding os.Stat() vs os.Lstat() in Go: File and Symlink Handling

moseeh -

trixsec profile image

50 Linux Networking Commands Every DevOps Engineer Should Master

Trix Cyrus -

brogrammerjohn profile image

A performant and extensible Web Server with Zig and Python

brogrammerjohn -

tallesl profile image

Cleaning up syslog

Talles L -