Gentoo Portage emerge TMPDIR extending zram btrfs volumes

You can probably see I was heading this way. This post follows a series, and you should read those posts again if you haven't done so recently. This blog post follows my journey through btrfs to gentoo to portage to tmpfs to zram back to btrfs again to finish a circle.

I have now wholly migrated over to gentoo, and left debian far behind. My gentoo is getting much better, as I use it more and more.. I continuously learn to fine-tune. I also find that I seem to be coming up with radical solutions extending ideas others have germinated. One such approach was moving most of my dozen+ different filesystems into btrfs subvolumes on a single partition^[1]. BTRFS has radically altered my diskspace efficiency, including backups (incremental send/receive across the network). But I digress..

As I mentioned before in a previous post^[2], one of my gentoo pain points has been the extremely (long and) resource intensive emerge on slow disk, particularly for large packages. Now I can't change devices like hard-disks or packages like firefox. But I can change the directory locations to reduce some of that pain factor. We can emerge in tmpfs^[2], which is a possible satisfactory solution, given the boundaries.

So what else can we do? We can't do much re hard boundaries like physical memory or package size, unless we choose to upsize hardware or downsize packages. I try to live within my hardware means, i.e. old hardware. But I do continuously downsize to packages with lesser requirements. Sometimes, we are stuck with runaway bloatwares.. like firefox. Is there anything we can do?

We can do something about the tmpfs boundaries. We will enlarge those boundaries! Yes, you read that right.. We do this by eliminating tmpfs with a combination of zram + btrfs to mahoossively enlarge the TMPDIR ramdisk size to cope with whatever package size might be thrown at it. Read on...

Check for btrfs support in your kernel. If not, you can't do anything further.

$ cat /proc/filesystems | grep btrfs
btrfs

You also need btrfs. In gentoo, sys-fs/btrfs-progs provides btrfs. Install this package, if you didn't already.

# emerge btrfs-progs

Check the package installed successfully. RTFM is useful.

$ man btrfs

We start with how we did tmpfs for gentoo portage emerge^[2]. We follow the same strategy, but migrate from tmpfs to zram^[3].

$ df -h | grep /var/tmp
/dev/zram2      3.0G   17M  2.8G   1% /var/tmp

Thus, we arrive at gentoo portage emerge TMPDIR /var/tmp/portage on a zram device. In this particular case, we have ~3G diskspace available for emerge. But with compression, zram could consume much more data in that space. Even that might not be sufficient for some of the bloatware packages that we need. Oh, when will these newish devs learn that ancient wisdom still has some merits and accept the unix philosophy of modular development -- extremely small programs designed to work well with other extremely small programs to satisfy requirements more efficiently than huge monolith bloatwares. This world just can't have enough of too-big-to-fails. But I digress again..

Some systems might have much less memory available. Hence, we need to increase/extend portage TMPDIR size to whatever the largest package might need. We do this by extending our btrfs device volume to add diskspace from other disks. These other disks could be internal or external, slow or fast, physical or virtual. BTRFS will take whatever you throw at it. I love this btrfs, and have used it for many years now, without any issues at all. Though I must confess that I have not really pushed any boundaries. My requirements are minimal.

If you remember from my previous post in this series: discovering and replacing all tmpfs^[3], we created zram devices with btrfs filesystem. Have a read again. You could have chosen any filesystem for your zram device. But for this requirement, we specifically need btrfs.

We need a new btrfs filesystem temporarily, just for the duration of an emerge compiling packages larger than our available memory. Some ebuilds like firefox check freespace. Others don't, and will just bomb out when emerge runs out of space in TMPDIR. Either way, you will find out that you need more space in that ramdisk. That is when I add a new btrfs filesystem into the existing /var/tmp volume. BTRfs will now extend this volume's diskspace to include the new device diskspace.

Note that this new btrfs device should not contain existing data. It is supposed to be new right? It should be empty, or will be wiped clean.

So we prepare our new device. You might use another partition or external usb/network disk unit. You could even use more than one. In this example, I use a raw file as disk.

$ df -h | grep /var/tmp
/dev/zram2      3.0G   17M  2.8G   1% /var/tmp

# dd if=/dev/zero of=/media/disks/btrfs8192.img bs=1G count=8
# losetup --find --show /media/disks/btrfs8192.img
/dev/loop0

# btrfs filesystem show /var/tmp/
Label: none  uuid: 16b8538d-951a-4449-bf9f-e90118410a96
        Total devices 1 FS bytes used 192.00KiB
        devid    1 size 3.00GiB used 276.00MiB path /dev/zram1
# btrfs device add /dev/loop0 /var/tmp/
Performing full device TRIM /dev/loop0 (8.00GiB) ...
# btrfs filesystem show /var/tmp/
Label: none  uuid: 16b8538d-951a-4449-bf9f-e90118410a96
        Total devices 2 FS bytes used 192.00KiB
        devid    1 size 3.00GiB used 276.00MiB path /dev/zram1
        devid    2 size 8.00GiB used 0.00B path /dev/loop0

$ df -h | grep /var/tmp
/dev/zram1       11G   17M   11G   1% /var/tmp

And voila! I have just extended my gentoo portage TMPDIR diskspace. Now I can remove those notmpfs workarounds for huge packages, knowing that emerge is having the best of both worlds while compiling those too-big bloatwares.

We don't need this to last across reboots, and so we don't do anything further. Our requirement is only temporary, till the next time emerge encounters another too-big package.

If you don't want to wait till the next reboot, you can remove this new device we added just now.

# btrfs filesystem show /var/tmp/
Label: none  uuid: 16b8538d-951a-4449-bf9f-e90118410a96
        Total devices 2 FS bytes used 192.00KiB
        devid    1 size 3.00GiB used 276.00MiB path /dev/zram1
        devid    2 size 8.00GiB used 0.00B path /dev/loop0
# btrfs device remove /dev/loop0 /var/tmp/
# btrfs filesystem show /var/tmp/
Label: none  uuid: 16b8538d-951a-4449-bf9f-e90118410a96
        Total devices 1 FS bytes used 192.00KiB
        devid    1 size 3.00GiB used 276.00MiB path /dev/zram1

You have to wait till this command finishes, before disconnecting any external disk(s).

This is just an example for a particular use-case, but can fit many other scenarios. If you find this useful and extend this elsewhere, please let me know.

This idea had been knocking around in my head for many months, and I was just missing a piece. I finally found some time this weekend to implement and test. Peace and goodwill to all :)

---

[1] multiboot btrfs subvolumes
[2] gentoo emerge in tmpfs
[3] discovering zram and replacing all tmpfs