emerge for large packages. However, it is expected and many have come up with their own tricks to make this more efficient.For me, this has been a gradual step-by-step approach, as I encountered these issues and discovered more about Gentoo.
I stated out with my portage configured as default -- compiling everything on the same disk/filesystem mount on my rotating (not ssd) disk. Any emerge affected everything else, which was too invasive to ignore.
I want emerge to run gently in the background, not affecting any foreground tasks. So I used
nice, and later also ionice. I did so on the commandline for each emerge. Anytime I forgot, the system would crawl for almost the entirety of the emerge.# nice -n19 ionice -c3 emerge ...
And then I found out about making them the default options in
man make.conf and /usr/share/portage/config/make.conf.example[1].$ cat /etc/portage/make.conf | grep NICE
PORTAGE_NICENESS="19"
PORTAGE_IONICE_COMMAND="ionice -c3 -p\${PID}"For
ionice, your kernel needs the CFQ ioscheduler. Check your kernel config. I found my system suffering (sometimes extreme) lags due to CFQ, and I switched back to noop. Apparently, CFQ is more efficient on larger systems and heavier loads. Most of the time, my minimal-system resource usage is almost nil. You might have a different experience.I still felt emerge invasive enough, although much less than before. Longer emerge and larger packages have a noticeable impact on foreground tasks. A kindly soul on the gentoo forums suggested[3] compiling on
tmpfs. I did a search and found this approach on the Gentoo Wiki[2].Since I keep a tight control on what is on my system, I find that I usually have more memory to play with than most others. So I migrated to Portage TMPDIR on tmpfs[2].
Portage
TMPDIR is /var/tmp/portage. To move it to tmpfs, we just mount it there. This can be set at boot in /etc/fstab.$ cat /etc/fstab | grep portage tmpfs /var/tmp/portage tmpfs defaults,noatime,mode=775,uid=portage,gid=portage,size=3G
All was well, till emerge started crashing.. for larger packages like firefox and libreoffice. Again the gentoo forum helped me understand why. I had to temporarily reset the Portage TMPDIR back to disk, by unmounting /var/tmp/portage, for larger packages.
The next step was identifying packages which needed more emerge diskspace than my tmpfs, and just setting those to emerge on disk, rather than tmpfs[2].
cat /etc/portage/package.env app-office/libreoffice notmpfs.conf games-action/supertuxkart notmpfs.conf sys-devel/gcc notmpfs.conf www-client/firefox notmpfs.conf
cat /etc/portage/env/notmpfs.conf PORTAGE_TMPDIR="/media/data/tmp/"
Now I did't need to unmount /var/tmp/portage when these larger packages hit emerge. But the pain points are back, as emerge on slow-disk, particularly for large packages, becomes a bottleneck again. The larger packages demanding more resources and consequent longer emerges was that particular pain point, and just for this I was entirely back on disk again, rather than tmpfs for even part of it. Still, this is much better than when I started with default portage config.
Looking back in hindsight, all this seems logical. But till I hit the wall, I didn't realise what needed to be done or how. And it happened, one step at a time. I seem to be learning from *my* mistakes, rather than learning from others'. Hmm..
[1] http://github.com/gentoo/portage/blob/master/cnf/make.conf.example
[2] http://wiki.gentoo.org/wiki/Portage_TMPDIR_on_tmpfs
[3] http://forums.gentoo.org/viewtopic-p-7868958.html#7868958
No comments:
Post a Comment