gentoo: How make emerge better use of more cores?

Question

I have Intel Atom N2800 (weak, but handy Aspire One D270, still enought for the work I use it for), but updates are really slow (like days).

I use MAKEOPTS=" -j5 " in /etc/make.conf and and emerge -DuNqv world and it sometimes goes to Load avg 5.0 5.0 5.0 or so, but usually it is just somewhere around 1.5 only.

Is there a way to make it be more time more paralel (and so faster)?

Answer 1

There are some things you can do to speed up portage, but often times these tweaks can have other side-effects without careful attention.

You can manipulate the intended loading by using --load-average=##.## in conjunction with --jobs=## on the command line. Replace the #'s with numbers. You can also make this and most other values (semi)permanent by adding EMERGE_DEFAULT_OPTS="--jobs=## --load-average=##.##" to make.conf. There is a decent article about using these values with real-world application. What isn't in the article is how to properly get around build failures due to a customized make.conf. If you want to make these settings global (make.conf), you should also understand how to bypass those settings with custom environment for specific packages. See here.

Some other notes (good and bad) on speeding up portage:

1) Enable parallel_fetch in your make.conf by adding / editing FEATURES="${FEATURES} parallel-fetch". This entry will allow downloading packages while it builds the current one. It doesn't really answer your question as such, but it's one of the least invasive portage "speed-ups".

2) Enable ccache. Install the ccache package, and add ccache to FEATURES like in #1. You will also need to define CCACHE_SIZE and CCACHE_DIR. Though it started out as a great idea, a lot of people have cooled on the benefits of ccache. It has been known to cause issues with some packages, and there are many reports that claim it can degrade compilation performance more often than speeding things up. I can't speak for how correct it is, but many of the points addressed in this article warning about using ccache are common belief.

3) ReNice emerge. You can define the niceness increment by adding PORTAGE_NICENESS=-#. The value of this number is added to the default process value. By making it negative, you will reduce the niceness of emerge processes. Be very careful with this setting as it will apply to subprocesses as well. With some packages having multiple subprocesses, this could get dicey in a hurry.

Final note - some of these features depend on your portage version. I mention this since you said you are editing /etc/make.conf, which is the old location. Newer portage versions have migrated config files to /etc/portage. If you have a make.conf in both places, the file in /etc/portage will supersede the one in /etc.