fidzu : gentoo

I have written my share of eclasses, and I have made my share of mistakes. Designing good eclasses is a non-trivial problem, and there are many pitfalls you should be watching for. In this post, I would like to highlight three of them.

Not all metadata variables are combined

PMS provides a convenient feature for eclass writers: cumulative handling of metadata variables. Quoting the relevant passage:

The IUSE, REQUIRED_USE, DEPEND, BDEPEND, RDEPEND and PDEPEND variables are handled specially when set by an eclass. They must be accumulated across eclasses, appending the value set by each eclass to the resulting value after the previous one is loaded. Then the eclass-defined value is appended to that defined by the ebuild. […]

Package Manager Specification (30th April 2018), 10.2 Eclass-defined Metadata Keys

That's really handy! However, the important thing that's not obvious from this description is that not all metadata variables work this way. The following multi-value variables don't: HOMEPAGE, SRC_URI, LICENSE, KEYWORDS, PROPERTIES and RESTRICT. Surely, some of them are not supposed to be set in eclasses but e.g. the last two are confusing.

This means that technically you need to append when defining them, e.g.:

# my.eclass
RESTRICT+=" !test? ( test )"

However, that's not the biggest problem. The real issue is that those variables are normally set in ebuilds after inherit, so you actually need to make sure that all ebuilds append to them. For example, the ebuild needs to do:

# my-1.ebuild
inherit my
RESTRICT+=" bindist"

Therefore, this design is prone to mistakes at ebuild level. I'm going to discuss an alternative solution below.

Declarative vs functional

It is common to use declarative style in eclasses - create a bunch of variables that ebuilds can use to control the eclass behavior. However, this style has two significant disadvantages.

Firstly, it is prone to typos. If someone recalls the variable name wrong, and its effects are not explicitly visible, it is very easy to commit an ebuild with a silly bug. If the effects are visible, it can still give you some quality debugging headache.

Secondly, in order to affect global scope, the variables need to be set before inherit. This is not trivially enforced, and it is easy to miss that the variable doesn't work (or partially misbehaves) when set too late.

The alternative is to use functional style, especially for affecting global scope variables. Instead of immediately editing variables in global scope and expecting ebuilds to control the behavior via variables, give them a function to do it:

# my.eclass
my_enable_pytest() {
  IUSE+=" test"
  RESTRICT+=" !test? ( test )"
  BDEPEND+=" test? ( dev-python/pytest[${PYTHON_USEDEP}] )"
  python_test() {
    pytest -vv || die
  }
}

Note that this function is evaluated in ebuild context, so all variables need appending. Its main advantage is that it works independently of where in ebuild it's called (but if you call it early, remember to append!), and in case of typo you get an explicit error. Example use in ebuild:

# my-1.ebuild
inherit my
IUSE="randomstuff"
RDEPEND="randomstuff? ( dev-libs/random )"
my_enable_pytest

Think what phases to export

Exporting phase functions is often a matter of convenience. However, doing it poorly can cause ebuild writers more pain than if they weren't exported in the first place. An example of this is vala.eclass as of today. It wrongly exports dysfunctional src_prepare(), and all ebuilds have to redefine it anyway.

It is often a good idea to consider how your eclass is going to be used. If there are both use cases for having the phases exported and for providing utility functions without any phases, it is probably a good idea to split the eclass in two: into -utils eclass that just provides the functions, and main eclass that combines them with phase functions. A good examples today are xdg and xdg-utils eclasses.

When you do need to export phases, it is wortwhile to consider how different eclasses are going to be combined. Generally, a few eclass types could be listed:

• Unpacking (fetching) eclasses; e.g. git-r3 with src_unpack(),
• Build system eclasses; e.g. cmake-utils, src_prepare() through src_install(),
• Post-install eclasses; e.g. xdg, pkg_*inst(), pkg_*rm(),
• Build environment setup eclasses; e.g. python-single-r1, pkg_setup().

Generally, it's best to fit your eclass into as few of those as possible. If you do that, there's a good chance that the ebuild author would be able to combine multiple eclasses easily:

# my-1.ebuild
PYTHON_COMPAT=( python3_7 )
inherit cmake-utils git-r3 python-single-r1

Note that since each of those eclasses uses a different phase function set to do its work, they combine just fine! The inherit order is also irrelevant. If we e.g. need to add llvm to the list, we just have to redefine pkg_setup().

06 Nov 2019 7:57am GMT

The Gentoo distfile mirror network is essential in distributing sources to our users. It offloads upstream download locations, improves throughput and reliability, guarantees distfile persistency.

The current structure of distfile mirrors dates back to 2002. It might have worked well back when we mirrored around 2500 files but it proved not to scale well. Today, mirrors hold almost 70 000 files, and this number has been causing problems for mirror admins.

The most recent discussion on restructuring mirrors started in January 2015. I have started the preliminary research in January 2017, and it resulted in GLEP 75 being created in January 2018. With the actual implementation effort starting in October 2019, I'd like to summarize all the data and update it with fresh statistics.

Continue reading

13 Oct 2019 2:34pm GMT

So I needed to set up qemu+kvm on a new server (After the old one died)

Seems like i forgot to mention how I set up the bridge network on my previous blog post so here you go:

First let me mention that I am using the second physical Interface on the server for the bridge. Depending on your available hardware or use case you might need / want to change this:

So this is fairly simple (if one has a properly configured kernel of course - details on the Gentoo Wiki article on Network Bridges):

First add this in your /etc/conf.d/net (adjust the interface names as needed):

# QEMU / KVM bridge
bridge_br0="enp96s0f1"
config_br0="dhcp"

then add an init script and start it:

cd /etc/init.d; ln -s net.lo net.br0
/etc/init.d/net.br0 start # to test it

So then I get this error when trying to start my kvm/qemu instance:

 * creating qtap (auto allocating one) ...
/usr/sbin/qtap-manipulate: line 28: tunctl: command not found
tunctl failed
 * failed to create qtap interface

seems like I was missing sys-apps/usermode-utilities .. so just emerge that, only to get this:

 * creating qtap (auto allocating one) ...
/usr/sbin/qtap-manipulate: line 29: brctl: command not found
brctl failed
 * failed to create qtap interface

yepp I forgot to install that too ^^ .. so Install net-misc/bridge-utils and now it starts the VM

04 Oct 2019 8:53am GMT