31 Dec 2016

feedPlanet Gentoo

Domen Kožar: Reflecting on 2016

Haven't blogged in 2016, but a lot has happened.

A quick summary of highlighted events:

2016 was a functional programming year as I've planned by end of 2015.

I greatly miss Python community and in that spirit, I've attended EuroPython 2016 and helped organize DragonSprint in Ljubljana. I don't think there's a place for me in OOP anymore, but I'll surely attend community events as nostalgia will kick in.

2017 seems extremely exciting, plans will unveil as I go, starting with some exciting news in January for Nix community.

31 Dec 2016 6:00pm GMT

22 Dec 2016

feedPlanet Gentoo

Sven Vermeulen: SELinux System Administration, 2nd Edition

While still working on a few other projects, one of the time consumers of the past half year (haven't you noticed? my blog was quite silent) has come to an end: the SELinux System Administration - Second Edition book is now available. With almost double the amount of pages and a serious update of the content, the book can now be bought either through Packt Publishing itself, or the various online bookstores such as Amazon.

With the holidays now approaching, I hope to be able to execute a few tasks within the Gentoo community (and of the Gentoo Foundation) and get back on track. Luckily, my absence was not jeopardizing the state of SELinux in Gentoo thanks to the efforts of Jason Zaman.

22 Dec 2016 6:26pm GMT

08 Dec 2016

feedPlanet Gentoo

Mike Pagano: (CVE-2016-8655) Linux Kernel Local Privilege Escalation Flaw Patch rolling out to Gentoo Sources kernels.

Just a quick note that I am walking the patch for CVE-2016-8655 down the gentoo-sources kernels.

Yesterday, I released the following kernels with the patch backported:


Updated: 12/08
Also patched:

Updated 12/09

Updated 12/11

If Alice does not get to the others before me, I will continue to walk down the versions until all of them are patched.


08 Dec 2016 1:24pm GMT

21 Nov 2016

feedPlanet Gentoo

Alexys Jacob: py3status v3.3

Ok I slacked by not posting for v3.1 and v3.2 and I should have since those previous versions were awesome and feature rich.

But v3.3 is another major milestone which was made possible by tremendous contributions from @tobes as usual and also greatly thanks to the hard work of @guiniol and @pferate who I'd like to mention and thank again !

Also, I'd like to mention that @tobes has become the first collaborator of the py3status project !

Instead of doing a changelog review, I'll highlight some of the key features that got introduced and extended during those versions.

The py3 helper

Writing powerful py3status modules have never been so easy thanks to the py3 helper !

This magical object is added automatically to modules and provides a lot of useful methods to help normalize and enhance modules capabilities. This is a non exhaustive list of such methods:

Powerful control over the modules' output

Using the self.py3.safe_format helper will unleash a feature rich formatter that one can use to conditionally select the output of a module based on its content.

Example format_string:

This will show artist - title if artist is present, title if title but no artist, and file if file is present but not artist or title.

"[[{artist} - ]{title}]|{file}"

More code and documentation tests

A lot of efforts have been put into py3status automated CI and feature testing allowing more confidence in the advanced features we develop while keeping a higher standard on code quality.

This is such as even modules' docstrings are now tested for bad formatting 🙂

Colouring and thresholds

A special effort have been put in normalizing modules' output colouring with the added refinement of normalized thresholds to give users more power over their output.

New modules, on and on !


The changelog is very big and the next 3.4 milestone is very promising with amazing new features giving you even more power over your i3bar, stay tuned !

Thank you contributors

Still a lot of new timer contributors which I take great pride in as I see it as py3status being an accessible project.

21 Nov 2016 12:40pm GMT

19 Nov 2016

feedPlanet Gentoo

Agostino Sarubbo: An alternative to git bisect with Gentoo and the live ebuild

Git bisect is absolutely powerful, but sometimes is more comfortable use emerge instead of compile the software outside the package manager.

That was my case with media-libs/jasper which I'm picking as example for this 'howto'

So basically, you are running Gentoo, you can install a live ebuild (9999) and you want to find which commit id fixes an issue. Let's see step-by-step what to do.

1) Clone the repository to obtain the commit ids and put them in a file

ago@blackgate ~ $ cd /tmp
ago@blackgate /tmp $ git clone https://github.com/mdadams/jasper.git
ago@blackgate /tmp $ cd jasper/
ago@blackgate /tmp/jasper $ git --no-pager log --format=%H > /tmp/commitlist.txt

The file should contain the git commit ids, for example:


2) Use a simple script which runs emerge and the command you need to test.

for COMMIT_ID in $( cat /tmp/commitlist.txt )
      echo "Testing with the commit id: "${COMMIT_ID}""
      EGIT_COMMIT="${COMMIT_ID}" emerge -q media-libs/jasper
      imginfo -f /tmp/myjpg.jpg
      echo -ne "\n\n\n"

With the EGIT_COMMIT variable from the git-* eclass, we can emerge the live ebuild at a specific commit id.
imginfo is in my case the command I need and then I print some blank lines to better separate the output of the commands and understand what is happening.

Now you need to wait and just check what is the output of the specified command.

- This howto looks to be valid when the project you are building is small; running this script with e.g. libreoffice will take months.
- This howto looks to be valid when you know that the problem is near to the commit master and will take few emerge cycles to found the issue.
- If you know that the problem is fixed e.g. a year ago, you can manually edit the commitlist.txt file and delete some recent ids, to have a specified and minor range of commits.

That's all.

19 Nov 2016 3:52pm GMT

10 Nov 2016

feedPlanet Gentoo

Alice Ferrazzi: 2016-11-8 Open Source Conference 2016 Tokyo summary

Open Source Conference 2016 Tokyo

Many people came to the Gentoo booth,
mainly students and Open Source users
asking for Gentoo information.

We gave away around 200 flyers, and
many many stickers during the two days.

Unfortunately the sticker we ordered
from unixsticker had some SVG problem.

We had also in exposition some esoteric
enviroment like the IS01 sharp,
off course mounting Gentoo as Native and
as prefix.
Of course one of the first things we tried
was the 5 minutes long Gentoo sl command.

image from: @NTSC_J

We also had a Gentoo notebook
running wayland (the one in the middle).

It was an amazing event and I would
like to thanks everyone that came to
the Gentoo booth, everyone that helped
making the Gentoo booth and all the
amazing Gentoo community.

10 Nov 2016 8:27am GMT

07 Nov 2016

feedPlanet Gentoo

Denis Dupeyron: SCALE 15x CFP is closing soon

Just a quick reminder that the deadline for proposing a talk to SCALE 15x is on November 15th. More information, including topics of interest, is available on the SCALE website.

SCALE 15x is to be held on March 2-5, 2017 at the Pasadena Convention Center in Pasadena, California, near Los Angeles. This is the same venue as last year and is much nicer than the original one form the years before.

I'll see you there.

07 Nov 2016 4:07am GMT

06 Nov 2016

feedPlanet Gentoo

Alice Ferrazzi: 2016-10-01 Gentoo Study Meeting

Gentoo Study Meeting talking (English Summary):  
homepage: http://gentoo.connpass.com/event/40906/  
Live broadcast: https://www.youtube.com/watch?v=j0SzulXKFwI  

    First Gentoo Study Meeting Tokyo with https://www.youtube.com/watch?v=j0SzulXKFwI  
    How to become Gentoo Developer introduction talk.  
            Contributing Ebuild:  
                - sending Git pull request  
                - Searching for a Mentor on proxy-maint  
                - Asking in #gentoo-proxy-maint  
                - Using https://bugs.gentoo.org/  
        Non committer developer:  
            - Contributing in Gentoo projects, with work that 
              not need gentoo git repository access.  
            - Contributing in the wiki (Not with translation also if 
              translator need the wiki translator permisson)  
    How to get help in Japanese:  
        - #gentoo-ja Freenode  
        - http://gentoo.slack.com http://slackin.aliceinwire.net  
        - https://github.com/gentoojp/issues  
        - ?forum  
        - http://www.gentoo.jp   
        - Gentoo勉強会 (Gentoo Study Meeting)  
        - https://wiki.gentoo.org/wiki/Handbook:Main_Page/ja  
    Gentoo News update:  
        Talk about Future EAPI 7 ulm slide  
            Question: When new EAPI are released?  
                I think there is not a setted release date for EAPI  
            New feature  
                - Runtime-switchable useflag  
                - eqwarn  
                - dohtml  
                - package.provided in profiles  
                - DESTTREE and INSDESTTREE  
    Talk about presence of Gentoo booth at Open Source Conference 
    2016 Tokyo http://www.ospn.jp/osc2016-fall/ :  
        - Stickers http://www.unixstickers.com/stickers/linux_os_distribution_stickers/gentoo-linux-os-badge-sticker  
        ask to fondation:  
        - Banner  
            size and format  
        - Table cover  
            size and format  
        Presenter: Matsuu san  
        Slide: Isucon 6 http://isucon.net/  
            - Team tuning speed contest  
            - This time was tuning speed contest on azure.microsoft.com  
                Only distribution with company can give sapport on azure.  
                Debian have a third party company that is supporting azure.  
                Gentoo also need something similar.  
            - Is good to do past problem for have more score on isucon.  
                vagrant is nice to use for the doing previous problem  
            - Go language, varnish+ESI, mysql  
            - https://github.com/matsuu/kataribe access log 
              analyzer for isucon/tuning  
            - sshrchttps://github.com/Russell91/sshrc 
              bring your .bashrc, .vimrc, etc. with you when you ssh.  
            - Matsu san as been choosen for become staff at the 
              isucon presentation in the future.  
        Presenter: @tkshnt  
        Slide: Report on last update  
            - let's make Gentoo goods shop for Gentoo-JP  
                previous OSC item:  
                    - t-shirt (@matsuu, @naota)  
                    - stickers (@matsuu)  
                next item:  
                    - Gentoo Tenugui (手拭い) https://goo.gl/SVeDHQ  
                OSC booth:  
                    - presentation  
                    - flyer  
                Design repository:  
                    - Github  
                        - project management  
                        - simple file upload  
        Presenter: @d_aki  
        Slide: my chaotic /etc/portage  
            - package.use can become chaotic  
            - /var/lib/portage/world difficult to 
              remeber when you added something and why  
            - let's use package.use directory and put file 
              name about what you are installing  
            - not what but why you installed the package  
        Presenter: alicef  
        Slide: How to contribute on Gentoo Github  
            - recently Gentoo CVS repository as been converted to Git  
            - Using the Github mirror is possible to send pull request.  
            - Good point of pull request:  
                - Code comment and review from more than one developer  
                - fast way to send ebuild patch upstream  
                - QA automatic check   
            - Bad point of pull request:  
                - the review are open to see to everyone  
                - basic git knowledge is needed  
            When we clone Gentoo repository:   
                Use git clone --depth=50   
                for fast pull request with less log information  
                git clone and git clone --depth=50 time difference:  
        Presenter: @usaturn  
        Slide: systemd-nspawn & btrfs  
            - On Gentoo using systemd-nspawn  
                - copy on write  
                - using subvolume we can make snapshot  
                - compress is possible  
                - cannot make swapfile  
                - unit is the process file manager  
                - using systemd stage 3 is simple to install  
                - not using syslog but journald  
                - network setting by networkd  
                - instead of cron there is timer  
                - instead of ntp there is systemd-timesyncd  
                - grub is not needed, instead systemd-boot 
                  (ex gummi boot) work as bootloader  
                - docker is not needed, instead systemd-nspawn 
                  using machinectl command (good for testing gentoo package)  
        Presenter: @naota344  
        Slide: automatically resolving conflict   
            Gentoo developer, btrfs, linux kernel, emacs, T-code  
                resolving conflict  
                    - when USE flag is needed it will ask to 
                      add a USE flag.  
                    - when circulation dependency is detected it will 
                      ask to remove a USE flag, for example  
                Why there is a conflict:  
                    - Before installing a new package, we 
                      have a package (for example perl-5.20) with all 
                      the the dependency package setted  
                    - when we are goin to update world and we get a 
                      new package update (for example perl-5.22),
                       also some dependency of perl-5.22 get new update  
                    - in this situation can happen that some dependency
                      to perl-5.20 get in conflict with perl-5.22  
                How can we fix such situation:  
                    - we have a option to add --reinstall-atoms="Y"
                      to emerge command (Y= name of the dependency
                      package that is causing problem)  
                    - this command will instead of just update the
                      package it will reinstall the package as if 
                      they are not installed, solving such dependency
                Why package is anyway deciding to automatically 
                not fixing dependency  
                    maybe because trying to fix all the dependecy will not
                    work correctly  
                When portage have conflict for many package  
                    it became more complicated and we will have a command
                    similar to this:  
                    --reinstall-atoms="A B C D E F G H I L M N ..."  
                For solving such problem there is emerge --reinstall-atoms
                    - automatically fixing circle dependecy  
                    - showing dependency graph  
                    - there is also a function for make try the 
                      dependency graph in a container  
                    - emerge analyzer tool  

06 Nov 2016 6:24pm GMT

25 Oct 2016

feedPlanet Gentoo

Diego E. Pettenò: Gentoo Miniconf 2016

Gentoo Miniconf, Prague, October 2016

As I noted when I resurrected the blog, part of the reason why I managed to come back to "active duty" within Gentoo Linux is because Robin and Amy helped me set up my laptop and my staging servers for singing commits with GnuPG remotely.

And that happened because this year I finally managed to go to the Gentoo MiniConf hosted as part of LinuxDays in Prague, Czech Republic.

The conference track was fairly minimal; Robin gave us an update on the Foundation and on what Infra is doing - I'm really looking forward to the ability to send out changes for review, instead of having to pull and push Git directly. After spending three years using code reviews with a massive repository I feel I like it and want to see significantly more of it.

Ulrich gave us a nice presentation on the new features coming with EAPI 7, which together with Michal's post on EAPI 6 made it significantly easier to pick up Gentoo again.

And of course, I managed to get my GnuPG key signed by some of the developers over there, so that there is proof that who's committing those changes is really.

But the most important part for me has been seeing my colleagues again, and meeting the new ones. Hopefully this won't be the last time I get to the Miniconf, although fitting this together with the rest of my work travel is not straightforward.

I'm hoping to be at 33C3 - I have a hotel reservation and flight tickets, but no ticket for the conference yet. If any of you, devs or users, is there, feel free to ping me over Twitter or something. I'll probably be at FOSDEM next year too, although that is not a certain thing, because I might have some scheduling conflicts with ENIGMA (unless I can get Delta to give me the ticket I have in mind.)

So once again thank you for CVU and LinuxDays for hosting us, and hopefully see you all in the future!

25 Oct 2016 6:02pm GMT

17 Oct 2016

feedPlanet Gentoo

Diego E. Pettenò: GnuPG Agent Forwarding with OpenPGP cards

Finally, after many months (a year?) absence, I'm officially back as a Gentoo Linux developer with proper tree access. I have not used my powers much yet, but I wanted to at least point out why it took me so long to make it possible for me to come back.

There are two main obstacles that I was facing, the first was that the manifest signing key needed to be replaced for a number of reasons, and I had no easy access to the smartcard with my main key which I've been using since 2010. Instead I set myself up with a separate key on a "token": a SIM-sized OpenPGP card installed into a Gemalto fixed-card reader (IDBridge K30.) Unfortunately this key was not cross-signed (and still isn't, but we're fixing that.)

The other problem is that for many (yet not all) packages I worked on, I would work on a remote system, one of the containers in my "testing server", which also host(ed) the tinderbox. This means that the signing needs to happen on the remote host, although the key cannot leave the smartcard on the local laptop. GPG forwarding is not very simple but it has sort-of-recently become possible without too much intrusion.

The first thing to know is that you really want GnuPG 2.1; this is because it makes your life significantly easier as the key management is handed over to the Agent in all cases, which means there is no need for the "stubs" of the private key to be generated in the remote home. The other improvement in GnuPG 2.1 is that there is better sockets' handling: on systemd it uses the /run/user path, and in general it uses a standard named socket with no way to opt-out. It also allows you to define an extra socket that is allowed to issue signature requests, but not modify the card or secret keys, which is part of the defence in depth when allowing remote access to the key.

There are instructions which should make it easier to set up, but they don't quite work the way I read them, in particular because they require a separate wrapper to set up the connection. Instead, together with Robin we managed to figure out how to make this work correctly with GnuPG 2.0. Of course, since that Sunday, GnuPG 2.1 was made stable, and so it stopped working, too.

So, without further ado, let's see what is needed to get this to work correctly. In the following example we assume we have two hosts, "local" and "remote"; we'll have to change ~/.gnupg/gpg-agent.conf and ~/.ssh/config on "local", and /etc/ssh/sshd_config on "remote".

The first step is to ask GPG Agent to listen to an "extra socket", which is the restricted socket that we want to forward. We also want for it to keep the display information in memory, I'll get to explain that towards the end.

# local:~/.gnupg/gpg-agent.conf

extra-socket ~/.gnupg/S.gpg-agent.remote

This is particularly important for systemd users because the normal sockets would be in /run and so it's a bit more complicated to forward them correctly.

Secondly, we need to ask OpenSSH to forward this Unix socket to the remote host; for this to work you need at least OpenSSH 6.7, but since that's now quite old, we can be mostly safe to assume you are using that. Unlike GnuPG, SSH does not correctly expand tilde for home, so you'll have to know the actual paths we want to write the unix at the right path.

# local:~/.ssh/config

Host remote
RemoteForward /home/remote-user/.gnupg/S.gpg-agent /home/local-user/.gnupg/S.gpg-agent.remote
ExitOnForwardFailure yes

Note that the paths need to be fully qualified and are in the order remote, local. The ExitOnForwardFailure option ensures that you don't get a silent failure to listen to the socket and fight for an hour trying to figure out what's going on. Yes, I had that problem. By the way, you can combine this just fine with the now not so unknown SSH tricks I spoke about nearly six years ago.

Now is the slightly trickier part. Unlike the original gpg-agent, OpenSSH will not clean up the socket when it's closed, which means you need to make sure it gets overwritten. This is indeed the main logic behind the remote-gpg script that I linked earlier, and the reason for that is that the StreamLocalBindUnlink option, which seems like the most obvious parameter to set, does not behave like most people would expect it to.

The explanation for that is actually simple: as the name of the option says, this only works for local sockets. So if you're using the LocalForward it works exactly as intended, but if you're using RemoteForward (as we need in this case), the one on the client side is just going to be thoroughly ignored. Which means you need to do this instead:

# remote:/etc/sshd/config

StreamLocalBindUnlink yes

Note that this applies to all the requests. You could reduce the possibility of bugs by using the Match directive to reduce them to the single user you care about, but that's left up to you as an exercise.

At this point, things should just work: GnuPG 2.1 will notice there is a socket already so it will not start up a new gpg-agent process, and it will still start up every other project that is needed. And since as I said the stubs are not needed, there is no need to use --card-edit or --card-status (which, by the way, would not be working anyway as they are forbidden by the extra socket.)

However, if you try at this point to sign anything, it'll just fail because it does not know anything about the key; so before you use it, you need to fetch a copy of the public key for the key id you want to use:

gpg --recv-key ${yourkeyid}
gpg -u ${yourkeyid} --clearsign --stdin

(It will also work without -u if that's the only key it knows about.)

So what is about keep-display in local:~/.gnupg/gpg-agent.conf? One of the issues I faced with Robin was gpg failing saying something about "file not found", though obviously the file I was using was found. A bit of fiddling later found these problems:

A bit of sniffing around the source code brought up that keep-display option, which essentially tells pinentry to ignore the session where gpg is running and only consider the DISPLAY variable when gpg-agent is started. This works for me, but it has a few drawbacks. It would not work correctly if you tried to use GnuPG out of the X11 session, and it would not work correctly if you have multiple X11 sessions (say through X11 forwarding.) I think this is fine.

There is another general drawback on this solution: if two clients connect to the same SSH server with the same user, the last one connecting is the one that actually gets to provide its gpg-agent. The other one will be silently overruled. I"m afraid there is no obvious way to fix this. The way OpenSSH itself handles this for the SSH Agent forwarding is to provide a randomly-named socket in /tmp, and set the environment variable to point at it. This would not work for GnuPG anymore because it now standardised the socket name, and removed support for passing it in environment variables.

17 Oct 2016 10:02pm GMT

16 Oct 2016

feedPlanet Gentoo

Patrick Lauer: Fixing gtk behaviour

Recently I've noticed all gtk2 apps becoming quite ... what's the word ... derpy?
Things like scrollbars not working and stuff. And by "not working" I mean the gtk3 behaviour of not showing up/down arrows and being a grey smudge of stupid.

So accidentally I stumbled over an old gentoo bug where it was required to deviate from defaults to have, like, icons and stuff.
That sounds pretty reasonable to me, but with gtk upstream crippling the Ad-Waiter, err, adwaita theme, because gtk3, this is a pretty sad interaction. And unsurprisingly by switching to the upstream default theme, Raleigh, gtk2 apps start looking a lot better.(Like, scrollbars and stuff)

The change might make sense to apply to Gentoo globally, locally for each user it is simply:

$ cat ~/.gtkrc-2.0
gtk-theme-name = "Raleigh"
gtk-cursor-theme-name = "Raleigh"

I'm still experimenting with 'gtk-icon-theme-name' and 'gtk-fallback-icon-theme', maybe that should change too. And as a benefit we can remove the Ad-Waiter from dependencies, possibly drop gnome-themes too, and restore a fair amount of sanity to gtk2.

16 Oct 2016 1:26pm GMT

Patrick Lauer: Changing console fontsize

Recently I accidentally aquired some "HiDPI" hardware. While it is awesome to use it quickly becomes irritating to be almost unable to read the bootup messages or work in a VT.
The documentation on fixing this is surprisingly sparse, but luckily it is very easy:

Now I'm wondering if such fonts can be embedded into the kernel so that on boot it directly switches to a 'nice' font, but just being able to read the console output is a good start ...

16 Oct 2016 10:09am GMT

13 Oct 2016

feedPlanet Gentoo

Diego E. Pettenò: What does #shellshock mean for Gentoo?

Gentoo Penguins with chicks at Jougla Point, Antarctica
Photo credit: Liam Quinn

This is going to be interesting as Planet Gentoo is currently unavailable as I write this. I'll try to send this out further so that people know about it.

By now we have all been doing our best to update our laptops and servers to the new bash version so that we are safe from the big scare of the quarter, shellshock. I say laptop because the way the vulnerability can be exploited limits the impact considerably if you have a desktop or otherwise connect only to trusted networks.

What remains to be done is to figure out how to avoid this repeats. And that's a difficult topic, because a 25 years old bug is not easy to avoid, especially because there are probably plenty of siblings of it around, that we have not found yet, just like this last week. But there are things that we can do as a whole environment to reduce the chances of problems like this to either happen or at least avoid that they escalate so quickly.

In this post I want to look into some things that Gentoo and its developers can do to make things better.

The first obvious thing is to figure out why /bin/sh for Gentoo is not dash or any other very limited shell such as BusyBox. The main answer lies in the init scripts that still use bashisms; this is not news, as I've pushed for that four years ago, while Roy insisted on it even before that. Interestingly enough, though, this excuse is getting less and less relevant thanks to systemd. It is indeed, among all the reasons, one I find very much good in Lennart's design: we want declarative init systems, not imperative ones. Unfortunately, even systemd is not as declarative as it was originally supposed to be, so the init script problem is half unsolved - on the other hand, it does make things much easier, as you have to start afresh anyway.

If either all your init scripts are non-bash-requiring or you're using systemd (like me on the laptops), then it's mostly safe to switch to use dash as the provider for /bin/sh:

# emerge eselect-sh
# eselect sh set dash

That will change your /bin/sh and make it much less likely that you'd be vulnerable to this particular problem. Unfortunately as I said it's mostly safe. I even found that some of the init scripts I wrote, that I checked with checkbashisms did not work as intended with dash, fixes are on their way. I also found that the lsb_release command, while not requiring bash itself, uses non-POSIX features, resulting in garbage on the output - this breaks facter-2 but not facter-1, I found out when it broke my Puppet setup.

Interestingly it would be simpler for me to use zsh, as then both the init script and lsb_release would have worked. Unfortunately when I tried doing that, Emacs tramp-mode froze when trying to open files, both with sshx and sudo modes. The same was true for using BusyBox, so I decided to just install dash everywhere and use that.

Unfortunately it does not mean you'll be perfectly safe or that you can remove bash from your system. Especially in Gentoo, we have too many dependencies on it, the first being Portage of course, but eselect also qualifies. Of the two I'm actually more concerned about eselect: I have been saying this from the start, but designing such a major piece of software - that does not change that often - in bash sounds like insanity. I still think that is the case.

I think this is the main problem: in Gentoo especially, bash has always been considered a programming language. That's bad. Not only because it only has one reference implementation, but it also seem to convince other people, new to coding, that it's a good engineering practice. It is not. If you need to build something like eselect, you do it in Python, or Perl, or C, but not bash!

Gentoo is currently stagnating, and that's hard to deny. I've stopped being active since I finally accepted stable employment - I'm almost thirty, it was time to stop playing around, I needed to make a living, even if I don't really make a life - and QA has obviously taken a step back (I still have a non-working dev-python/imaging on my laptop). So trying to push for getting rid of bash in Gentoo altogether is not a good deal. On the other hand, even though it's going to be probably too late to be relevant, I'll push for having a Summer of Code next year to convert eselect to Python or something along those lines.

Myself, I decided that the current bashisms in the init scripts I rely upon on my servers are simple enough that dash will work, so I pushed that through puppet to all my servers. It should be enough, for the moment. I expect more scrutiny to be spent on dash, zsh, ksh and the other shells in the next few months as people migrate around, or decide that a 25 years old bug is enough to think twice about all of them, o I'll keep my options open.

This is actually why I like software biodiversity: it allows to have options to select different options when one components fail, and that is what worries me the most with systemd right now. I also hope that showing how bad bash has been all this time with its closed development will make it possible to have a better syntax-compatible shell with a proper parser, even better with a proper librarised implementation. But that's probably hoping too much.

13 Oct 2016 7:02pm GMT

Diego E. Pettenò: The end of an era, the end of the tinderbox

I'm partly sad, but for the most part this is a weight that goes away from my shoulders, so I can't say I'm not at least in part joyful of it, even though the context in which this is happening is not exactly what I expected.

I turned off the Gentoo tinderbox, never to come back. The S3 storage of logs is still running, but I've asked Ian to see if he can attach everything at his pace, so I can turn off the account and be done with it.

Why did this happen? Well, it's a long story. I already stopped running it for a few months because I got tired of Mike behaving like a child, like I already reported in 2012 by closing my bugs because the logs are linked (from S3) rather than attached. I already made my position clear that it's a silly distinction as the logs will not disappear in the middle of nowhere (indeed I'll keep the S3 bucket for them running until they are all attached to Bugzilla), but as he keeps insisting that it's "trivial" to change the behaviour of the whole pipeline, I decided to give up.

Yes, it's only one developer, and yes, lots of other developers took my side (thanks guys!), but it's still aggravating to have somebody who can do whatever he likes without reporting to anybody, ignoring Council resolutions, QA (when I was the lead) and essentially using Gentoo as his personal playground. And the fact that only two people (Michał and Julian) have been pushing for a proper resolution is a bit disappointing.

I know it might feel like I'm taking my toys and going home - well, that's what I'm doing. The tinderbox has been draining on my time (little) and my money (quite more), but those I was willing to part with - draining my motivation due to assholes in the project was not in the plans.

In the past six years that I've been working on this particular project, things evolved:

You can see that it's not that 'm trying to avoid spending time to engineer solutions. It's just that I feel that what Mike is asking is unreasonable, and the way he's asking it makes it unbearable. Especially when he feigns to care about my expenses - as I noted in the previously linked post, S3 is dirty cheap, and indeed it now comes down to $1/month given to Amazon for the logs storage and access, compared to $600/month to rent the cabinet at Hurricane.

Yes, it's true that the server is not doing only tinderboxing - it also is running some fate instances, and I have been using it as a development server for my own projects, mostly open-source ones - but that's the original use for it, and if it wasn't for it I wouldn't be paying so much to rent a cabinet, I'd be renting a single dedicated server off, say, Hetzner.

So here we go, the end of the era of my tinderbox. Patrick and Michael are still continuing their efforts so it's not like Gentoo is left without integration test, but I'm afraid it'll be harder for at least some of the maintainers who leveraged the tinderbox heavily in the past. My contract with Hurricane expires in April; at that point I'll get the hardware out of the cabinet, and will decide what to do with it - it's possible I'll donate the server (minus harddrives) to Gentoo Foundation or someone else who can use it.

My involvement in Gentoo might also suffer from this; I hopefully will be dropping one of the servers I maintain off the net pretty soon, which will be one less system to build packages for, but I still have a few to take care of. For the moment I'm taking a break: I'll soon send an email that it's open season on my packages; I locked my bugzilla account already to avoid providing harsher responses in the bug linked at the top of this post.

13 Oct 2016 7:02pm GMT

Diego E. Pettenò: On professionalism (my first and last post explicitly about systemd)

I have been trying my best not to comment on systemd one way or another for a while. For the most part because I don't want to have a trollfest on my blog, because moderating it is something I hate and I'm sure would be needed. On the other hand it seems like people start to bring me in the conversation now from time to time.

What I would like to point out at this point is that both extreme sides of the vision are, in my opinion, behaving childishly and being totally unprofessional. Whether it is name-calling of the people or the software, death threats, insults, satirical websites, labeling of 300 people for a handful of them, etc.

I don't think I have been as happy to have a job that allows me not to care about open source as much as I did before as in the past few weeks as things keep escalating and escalating. You guys are the worst. And again I refer to both supporters and detractors, devs of systemd, devs of eudev, Debian devs and Gentoo devs, and so on so forth.

And the reason why I say this is because you both want to bring this to extremes that I think are totally uncalled for. I don't see the world in black and white and I think I said that before. Gray is nuanced and interesting, and needs skills to navigate, so I understand it's easier to just take a stand and never revise your opinion, but the easy way is not what I care about.

Myself, I decided to migrate my non-server systems to systemd a few months ago. It works fine. I've considered migrating my servers, and I decided for the moment to wait. The reason is technical for the most part: I don't think I trust the stability promises for the moment and I don't reboot servers that often anyway.

There are good things to the systemd design. And I'm sure that very few people will really miss sysvinit as is. Most people, especially in Gentoo, have not been using sysvinit properly, but rather through OpenRC, which shares more spirit with systemd than sysv, either by coincidence or because they are just the right approach to things (declarativeness to begin with).

At the same time, I don't like Lennart's approach on this to begin with, and I don't think it's uncalled for to criticize the product based on the person in this case, as the two are tightly coupled. I don't like moderating people away from a discussion, because it just ends up making the discussion even more confrontational on the next forum you stumble across them - this is why I never blacklisted Ciaran and friends from my blog even after a group of them started pasting my face on pictures of nazi soldiers from WW2. Yes I agree that Gentoo has a good chunk of toxic supporters, I wish we got rid of them a long while ago.

At the same time, if somebody were to try to categorize me the same way as the people who decided to fork udev without even thinking of what they were doing, I would want to point out that I was reproaching them from day one for their absolutely insane (and inane) starting announcement and first few commits. And I have not been using it ever, since for the moment they seem to have made good on the promise of not making it impossible to run udev without systemd.

I don't agree with the complete direction right now, and especially with the one-size-fit-all approach (on either side!) that tries to reduce the "software biodiversity". At the same time there are a few designs that would be difficult for me to attack given that they were ideas of mine as well, at some point. Such as the runtime binary approach to hardware IDs (that Greg disagreed with at the time and then was implemented by systemd/udev), or the usage of tmpfs ACLs to allow users at the console to access devices - which was essentially my original proposal to get rid of pam_console (that played with owners instead, making it messy when having more than one user at console), when consolekit and its groups-fiddling was introduced (groups can be used for setgid, not a good idea).

So why am I posting this? Mostly to tell everybody out there that if you plan on using me for either side point to be brought home, you can forget about it. I'll probably get pissed off enough to try to prove the exact opposite, and then back again.

Neither of you is perfectly right. You both make mistake. And you both are unprofessional. Try to grow up.

Edit: I mistyped eudev in the original article and it read euscan. Sorry Corentin, was thinking one thing and typing another.

13 Oct 2016 7:02pm GMT

Diego E. Pettenò: New devbox running

I announced it in February that Excelsior, which ran the Tinderbox, was no longer at Hurricane Electric. I have also said I'll start on working on a new generation Tinderbox, and to do that I need a new devbox, as the only three Gentoo systems I have at home are the laptops and my HTPC, not exactly hardware to run compilation all the freaking time.

So after thinking of options, I decided that it was much cheaper to just rent a single dedicated server, rather than a full cabinet, and after asking around for options I settled for Online.net, because of price and recommendation from friends. Unfortunately they do not support Gentoo as an operating system, which makes a few things a bit more complicated. They do provide you with a rescue system, based on Ubuntu, which is enough to do the install, but not everything is easy that way either.

Luckily, most of the configuration (but not all) was stored in Puppet - so I only had to rename the hosts there, changed the MAC addresses for the LAN and WAN interfaces (I use static naming of the interfaces as lan0 and wan0, which makes many other pieces of configuration much easier to deal with), changed the IP addresses, and so on. Unfortunately since I didn't start setting up that machine through Puppet, it also meant that it did not carry all the information to replicate the system, so it required some iteration and fixing of the configuration. This also means that the next move is going to be easier.

The biggest problem has been setting up correctly the MDRAID partitions, because of GRUB2: if you didn't know, grub2 has an automagic dependency on mdadm - if you don't install it it won't be able to install itself on a RAID device, even though it can detect it; the maintainer refused to add an USE flag for it, so you have to know about it.

Given what can and cannot be autodetected by the kernel, I had to fight a little more than usual and just gave up and rebuilt the two (/boot and / - yes laugh at me but when I installed Excelsior it was the only way to get GRUB2 not to throw up) arrays as metadata 0.90. But the problem was being able to tell what the boot up errors were, as I have no physical access to the device of course.

The Online.net server I rented is a Dell server, that comes with iDRAC for remote management (Dell's own name for IPMI, essentially), and Online.net allows you to set up connections to through your browser, which is pretty neat - they use a pool of temporary IP addresses and they only authorize your own IP address to connect to them. On the other hand, they do not change the default certificates, which means you end up with the same untrustable Dell certificate every time.

From the iDRAC console you can't do much, but you can start up the remove, JavaWS-based console, which reminded me of something. Unfortunately the JNLP file that you can download from iDRAC did not work on either Sun, Oracle or IcedTea JREs, segfaulting (no kidding) with an X.509 error log as last output - I seriously thought the problem was with the certificates until I decided to dig deeper and found this set of entries in the JNLP file:

 <resources os="Windows" arch="x86">
   <nativelib href="https://idracip/software/avctKVMIOWin32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin32.jar" download="eager"/>
 <resources os="Windows" arch="amd64">
   <nativelib href="https://idracip/software/avctKVMIOWin64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin64.jar" download="eager"/>
 <resources os="Windows" arch="x86_64">
   <nativelib href="https://idracip/software/avctKVMIOWin64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLWin64.jar" download="eager"/>
  <resources os="Linux" arch="x86">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  <resources os="Linux" arch="i386">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  <resources os="Linux" arch="i586">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  <resources os="Linux" arch="i686">
    <nativelib href="https://idracip/software/avctKVMIOLinux32.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux32.jar" download="eager"/>
  <resources os="Linux" arch="amd64">
    <nativelib href="https://idracip/software/avctKVMIOLinux64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux64.jar" download="eager"/>
  <resources os="Linux" arch="x86_64">
    <nativelib href="https://idracip/software/avctKVMIOLinux64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLLinux64.jar" download="eager"/>
  <resources os="Mac OS X" arch="x86_64">
    <nativelib href="https://idracip/software/avctKVMIOMac64.jar" download="eager"/>
   <nativelib href="https://idracip/software/avctVMAPI_DLLMac64.jar" download="eager"/>

Turns out if you remove everything but the Linux/x86_64 option, it does fetch the right jar and execute the right code without segfaulting. Mysteries of Java Web Start I guess.

So after finally getting the system to boot, the next step is setting up networking - as I said I used Puppet to set up the addresses and everything, so I had working IPv4 at boot, but I had to fight a little longer to get IPv6 working. Indeed IPv6 configuration with servers, virtual and dedicated alike, is very much an unsolved problem. Not because there is no solution, but mostly because there are too many solutions - essentially every single hosting provider I ever used had a different way to set up IPv6 (including none at all in one case, so the only option was a tunnel) so it takes some fiddling around to set it up correctly.

To be honest, Online.net has a better set up than OVH or Hetzner, the latter being very flaky, and a more self-service one that Hurricane, which was very flexible, making it very easy to set up, but at the same time required me to just mail them if I wanted to make changes. They document for dibbler, as they rely on DHCPv6 with DUID for delegation - they give you a single /56 v6 net that you can then split up in subnets and delegate independently.

What DHCPv6 in this configuration does not give you is routing - which kinda make sense, as you can use RA (Route Advertisement) for it. Unfortunately at first I could not get it to work. Turns out that, since I use subnets for the containerized network, I enabled IPv6 forwarding, through Puppet of course. Turns out that Linux will ignore Route Advertisement packets when forwarding IPv6 unless you ask it nicely to - by setting accept_ra=2 as well. Yey!

Again this is the kind of problems that finding this information took much longer than it should have been; Linux does not really tell you that it's ignoring RA packets, and it is by far not obvious that setting one sysctl will disable another - unless you go and look for it.

Luckily this was the last problem I had, after that the server was set up fine and I just had to finish configuring the domain's zone file, and the reverse DNS and the SPF records… yes this is all the kind of trouble you go through if you don't just run your whole infrastructure, or use fully cloud - which is why I don't consider self-hosting a general solution.

What remained is just bits and pieces. The first was me realizing that Puppet does not remove the entries from /etc/fstab by default, so I noticed that the Gentoo default /etc/fstab file still contains the entries for CD-ROM drives as well as /dev/fd0. I don't remember which was the last computer with a floppy disk drive that I used, let alone owned.

The other fun bit has been setting up the containers themselves - similarly to the server itself, they are set up with Puppet. Since the server used to be running a tinderbox, it used to also host a proper rsync mirror, it was just easier, but I didn't want to repeat that here, and since I was unable to find a good mirror through mirrorselect (longer story), I configured Puppet to just provide to all the containers with distfiles.gentoo.org as their sync server, which did not work. Turns out that our default mirror address does not have any IPv6 hosts on it ­- when I asked Robin about it, it seems like we just don't have any IPv6-hosted mirror that can handle that traffic, it is sad.

So anyway, I now have a new devbox and I'm trying to set up the rest of my repositories and access (I have not set up access to Gentoo's repositories yet which is kind of the point here.) Hopefully this will also lead to more technical blogging in the next few weeks as I'm cutting down on the overwork to relax a bit.

13 Oct 2016 7:02pm GMT