fidzu : Gentoo

The Arch Testers team is responsible with helping x86 devs test packages which are to be marked stable. Arch testers are respected members of the community which the x86 team has recognized as being reliable and trustworthy. Have you been wanting to get involved? This is the perfect starting point!

LINKS:
Gentoo/x86 Arch Testers
http://www.gentoo.org/proj/en/base/x86/at.xml

Writing Shell Scripts
http://www.linuxcommand.org/writing_shell_scripts.php

Commandline fu
http://www.commandlinefu.com/commands/browse

Doug Hellmann PyMOTW
http://www.doughellmann.com/PyMOTW/

Python email module
http://docs.python.org/release/2.6/library/email.html

The Easi-Reader Bookstand
http://www.amazon.com/Maxi-Aids-The-Easi-Reader-Bookstand/dp/B00014VVWG

Notebook sticker - Gentoo grey
http://www.linuxpusher.com/content/notebook-sticker-gentoo-grey

Download

31 Jul 2010 1:50am GMT

In my previous post where I declared myself up for hiring by those who really really want Ruby 1.9 sooner than we're currently planning to release it, I've said that the Ruby team doesn't want to "Pull a Python 3". I guess that I should explain a bit what I meant just there.

Ruby 1.9 and Python 3 are, conceptually, actually similar: while Python 3 actually make a much wider change in syntax as well as behaviour, both requires explicit, often non-trivial, porting of the software to work. Thus, they both require you to be slotted, installed side-by-side, with the older, more commonly used alternative, and so do the libraries and programs.

There is more similitude between the way the two are handled than you'd expect, mostly because the Python support for that has been partly copied out of Ruby NG stripped of a few features. These features are, for the most part, what I'd say protect us from pulling a Python 3.

As it is, installation of Python 3-powered packages is done once Python 3 is installed; and Python 3 is installed, unless explicitly masked, on every system, stable or not, because of the way Portage resolves dependencies. In my case, I don't care about having it around, so it's masked on all my systems (minus the tinderbox, for obvious reasons). You cannot decide whether a given package is installed for 2.6, 2.7 or 3.1, and you can only keep around safely one Python for the 2.x series as it will only install for that - which is going to be fun, because 2.7 seem to break so many things.

Ruby packages instead is coordinated through the use of the RUBY_TARGETS variable, that allows us (and you) to choose for which implementation (if supported) install a given package; you can even tweak it package-per-package via package.use! This, actually, makes the maintenance burden quite higher on our side because we have to make sure that all the dependency tree is up-to-date with a given target, on the other hand though it allows us be sure that the packages are available, and it would scream at us if they weren't (or rather Mr Bones would).

Most importantly, we don't need no stinkin' script like python-updater to add or remove an implementation; since the implementations are user-chosen via an USE-expanded variable (RUBY_TARGETS as I said), what you otherwise do with python-updater (or even perl-cleaner) is done through …. emerge -avuDN world.

Even though, I'll admit, there is one thing that at least python-updater seems to take into consideration and that for now we can't cater: using the Ruby interpreter rather than binding a library to be usable via Ruby; as I said in the post I linked above, it's one of the few cases that needs to be kinked out still before it can be unmasked. Again you can either wait or hire somebody to do the dirty job for you.

A note about the "stinkin' script" notion: one of the reason why I dislike the python-updater approach is that it lists a few "manual" packages to be rebuilt. The reason for that to happen is the old Python bug that caused packages to link the Python interpreter statically. The problem has since been fixed, but the list (which is very limited compared to what the tinderbox found at the time), is still present.

It is not all. I said at the start that right now Python 3 is installed unconditionally by default on all systems; we're going to do double- and triple-work to make sure that the same won't happen with Ruby 1.9 until we're ready to switch the defaults. Switching the defaults will likely take a much longer time; we're going to make 1.9 stable first, and start stabling packages supporting that… from there on, we'd be considering removing packages that are 1.8-only.

Well, to be honest, we're going to consider switching some packages that won't work with 1.9 (or JRuby) and neither have use nor they are maintained upstream. For good or bad, a lot of the packages in the tree have been added by the previous team members, and they, like us, often did so when they had a personal interest in the package… those packages often times are no longer maintained and are dead in the water, but we still carry them around.

Anyway, once again, the road is still bumpy, but it's not impossible; I'm not sure if we can get to unmasking it before end of the summer as I was hoping to, but we're definitely on track to provide a good user experience for Gentoo users who develop in Ruby, and most of the time, we can even provide a better upstream experience.

29 Jul 2010 3:02pm GMT

And the letter I received yesterday, certainly belongs into that category. A user from Austria send me a personal letter (with hand-written signature, imagine that) with a thanks as the aftermath of the unfortunate KDE 4.4.4 stabilisation. But apart from a simple and humbly accepted thanks (read all the comments for the linked post and I received some emails, too) a voucher for Amazon is included. A huge thanks from my wife (she will use it and she was surprised that there are actually users for the stuff I do) and of course from me, and I want to emphasize that I don't expect any reward for my work on Gentoo but the warm fuzzy feeling of doing something right (and the fun of course).

29 Jul 2010 7:47am GMT

It is under experimental/mips/stages/loongson/. Like:
http://mirrors.xmu.edu.cn/gentoo/experimental/mips/stages/loongson/
I should have done it long ago, just don't familiar with catalyst and catalyst lacks documentation.

Thanks to leio and robbat2 for making it happen.
http://bugs.gentoo.org/show_bug.cgi?id=330021

I have already included kernel modules in the stage3. You can find the corresponding kernel here:
http://www.gentoo-cn.org/~zhangle/

If you want to build your own kernel, use the source here:
http://dev.lemote.com/code/linux-loongson-community/wiki/WikiStart

You also need loongson overlay. Just use layman to add it. Of course you need to emerge layman first.
http://www.gentoo-cn.org/gitweb/?p=loongson.git;a=summary

If you find portage ask you to downgrade packages, you can try this trick
ACCEPT_KEYWORDS="x86 ~x86 ~mips"
However you need to understand that there is no guarantee that software installed in this way would work. But at least some of them will work. Like perl. I have been using this for a long time. Some packages are not so easy to downgrade. So maybe you need to use this too. But remember, if the package really works, better file a keyword bug asking devs to add mips keyword to it.

Please note this is experimental, so I am afraid there will be problems. If you hit any, please don't hesitate to contact me.

29 Jul 2010 1:51am GMT

Gentoo currently does not offer Ruby 1.9 available to users directly; there are a number of reasons for that, and can be summed up in what Alex described as "not pulling a Python 3 on our users". Right now, there are near to no packages that need Ruby 1.9, and a lot that does not even work with it. While a minority nowadays, a few won't even work if it's installed together with 1.8, let alone configured as primary provider for Ruby.

Me, Alex and Hans have been working for a long time to find a solution, and since last year the definite solution seems to be Ruby NG which I originally started in May 2009 after having trouble with keeping this very blog alive on the previous vserver - which nowadays only hosts the xine bugzilla .

The road has been still uphill from there, as the three pages of posts tagged with RubyNG on this blog can document; trouble with the ideas and implementations, compatibility problems, a huge web of dependencies between packages, various fixes, all of it makes the road to Ruby 1.9 quite difficult for us packagers. At the same time, we've been doing our best to ensure that what the users are given with proper software, of good quality. Maybe it's because I'm deeply involved with QA, maybe it is because I'm not writing production software daily, but I still think that we shouldn't be providing with half-assed software easily, just for the sake of it.

That means that most of the time we either don't add support for Ruby 1.9, or we go deeply into fixing the underlying issues to make sure that the software will work upstream, and not just in Gentoo (as otherwise there could be nasty surprises, like some I got, where an application works perfectly fine locally, where software is installed through Portage, and fails on Heroku that uses plain Rubygems). You can tell how that can be a PITA by looking at my github page - it lists mostly Ruby packages that I had to "fork" (branch, actually) to get the fixes in; mostly they have been merged upstream, sometimes they are dead in the water though.

All of this makes the situation quite complex; while I sort-of enjoy working with Ruby and these things, I also noted that it takes a very long time to get all the dependency web tested and fixed… and it's the sort of time that, in my personal free time, I just don't have. I have been packaging (and thus testing and fixing) a few packages that I triaged for a few job tasks, and some that I'm still using, using the paid work time, but that can't cut it to work for every package out there. I guess the same thing goes on for Alex, Hans and Gordon.

What's the bottom-line? Well, Hans in particular has been doing a huge work to port the ebuilds from the old gems.eclass to ruby-fakegem.eclass so that they can be installed when Ruby 1.9 is present without messing that up, even though they wouldn't work with it. This makes the day that we can get it unmasked much nearer. But there are quite a few cases where we can't just drop the old version so easily, and it mostly relates to non-gem bindings and the usage of Ruby as a scripting engine (rather than adding support for a library to Ruby itself). And this is without counting further issues like bundler not working altogether too well because it lacks dependency information, or getting Rubygems to refuse messing with the Portage-installed gems altogether (that is now much more feasible than before, since we no longer use the gem command from within Portage to install the stuff).

So what can you do to get this sooner? You can help out by making sure packages work with Ruby 1.9; when they have been positively tested not to work on that version, they are usually marked as so in the ebuild itself; for my part, I always note the problems with an Unicode right-pointed arrow, so running a fgrep command on the tree for "ruby19 →" should give you a very good idea of how many problems (and how many different problems there are out there).

You have no idea where to start with this thing? There is another option: hire me. Well, I would have liked to say "hire us", but it turns out at least both Alex and Hans are not looking to be hired for this, while a project of mine is delivered this week and then I have some extra time for the next few months. I wouldn't mind being paid to work full-time on getting Ruby 1.9-ready packages in the tree. I'm a registered freelancer in Italy so I have an European VAT ID and I can make proper invoices, so it's going to be all clear in the books. If you're interested you can contact me to discuss pricing and amount of work you're looking for.

Just please, stop harassing the team because we're not as fast as you'd like us to be… we're already doing a hell of a job in a hell of a hurry!

28 Jul 2010 5:22pm GMT

Hi folks,

here is my weekly report #9.

28 Jul 2010 3:10am GMT

Genkernel integration

1. Fixed few issues. By the way discovered some small in Dracut itself and it's already fixed now in mainline.
   There's one issues I'd like to mention. Dracut is designed with thought in mind that almost everything is in .ko modules. In Gentoo we cannot assume that, but fixing it isn't so simple. As a workaround I proposed symlinking modprobe and rmmod to /bin/true when dracut is called with --no-kernel (means without kernel modules) option (patch on LKML [0]). It prevents showing unnecessary errors. Although here comes desired feature of modprobe: check if module is built into kernel; only if not - try to load module. I'd like to handle it later, with pleasure.
2. Updated --help and genkernel.conf.
3. Refactorized the code.
4. Added support for Gentoo Splash as Dracut module.
   Now it's in branch gen2splash in my repo [1], but I'd like to see it in separate package. It might something like dracut-gentoo-modules.
   I've also sent bug report (#33010) [2] with tiny patch for splashutils. I'm using initrd.splash included there, but there's tiny cosmetic issue when using it as a standalone lib.
5. Enhanced with some options making easier to test Genkernel from git checkout even in combination with Dracut from git checkout.
6. Attached to patch to bug-report #330127 [4]

Still TODO

1. man page and some howto :-(
   I'm so lazy to do the same thing again and again (first updating case..esac parsing options, next updating --help usage, next - config and next… man page? What else?) that got some idea to make life easier. Imagine yourself that you create just one XML file describing options. Then you run generate_everything_I_need program.xml. In result you have program.8 manpage, example program.conf with comments describing options, program-options.sh with usage() function and with automatically generated loop parsing cmdline and config options (and even checking their correctness!) in some parse_options() function. Would it be cool? Look forward for that, I have to write it soon or later.

aidecoe's overlay

I decided to put my ebuilds into overlay [3] at last… I moved for now just my Dracut ebuilds, but later I'll put here updated Genkernel and Catalyst ebuilds, mentioned dracut-gentoo-modules ebuild and Plymouth to have all of this in one place and also for your convenience.

Plan for this week

Switching to Catalyst integration at last.

References

• Genkernel with Dracut integrated: http://github.com/aidecoe/genkernel/tree/gsoc2010
• [0] LKML: http://www.spinics.net/lists/linux-initramfs/
• [1] gen2splash module: http://github.com/aidecoe/dracut/tree/gen2splash/modules.d/50gen2splash/
• [2] initrd.splash fix: http://bugs.gentoo.org/show_bug.cgi?id=33010
• [3] my overlay: http://github.com/aidecoe/aidecoe-overlay/
• [4] Patch for Genkernel on BugZilla: http://bugs.gentoo.org/show_bug.cgi?id=330127

27 Jul 2010 9:55pm GMT

After my last notebook disappeared just a short while ago, I have now the successor of the old model. A Dell Vostro 3500 is now mine and installing Gentoo on it (amd64 on it this time) has been dissappointingly easy, kernel configuration was straight-forward and X done so fast, I did not even notice. Even in comparison to last year (let alone the first time I ever installed Gentoo on a computer) a lot improved at every corner of the ecosystem. The only drawback is the use of the proprietary broadcom-sta driver for the WLAN module.

Specification:

• Intel Core i5-450M (2.4Ghz, dual core with hyperthreading)
• 4GB RAM
• 1 HDMI port
• 350GB hard drive
• optical drive
• 4 USB ports

27 Jul 2010 2:10pm GMT

I can be quite difficult to read for what concerns alternative approaches to the same problem; while I find software diversity to be an integral part of the Free Software ideal and very helpful to find the best approach to various situations, I also am not keen on maintaining the same code many time because of that, and I'd rather have projects to share the same code to do the same task. This is why I think using FFmpeg for almost all the multimedia projects in the Free Software world is a perfectly good thing.

Yesterday, while trying to debug (with the irreplaceable help of Jürgen) a problem I was having with Gwibber (which turned out to be an out-of-date ca-certificates tree), I noted one strange thing with pycurl, related to this fact, that proves my point to a point.

CURL can make use of SSL/TLS encryption using one out of three possible libraries: OpenSSL, GnuTLS and Mozilla NSS. The first option is usually avoided by binary distributions because it is incompatible with some licensing terms; the third option is required for instance by the Thunderbird binary package in Gentoo as it is. By default Gentoo uses OpenSSL, that you like it or not.

When CURL is built against OpenSSL (USE="ssl -gnutls -nss"), PyCURL linked to libcrypto; given that my system is built with forced --as-needed, it also means it uses it. I found it quite strange so I went to look at it; if you rebuild CURL (and then PyCURL) with GnuTLS (USE="ssl gnutls -nss") you'll see that it only links to libgnutls, but if you look closer, it's using at least one libgcrypt symbol. Finally if you build it with Mozilla NSS (USE="ssl -gnutls nss") then it will warn that it didn't detect the SSL library used.

The problem here is that CURL seems not to provide a total abstraction of the SSL implementation it uses, and for proper threading support, PyCURL needs to run special code for the crypto-support library (libcrypto for OpenSSL; libgcrypt for GnuTLS). I'm sincerely not sure how big the problem would be when you mix and match the CURL and PyCURL implementations, I also have no idea what would happen if you were to use CURL with NSS and PyCURL with that (which will not provide locking for crypto at all). What I can tell you, is that if you change the SSL provider in CURL, you'd better rebuild PyCURL, to be on the safe side. And there is currently no way to let Portage do that automatically for you.

And if you are using CURL with NSS and you see Portage asking you to disable it in favour of GnuTLS or OpenSSL, you'll know why: PyCURL is likely to be your answer. At least once the bug will be addressed.

27 Jul 2010 10:04am GMT

Do you remember my latest libvirt ranting and the recent post about Kerberos and NFSv4 don't you? Well, let's tie the two up and consider a couple of good and bad things related to both.

First of all, as Daniel Berrange pointed out, QEmu does support IPv6; unfortunately it doesn't seem to work just as he supposed it to: even though my hostname resolves to both IPv4 and IPv6, QEmu by default only listens to v4. The same goes if you don't provide a listening socket (such as ""), and again the identical same happens with the default setting provided by libvirt (0.0.0.0). You can force it to listen to v6 by either providing a v6-only hostname, a v6 IP address or the v6 catch-all [::] which makes it work on both v6 and v4, lovely, isn't it?

Then, about libvirt-remote, as many pointed out it is possible to use it with SSH as user, but there are two catches there: the first is that with the way the arguments are passed down from virt-manager down to libvirt, to ssh and zsh on the other side, something goes funky; it works fine with bash because it splits the parameters again, but with zsh as login shell for my user it tries to call a binary called nc -U ... which as you might have guessed is not correct. The second problem is that even if you set the unix socket access for your user, it won't let it work if you are using SSH and the system is configured with PolicyKit. I guess this was designed to work in two distinct configuration (desktop and server) and trying to mix the two creates a bit of trouble.

This does not solve two problems though: the dangling connections that are kept alive even after closing virt-manager and its inability to provide diagnostic more human-readable than the Python exceptions. This became tremendously obvious today as I went to consider the idea of using Kerberos for the authentication of libvirt itself, given that it can do that via SASL. It would make more sense, since I'll be having a Kerberos install anyway at this point, to use the Kerberos credentials for more than a couple of services.

Using Kerberos for libvirt actually makes quite a bit of sense: you can set up properly TLS support for the connection and have an user-based authentication (rather than the whole host-based authentication that is supported with the TLS-only login). Setting up libvirt itself is not difficult, if it wasn't for the single problem that most of the documentation tells you to use /etc/libvirt/krb5.keytab while it'll be looking only at /etc/krb5.keytab by default - maybe it's worth for Gentoo to change the init script so that it searches for the one documented. After that, I can properly login on the libvirt-remote access with virt-manager and Kerberos…. but I still am having trouble with QEmu and VNC this time around.

Now a little note regarding pambase: as I've been brought to note the default configuration used by pambase with the kerberos USE flag enabled might not be well suited for all the sites using Kerberos right now. I know that, but Gentoo never pretended to give perfect defaults, or defaults that suit everybody; on the other hand I think it's important to give a default for Kerberos in our packaging. I'll have to talk with Robin or someone else for integrating a default regarding pam_ldap as well, since the LDAP guide we provide is hinting at the wrong solution for the PAM configuration, if the system also want to be a desktop.

Having found a decent way to provide multiple optional login systems for users is actually finally paving the way to provide token-based login that I talked about last year.

25 Jul 2010 10:35pm GMT

As previously mentioned, concurrentmodphp is currently available as a USE flag for dev-lang/php. It enables you to run two versions of PHP in parallel, loaded into your apache's mod_php.

With the advent of FastCGI/fpm in PHP-5.3.3 (very soon on a mirror near you), the preferred way to run multiple versions of PHP in parallel is CGI. Not only can you run several PHP versions independently, you also get proper script isolation (running each script as a different user, if you wish so) and thus enhanced security.

This is why I've decided to end support for USE=concurrentmodphp after php-5.3.3 and php-5.2.14. Versions after those will simply stop shipping the patches required to support this USE flag. Please use the time with PHP-5.3.3 to switch to fpm, if you need the functionality currently provided by concurrentmodphp.

24 Jul 2010 11:52pm GMT

I've been fighting the past few days with finding a solution to strengthen the internal security of my network. I'm doing this for two main reasons; from one side, having working IPv6 on the network means that I have to either set up a front-end firewall on the router, or I have to add firewalls on all the devices, and that's not really so nice to do; on the other side, I'd like to give access to either containers (such as the tinderbox) or other virtual machines to other people, developers for Gentoo or other projects.

I'm not ready to just give access to them as the network is because some of the containers and VMs have still password-login, and from there, well, there would be access to some stuff that is better kept private. Even though I might trust some of the people I'm thinking to give access to, I won't trust anybody else's security practice with accessing my system. And this is even more critical since I have/had NFS-writeable directories around the system, including the distfiles cache that the tinderbox works with.

Unfortunately, most of the alternatives I know for this only work with a single user ID, and that means among other things that I can't use them with Portage. So I decided to give a try to using NFSv4 and Kerberos. I'm not sure if I'll stick with that sincerely, since it makes the whole system a whole lot more complex and, as I'll show in a moment, it's also not really solving my problem at its root, so it's of little use to me.

The first problem is that the NFSv4 client support in Gentoo seems to have been totally broken up to now, bug #293593 was causing one of the necessary services to simply kill itself rather than running properly, and it was fun to debug. There is a library (libgssglue) that is used to select one out of a series of GSS API providers (either Kerberos or others); interestingly enough, this is yet another workaround for Linux missing libmap.conf and the problem with multiple implementations of the same interface. This library provides symbols that are also provided by the MIT-KRB5 GSS API library (libkrb5_gssapi); when linking the GSS client daemon (rpc.gssd) it has to link both, explicitly causing symbol collisions, sigh. Unfortunately this failed for two reasons again: .la files for libtirpc (don't ask) caused libtool to actually reorder the linking of libraries, getting the wrong symbols in (bad, and shows again why we should be dropping those damn files), plus there was a stupid typo in the configure.ac file for nfs-utils where instead of setting empty enable_nfsv41 variable they set enable_nfsv4, which in turn caused libgssglue from not being searched for.

The second problem is that right now, as I know way too well, we have no support for Kerberos in the PAM configuration for Gentoo, this is one of the reason why I was considering more complex PAM configurations - main problem is that most of the configurations you find in tutorials, and those that I was proposed, make use of pam_deny to allow using either pam_unix or pam_krb5 at the same time; this in turn breaks the proper login chain used by the GNOME Keyring for instance. So I actually spent some time to find a possible solution to this. Later today when I have some extra time I'll be publishing a new pambase package with Kerberos support. Note: this, and probably a lot more features of pambase, will require Linux-PAM. This is because the OpenPAM syntax is just basic, while Linux-PAM allows much more flexibility. Somebody will have to make sure that it can work properly on FreeBSD!

There is also a request for pam_ccreds to cache the credentials when running offline, I'm curious about it but upstream does not seem to be working on it as much as it should, so I'm not sure if it's a good solution.

Unfortunately, as I said, NFSv4 does not seem so much of a good solution; beside the still lack of IPv6 support (which would have been nice to have, but it's not required for me), if I export the distfiles over NFSv4 (with or without Kerberos), the ebuild fetch operation remain stuck in D-state for the process (blocked on I/O wait). And if I try to force the unmount of the mounted, blocked filesystem, I get the laptop to kernel panic entirely. Now, to make the thing easier to me I'm re-using a Gentoo virtual machine (which I last used for writing a patch for the SCTP support in the kernel) to see if I can reproduce the problem there, and get to fix it, in one way or another.

Unfortunately I've spent the whole night working and trying to get this working, so now I'll try to get some rest at least (it's 9.30am, sigh!). All the other fixes will wait for tomorrow. On the other hand, I'd welcome thank yous if you find the help on Kerberos appreciated; organisations who would like to have even better Gentoo support for Kerberos are welcome to contact me as well…

23 Jul 2010 6:44am GMT

A lot of Gentoo teams have bug queues. Each herd, the security team, the arch teams, and each individual developer. It's better when these queues are short, but not only because it looks better and solves problems that people have.

Long bug lists tend to grow over time. They're harder to navigate, it's just easier to skip an easy to fix bug, or forget to answer user's question. Even browsing such list takes time. Recently the x86 team has reduced the queue size to 10 bugs or so. It's now much easier to manage, and stabilizations take a lot less time. Also, we can pay more attention to issues that prevent or delay stabilizations.

So the summary is... keep your bug queues short, and don't be afraid to put an entry on staffing needs page if needed.

Oh, and if you're a user looking for a way to contribute, helping shorten someone's bug list is a great way to do so.

22 Jul 2010 4:43am GMT

The past week I spend on integrating Dracut with Genkernel and eventually manged that [0][1]. How's it done? I've created additional script (gen_dracut.sh) which is sourced conditionally. My initial idea (discussed with Andrew Gaffney) was to simply replace the body of create_initramfs in gen_initramfs.sh with dracut evaluation, but recently I've got idea to leave existing generation tool and give to user choice to not use dracut engine but the internal one. It's better since it's less intrusive and if dracut fails for some reason there's easy rescue.

How can one use Genkernel with Dracut? (And here comes what's not yet done… - docs) Just the same as before to generate initramfs, but difference is in kernel arguments. In most cases no arguments are needed. Most of things are handled automatically with udev magic. For details there's man 8 dracut. If you'd like to use it ASAP, just checkout my repo (branch: gsoc2010) and run it like here: sudo ./genkernel --local --lvm initramfs

Moreover I was trying to organize Plymouth maintenance. Contacted with several people and there's a chance for it to be put into Portage tree. Or… maybe You are a developer willing to handle it?

And thinking of splash stuff… for now I've used Plymouth, but I guess I'd have to write some module for Dracut supporting old splash for people who don't like new trendy eye-candy and unnecessarily animating stuff … ;-)

There's also updated dracut ebuild (see bug #329003 [2]). If we're around the Dracut itself, unfortunately there's RHEL6 release coming and that's why there's no time for my latest patches to be merged into mainline by Harald. Hope the phase will pass quickly and then my patches will come into.

References:

• [0] my fork: http://github.com/aidecoe/genkernel/tree/gsoc2010
• [1] single patch: http://wklej.org/hash/69a5df3848/
• [2] dracut-006-r1.ebuild: http://bugs.gentoo.org/show_bug.cgi?id=329003

20 Jul 2010 10:21pm GMT

Greetings, following is the weekly update for gpypi2 project. It's main purpose is to generate ebuilds from Python Package Index. Quick links to project info:

• Repository, issue tracker
• Hudson CI instance
• Sphinx documentation generated by hudson on each commit

Very successful 2 weeks. Two days of vacations and nice shiney configuration utility together with questionnaire.

Previous 2 weeks (5th - 19th July)

Task: Write configuration skeleton

Currently, gpypi2 code only takes configuration from command line.
I'd like to extend that to config file (.ini), get data directly from setup.py
and interactive questionaire. In order for that to function properly,
configuration needs to be done in right sequence.

Following week I will write down the design and implement initial working prototype.

Configuration dispatcher fully implemented! Also we have already working questionnaire functionality. Details can be found at http://docs.fubar.si/gpypi2/userguide.html#configuration

Task: sync command - populate overlay with successful ebuilds from pypi

When I will get bored or stuck with task #1, I'll start working on command that will
generate ebuilds from all packages in pypi to an overlay.

Initial "sync" command is implemented. Still lacking proper error handling, metadata generation and manifest update support. Will be a lot easier to implement with current configuration management.

Additional, following mini-tasks were performed:

• fixed echo command to work properly
• correct handling of whitespace in rendered ebuilds
• replace 4 spaces with a tab in rendered ebuilds

Upcoming week (20th - 26th July)

Task: fix TODOs in the code

Code currently has about 60 TODOs that need to be address for a better codebase.

Task: Implement "python setup.py bdist_ebuild" command

I'll peek into distutils heart and implement most basic bdist command to output an ebuild.
I will basically the same stuff we currently do,
except it will parse data from setup.py itself and not through pypi.

20 Jul 2010 2:52pm GMT

I've been smashing documentation bugs left and right since getting back from vacation, as well as searching out old documents and project pages and fixing 'em up.

Most of the updates have been to the installation & Portage handbooks, but there are many changes to the other documentation, including the desktop guides for graphics cards, and my Xfce guide. There's even a new doc on Logcheck, written by one of our developers.

Here's a brief summary of what I've done in the last week:

New documentation:

• Logcheck guide: Thanks to phajdan.jr. bug 322223

Handbook updates:

• Change ccache recommendation; it's really only for developers: bug 327945
• Use layman rather than gensync for working with Portage overlays: bug 305047
• Add another note on IA32 emulation in the kernel for (non-)multilib users: bug 326691
• Fix file verification process for the Alpha, AMD64, ARM, HPPA, IA64, and x86 handbooks: bug 283402. This was an old one: when we went to the weekly media autobuilds, Release Engineering signed the files with a new GPG key, and changed how the files were signed. All the handbooks need to be updated, as they still have the old keys and instructions from the previous release.
• Update installation instructions for the autobuilds. Completed Alpha, AMD64, ARM, HPPA, IA64, and x86: bug 283402, bug 292726, bug 260403. Still need to do PPC, PPC64, Sparc, and possibly MIPS, if they have sufficiently recent media.
• Use -march=core2 for recent Intel EM64T chips, rather than the old -march=nocona. Fix MCE section of kernel config. Add new Atom processor type: bug 323381
• Update Grub documentation links. Upstream removed all grub legacy instructions in favor of grub2, which won't be stable any time soon. Fixed the handbooks and other docs to use the offsite Grub Wiki: bug 328679
• Fix a missing fstab. Gave ARM the same generic fstab example as the other arches: bug 328095
• category/package move for chkrootkit

Desktop doc updates:

• Xfce guide: Change USE flags for opera; no longer needs qt-static. bug 328087
• nVidia guide: Use new driver installation methods. Add links to xorg-server guide to get X configured before dealing with nVidia-specific issues. Update kernel and module info. bug 307481
• ATI FAQ: General cleanups. Add R800 (Evergreen) info. Remove old GATOS project text. Update Catalyst availability section.

Other doc updates:

• FAQ: Update Grub documentation link. Update gcc -march info for x86 and AMD64. Fix internal GuideXML code. bug 328679
• Quickinstall guides (x86, LVM2+RAID): Fix ccache recommendation. bug 327945
• LDAP guide: use more recent 2.3 configuration file shipped with the ebuild. bug 325497
• SHOUTcast guide: Miscellaneous typo fixes. bug 323401
• IPv6 guide: update net-dns/totd info now that it's stable. Fix GuideXML and minor text issues throughout. bug 326771. This doc presents an ongoing problem, because it recommends a package it shouldn't. I sent an email to the gentoo-dev mailing list asking for help with this one.
• AMD64 FAQ: Update Flash installation info. Adobe decided to drop 64-bit versions (again) beginning in version 10.1, and our developers had to mask 10.0 for security reasons. This means that there is no Adobe Flash for non-multilib profile users. And nspluginwrapper is (once again) too unstable, so 32-bit Flash with a 64-bit browser is not recommended. Probably will have to install firefox-bin or some other 32-bit browser. Stupid Adobe.
• UTF-8 guide: Fix wrong category for the Xfce terminal, leftover from when it was moved out of xfce-extra. bug 328977
• Fix metadoc index for retired developers and add logcheck guide entry

Project page updates:

• Overlays userguide: Extensive GuideXML, grammar, etc. rewrites to make the guide more readable and more helpful. Add more instructions for things like keywording packages per the Portage handbook. Add SCM homepage links. This series of updates was prompted by bug 305047, the gensync to layman change.
• GUIs: Update retired developers
• PR: Add note stating that PR does not offer user support, and list available support resources. Hopefully this will cut down on the amount of support requests the PR team receives in our inbox every month.

Website updates:

• IRC: Add the Qt project channel, #gentoo-qt. bug 328665

One of my fellow developers, jkt, has been helping out a bit in the last couple of weeks, closing bug 301840 and bug 325885. This was especially important when I was on vacation and then out sick. I'm always happy when someone besides me steps up and gets our docs into shape. Thanks, Jan!

So that's about it. There are still plenty of open documentation bugs, but the list has shrunk significantly. My biggest project now is to finish the rest of the handbooks for the weekly autobuild instructions. The rest of our open bugs will require just as many hours and days to fix, as large portions of our handbooks and guides will need to be rewritten. Hopefully I can at least get the autobuild updates done in the next few days.

20 Jul 2010 6:35am GMT