17 Apr 2014

feedPlanet Gentoo

Sven Vermeulen: If things are weird, check for policy.29

Today we analyzed a weird issue one of our SELinux users had with their system. He had a denial when calling audit2allow, informing us that sysadm_t had no rights to read the SELinux policy. This is a known issue that has been resolved in our current SELinux policy repository but which needs to be pushed to the tree (which is my job, sorry about that). The problem however is when he added the policy - it didn't work.

Even worse, sesearch told us that the policy has been modified correctly - but it still doesn't work. Check your policy with sestatus and seinfo and they're all saying things are working well. And yet … things don't. Apparently, all policy changes are ignored.

The reason? There was a policy.29 file in /etc/selinux/mcs/policy which was always loaded, even though the user already edited /etc/selinux/semanage.conf to have policy-version set to 28.

It is already a problem that we need to tell users to edit semanage.conf to a fixed version (because binary version 29 is not supported by most Linux kernels as it has been very recently introduced) but having load_policy (which is called by semodule when a policy needs to be loaded) loading a stale policy.29 file is just… disappointing.

Anyway - if you see weird behavior, check both the semanage.conf file (and set policy-version = 28) as well as the contents of your /etc/selinux/*/policy directory. If you see any policy.* that isn't version 28, delete them.

17 Apr 2014 7:01pm GMT

16 Apr 2014

feedPlanet Gentoo

Alexys Jacob: py3status v1.4

I'm glad to announce the release of py3status-1.4 which I'd like to dedicate to @guiniol who provided valuable debugging (a whole Arch VM) to help me solve the problem he was facing (see changelog).

I'm gathering wish lists an have some (I hope) cool ideas for the next v1.5 release, feel free to post your most adventurous dreams !

changelog

contributors

Special thanks to this release's contributors !

16 Apr 2014 10:18am GMT

14 Apr 2014

feedPlanet Gentoo

Gentoo News: Action required: Password reset on all Gentoo services

Recent versions of OpenSSL were found to be affected by an information disclosure vulnerability related to TLS heartbeats, nicknamed Heartbleed. It allows attackers to read up to 64kb of random server memory, possibly including passwords, session IDs or even private keys.

After the public disclosure on April 7, we have confirmed that several services provided by Gentoo Infrastructure were vulnerable as well. We have immediately updated the affected software, recreated private keys, reissued certificates, and invalidated all running user sessions. Despite these measures, we cannot exclude the possibility of attackers exploiting the issue during the time it was not publicly known to gain access to credentials or session IDs of our users. There are currently no indications this has happened.

However, to be safe, we are asking you to reset your passwords used for Gentoo services within the next 7 days. You need to take action if you have an account on one of the following sites:

After 7 days, we will be removing all passwords to avoid abuse. For more information and the full announcement, visit http://infra-status.gentoo.org/notice/20140413-heartbleed.

14 Apr 2014 9:02am GMT

10 Apr 2014

feedPlanet Gentoo

Luca Barbato: Security and Tools

Everybody should remember than a 100% secure device is the one unplugged and put in a safe covered in concrete. There is always a trade-off on the impairment we inflict ourselves in order to stay safe.

Antonio Lioy

In the wake of the heartbleed bug. I'd like to return again on what we have to track problems and how they could improve.

The tools of the trade

Memory checkers

I wrote in many places regarding memory checkers, they are usually a boon and they catch a good deal of issues once coupled with good samples. I managed to fix a good number of issues in hevc just by using gcc-asan and running the normal tests and for vp9 took not much time to spot a couple of issues as well (the memory checkers aren't perfect so they didn't spot the faulty memcpy I introduced to simplify a loop).

If you maintain some software please do use valgrind, asan (now also available on gcc) and, if you are on windows, drmemory. They help you catch bugs early. Just beware that sometimes certain versions of clang-asan miscompile. Never blindly trust the tools.

Static analyzers

The static analyzers are a mixed bag, sometimes they spot glaring mistakes sometimes they just point at impossible conditions.
Please do not put asserts to make them happy, if they are right you just traded a faulty memory access for a deny of service.

Other checkers

There are plenty other good tools from the *san family one can use, ubsan is maybe the newest available in gcc and it does help. Valgrind has plenty as well and the upcoming drmemory has a good deal of interesting perks, if only upstream hadn't been so particular with release process and build systems you'd have it in Gentoo since last year…

Regression tests

I guess everybody is getting sick of me talking about fuzzy testing or why I spent weeks to have a fast regression test archive called playground for Libav and I'm sure everybody in Gentoo is missing the tinderbox runs Diego used to run.
Having a good and comprehensive batch of checks to make sure new code and new fixes do not have the uncalled side effect of breaking stuff is nice, coupled with git bisect makes backporting to fix issues in release branches much easier.

Debuggers

We have gdb, that works quite well, and we have lldb that should improve a lot. And many extensions on top of them. When they fail we can always rely on printf, or not

What's missing

Speed

If security is just an acceptable impairment over performance in order not to crash, using the tools mentioned are an acceptable slow down on the development process in order not to spend much more time later tracking those issues.

The teams behind valgrind and *san are doing their best to just make the execution three-four times as slow when the code is instrumented.

The static analyzers are usually just 5 times as slow as a normal compiler run.

A serial regression test run could take ages and in parallel could make your system not able to do anything else.

Any speed up there is a boon. Bigger hardware and automation mitigates the problem.

Precision

While gdb is already good in getting you information out of gcc-compiled data apparently clang-compiled binaries are a bit harder. Using lldb is a subtle form of masochism right now for many reasons, it getting confused is just the icing of a cake of annoyance.

Integration

So far is a fair fight between valgrind and *san on which integrates better with the debuggers. I started using asan mostly because made introspecting memory as simple as calling a function from gdb. Valgrind has a richer interface but is a pain to use.

Reporting

Some tools are better than other in pointing out the issues. Clang is so far the best with gcc-4.9 coming closer. Most static analyzers are trying their best to deliver the big picture and the detail. gdb so far is incredibly better compared to lldb, but there are already some details in lldb output that gdb should copy.

Thanks

I'm closing this post thanking everybody involved in creating those useful, yet perfectible tools, all the people actually using them and reporting bugs back and everybody actually fixing the mentioned bugs so I don't have to do myself alone =)

Everything is broken, but we are fixing most of it together.

10 Apr 2014 9:51am GMT

08 Apr 2014

feedPlanet Gentoo

Alexys Jacob: mongoDB 2.4.10 & pymongo 2.7

I'm pleased to announce those latest mongoDB related bumps. The next version bump will be for the brand new mongoDB 2.6 for which I'll add some improvements to the Gentoo ebuild so stay tuned ;)

mongodb-2.4.10

All mongodb-2.4.10 changelog here.

pymongo-2.7

All pymongo-2.7 changelog here.

08 Apr 2014 4:05pm GMT

06 Apr 2014

feedPlanet Gentoo

Raúl Porcel: New AArch64/arm64 stage3 available

Hello all,

Following up with my AArch64/ARM64 on Gentoo post, in the last months Mike Frysinger (vapier) has worked in bringing arm64 support to the Gentoo tree.

He has created the profiles and the keyword, along with keywording a lot of packages(around 439), so props to him.

Upstream qemu-2.0.0-rc now supports aarch64/arm64, so I went ahead and created a stage3 using the new arm64 profile. Thanks to Mike I didn't had to fight with a lot of problems like in the previous stage3.

For building I just had to have this in my package.keywords file:

=app-text/opensp-1.5.2-r3 **
=dev-util/gperf-3.0.4 **
=sys-apps/busybox-1.21.0 **
=app-text/sgml-common-0.6.3-r5 **
=app-text/openjade-1.3.2-r6 **
=app-text/po4a-0.42 **
=dev-perl/Text-CharWidth-0.40.0 **
=dev-perl/SGMLSpm-1.03-r7 **
=dev-util/intltool-0.50.2-r1 **
=dev-perl/XML-Parser-2.410.0 **
=dev-perl/Text-WrapI18N-0.60.0 **
=sys-apps/coreutils-8.22

And in my package.use file:

sys-apps/busybox -static

coreutils-8.21 fails to build, 8.22 built fine. And building busybox with USE="static" still fails.

Also I've just found out that USE="hpn" on net-misc/openssh makes the client segfault. Not sure if its because of qemu or because the unaligned accesses hpn had aren't happy on arm64. So if you plan to use the ssh client in the arm64 chroot, make sure you have USE="-hpn"

By the way, app-arch/lbzip2 seems to fail to run here, not sure if its because of qemu or it simply doesn't work on arm64. It segfaults.

You can download it from: http://gentoo.osuosl.org/experimental/arm/arm64

I've also starting to upload some binary packages: http://tinderbox.dev.gentoo.org/default/linux/arm64/

Also, if someone wants to give us access to arm64 hardware, we would be really happy :)


06 Apr 2014 12:54pm GMT

05 Apr 2014

feedPlanet Gentoo

Patrick Lauer: "smart" software

1) Grab webbwrowser
2) Enter URL
3) Figure out that webbrowser doesn't want to use HTTP because ... saturday? I don't know, but ass'u'me'ing that some URLs are ftp is just, well stupid, because your heuristic is whack.

Or, even more beautiful:

$ clementine
18:02:59.662 WARN  unknown                          libpng warning: iCCP: known incorrect sRGB profile 
Bus error



I have no idea what this means, so I'll be explicitly writing http:// at the beginning of all URL I offer to Firefox. And Clementine just got a free travel to behind the barn, where it'll get properly retired - after all it doesn't do the simple job it was hired to do. Ok, before it randomly didn't play "some" music files because gstreamer, which makes no sense either, but open rebellion will not have happy results.

I guess the moral of the story is: Don't misengineer things, clementine should output music and not be a bus driver. Firefox should not interpret-dance the URLS offered to it, but since it's still less retarded than the competition it'll be allowed to stay a little bit longer.

Sigh. Doesn't anyone engineer things anymore?

05 Apr 2014 10:29am GMT

02 Apr 2014

feedPlanet Gentoo

Gentoo News: Gentoo Monthly Newsletter - March 2014

The March 2014 GMN issue is now available online.

This month on GMN:

02 Apr 2014 7:02am GMT

31 Mar 2014

feedPlanet Gentoo

Gentoo Monthly Newsletter: Gentoo Monthly Newsletter: March 2014

Gentoo News

Interview with Tom Wijsman (TomWij)

(by David Abbott)

1. To get started, can you give us a little background information about yourself?

Tom Wijsman is my full name; TomWij is formed as a shorter nickname, taking the first three letters twice. 24 years is how long I've been alive and Antwerp, Belgium is where you can find me eating, hanging around, sleeping, studying, working and so on…

At university, I study the Computer Science programme with a specialization in Software Engineering. As the last year starts now, my student time is almost over.

Over the last years, a lot of programming languages have passed by there and on Gentoo Linux; which makes participation in both of them really worth it.

Besides programming, listening and playing some music is what I like to do. Currently I own an electric guitar, which sometimes is played on; but maybe I go for another instrument soon and practice in a more dedicated manner. Occasionally, I play FPS or RTS games too.

2. Tell us about your introduction to Gentoo?

The first look at Gentoo was when I was a dedicated enthusiast Windows user, who would run as much on Windows as possible. Once I've tried to set up a Windows / Linux combination by running SUA / Interix together with Xming, but as I barely knew Linux back then that didn't come to a good end. Later, Linux was needed for university; as we needed to guarantee our software compiles and works on the lab computers that run Linux.

Having used another distribution in a virtual machine for some time; I discovered that it was slow without hardware virtualization, which we didn't have yet back then. Something fast and small on a separate partition was needed; and thus, a small bit of space was cleaned out at the end of the partition and Gentoo was used to create a quite minimal setup with just what's necessary to develop, compile and test.

When the need for that was over, the small partition was ditched; thus I have been using Windows for several years, but with Windows 8 going RTM and the changes that happened I started to realize that I wanted an OS that can be changed to what I like, instead of doing things the way in the limited amount of ways they can be done.

So, Gentoo Linux came back in mind; and that's how I made the switch to it last year.

3. Describe your journey to become a Gentoo developer?

Not long after becoming an user of Gentoo, I decided to contribute back; so, I started to try to package some things that I used on Windows or which fitted the current needs back then. From there on I looked for ways to contribute, at which time I found a blog post that the kernel team is looking for users to help; there was too many users, so, I didn't make the cut.

Apparently, none of them sticked to it; so, later I got back to try again and then the kernel lead mentored me. As this was a good opportunity, the next days were spent on studying the development manual and answering the quizzes as detailed as possible; I took a self-study approach here, looking back on it having seen every part of the devmanual certainly gains you a lot, as you can recall where things are and know enough to not break the Portage tree.

After a recruiter reviewed the quiz responses a year ago; I learned more during the review, that's how I became Gentoo Developer and six months after I switched from Windows.

4. What are some of the projects you are involved with and the packages you help maintain?

Besides working on our Kernel releases, recently I have joined the QA and Portage team to keep the quality of our distribution high and improve repoman; in the longer end I plan to improve Portage and/or pkgcore when I get to learn their code base better. Other teams I am on are the Proxy Maintainers (to help Gentoo users maintain packages without them needing to become a Gentoo Developer); as well as the Java, Dotnet, Bug Wranglers and Bug Cleaners projects. The last two projects help get bugs assigned and cleaned up.

Next to those projects I maintain or help maintain some packages that I either personally use, am interested in or where work was needed. One of the last introduced packages is Epoch, a new minimal init system. It boots extremely fast on the Raspberry Pi.

5. I proxy-maintain a few packages myself. I am a staff member without commit rights. Its a great way to give back and also help maintain a package that you like and use. To prepare I did the ebuild quiz for my own understanding of ebuild writing and set up a local overlay to test my ebuilds. What are some other ways a user can become confident enough to maintain some packages?

The basic guide to write Gentoo Ebuilds is a guide that was started to cover the very first steps to writing an ebuild; this resource was previously non existing, it was written to close the gap between having no prior knowledge and the Gentoo Development Guide.

The Gentoo Development Guide is a great reference to find most details and policy one needs to know for writing ebuilds; when working in the terminal, checking out man 5 ebuild can be handy to quickly look up syntax, variables and functions of the ebuild format.

Creating a local overlay allows you to start locally experimenting with ebuilds. When you feel confident you can request a hosted overlay (or create one yourself on a third party service like GitHub and file a similar bug requesting it to be added to the overlay list) or contribute to the Portage tree (through proxy maintenance or you can become developer if you want to) or an existing overlay.

When you do proxy maintenance, the proxy maintainers will help you by advising and reviewing the ebuild and letting you know how to improve it; if you work on an overlay, there are other mediums (where proxy maintainers are present as well) to ask questions or get your ebuild reviewed. For example, #gentoo-dev-help on the Freenode IRC network is helpful.

Besides that users are advised to run

repoman manifest && repoman full

to check for QA errors, QA keywords are explained in the last part of man repoman it can help find common mistakes, as well as help increase the quality for it to be added to the Portage tree.

6. What do you think Gentoo's strengths and weaknesses are both as a development platform and as a general purpose Linux Distribution?

That you can very easily patch up packages is a very nice feature, as well as the code that gets compiled by those packages; you can simply unpack the code;

ebuild unpack foo-1.ebuild

and write a patch for one or more file(s), then put the patch in /etc/portage/patches/app-bar/foo and there you have your patched code.

Besides patching up packages, the USE flag control in Gentoo is what makes Gentoo powerful. This controls the features of packages to allow you to have packages fit your usage rather than become bloated with features, libraries and other size hogs you never need. Alongside USE flag control becomes the ability to choose alternative libraries, alternative GUIs more; which is nice when you prefer the way something works or looks like.

What I think Gentoo could use more is more manpower; what made Gentoo powerful is its community, and its community is formed by users who contribute. And to this extent the amount of contributions determine how powerful Gentoo becomes.

If users are interested; they are welcome to contribute to Gentoo, to make it even more powerful than ever before. They don't necessarily need much prior knowledge, there's something for everybody; and if needed, we can help them learn more.

7. Can you describe your personal desktop setup (WM/DE)?

As desktop environment; I use GNOME 3, I'm glad to see the way they have progressed in terms of their user interface. GNOME 2 I've also used in the past, because I didn't bother searching further too much; but didn't really like GNOME 2's UI. GNOME 3's UI gets out of the way and I like how it focuses on the more typical user that has no special requirements.

Alongside that comes the requirement to run systemd; though that was in use long before running GNOME 3, as a while ago I was on XFCE and was experimenting around to see if systemd fits certain needs. It does; so does XFCE as well, so while I don't really like it UI like with GNOME 2, I considered XFCE as an alternative DE to switch to. However, very recently I'm using MATE on top of GNOME 3; if GNOME 3 breaks, MATE is my new alternative DE.

The particular thing that I like about systemd is that it allows you to easily make a huge cut in boot time; while this kind of parameter has no good purpose in general, it does help as I need to test kernel releases and sometimes switch between NVIDIA and Nouveau module. The boot is down to two seconds after the boot loader hands over; at this point, you discover that the bootchart PNG export feature doesn't allow you to scale the graph…

On the Raspberry Pi, Epoch gets the boot time down to seconds; as it was bothering that it previously took over a minute, as that is what running init scripts (which are shell) does together with all what they call when you run it on slow embedded hardware. Whereas Epoch is a daemon with a single configuration file that just starts a few processes and that's it.

It also helped for bisecting as well as hacking up a reclocking patch for the Nouveau module a bit; while making it work on the NVIDIA card, the patch is still unstable and might break other cards and further improving it is quite a steep learning curve and a lot of work.

Other software that I use is AutoKey to quickly paste text that I need to repeat often (comments on bugs, e-mail responses, …); Chromium which I think is a browser that gets out of the way with its minimal interface; WeeChat (actively developed irssi clone with a ton of extra features); a mail client that does what I need (Claws Mail); and I could go on for hours, so, feel free to ask if you want to know more…

8. What are the specs of your current boxes?

Currently I own a Clevo W870CU barebone laptop that is put together; it features a Intel Corporation 5 Series/3400 Series Chipset, a Full HD 17 inch screen and enough interface ports. The processor in it is an Intel(R) Core(TM) i7 CPU Q 720. As hard disks I use a Intel X25-M 160 GB SSD and a Seagate Momentus 7200.3 320 GB HDD. There are also a NVIDIA GeForce GTX 285M, Intel Corporation WiFi Link 5100 and Realtek RTL8111/8168/8411 PCIE Gigabit Ethernet Controller inside.

As for the Raspberry Pi, it is a model B; you can find its specifications here. I gave it a 32 GB SD card with Gentoo on it where the 32 GB gives it some room before wearing it out. Alongside there are two external drives of a few terabytes to store big data and backups.

The Raspberry Pi here kind of acts like a cheap all-in-one NAS and/or media solution.

9. As a Gentoo Developer what are some of your accomplishments?

On the kernel team, the kernel eclass and genpatches scripts were adapted to bring support for experimental patches; this allows adding experimental patches to kernel packages using USE=experimental, without applying them by default. A condition for an experimental patch to be added is that applying the patch does not change the runtime behavior; in other words, we want changes to be guarded by a config option, in addition to USE=experimental. The eventual end goal is to have a lot of the regular experimental patches supported, to deduplicate work amongst kernel packages and our users.

Besides making improvements to the kernel packaging I maintain packages that I use and/or packages that need maintenance; at the moment, MATE is being brought to the Portage tree. Quality Assurance work I also do to keep the quality of the Portage tree high.

10. What would be your dream job?

While not having anything specific in mind, developing on "something" is what I have in mind.

In the context of the business world, that could be solutions that aid users with their daily tasks; in the context of the gaming world, maybe some indie game in the hope that it grows out; and last, I listen to music a lot, so, maybe within that context it could be some kind of computer science solution within that field.

Relying on yet-to-discover science is what I'd like to avoid, and rather rely on what is a given already; such that becoming popular is the only real risk. Once popularity has been obtained, then exploration might become an option; although one should not ignore that exploration can lead to popularity, but as said that is not without risk.

11. What users would you like to recruit to become Gentoo Developers?

Good question; many people are qualified, anyone that's interested can expect help from us.

12. What gives you the most enjoyment within the Gentoo community?

Giving back to the community as an appreciation of what the community has given to me.

Gentoo Galaxy: Keeping History of Gentoo

(by Seemant Kulleen)

Gentoo Galaxy aims to make sure that Gentoo's history is as accurate as possible, that every Gentoo developer's contribution is acknowledged and valued. We're starting with our list of Gentoo developers. We currently have all developers who have been active in Bugzilla and/or the 4 main CVS repositories throughout Gentoo's history represented in a visualization here: http://kulleen.org/gentoo/galaxy

That page contains a list of developers for whom we need more information - we want to visualize everybody's contributions. If you are or know a developer on that list, please get in touch with us via bugzilla. e-mail, twitter, google plus or IRC in #gentoo or #gentoo-dev.

Trustee News

Gentoo Foundation 2013 Treasure Summary
In the fiscal year 2013, for the period of July 1st through June 30th we had total assets of $73,494.40. Our main income was $7,000.00 from GSOC, next was donations thru paypal for $6,386.94 and the official Gentoo store generated $558.85 in commissions.

Our expenses totaled $3,396.01 with $2,399.23 to Gentoo GSoC 2012 mentor's summit travel reimbursement.

Our expenses are kept to a minimium because of all our generous sponsors plus the work of our Infrastructure team to secure donations of hosting, hardware and bandwidth.

Requests for Funds, Project Support, or Equipment
Requests for funds, project support, or equipment need to be sent to the Foundation in the form of a proposal. This proposal is to inform all trustees of the need (not all of them will be aware of the need or the background of the situation). The proposal process will also help to maintain a trusting relationship between the Foundation and its donors. Donors know and expect that without exception money will only be spent after a proposal and vote by the Board of Trustees. Additionally, the proposals will be archived to provide accountability for money spent.

Please review our policy documentation for more information.

News Items

Subject: Ruby 1.8 removal, Ruby 1.9 and Ruby 2.0 activated by default

The Gentoo Ruby team would like to inform you, that the default active ruby targets changed from "ruby19 ruby18″ to "ruby19 ruby20″.

It is about time, because Ruby 1.8 was retired by upstream in July 2013 [1] and has got known security issues (CVE-2013-4164). In Gentoo, we're going to remove the currently package.masked Ruby MRI 1.8 soon. All packages, depending on ruby, have been converted to support at least Ruby 1.9 or were added to the package.mask at the same time with Ruby 1.8. In case of issues during or after the upgrade, feel free to fill a bug at bugs.gentoo.org

If your currently eselected Ruby interpreter is ruby18, our recommendation is to change it to ruby19. [2] At the moment Ruby MRI 1.9 delivers the best possible support of all Ruby interpreters in tree.

Check the current setting via:
eselect ruby show

Change the current setting to Ruby MRI 1.9 via:
eselect ruby set ruby19

[1] https://www.ruby-lang.org/en/news/2013/06/30/we-retire-1-8-7/
[2] https://wiki.gentoo.org/wiki/Project:Ruby/Ruby_1.9_migration

Gentoo Developer Stats

Summary

Gentoo is made up of 252 active developers, of which 38 are currently away.
Gentoo has recruited a total of 794 developers since its inception.

Changes

The following developers have recently changed roles:
Jason A. Donenfeld (zx2c4) Joined the systemd project

Additions

The following developers have recently joined the project:
None this month

Moves

The following developers recently left the Gentoo project:
None this month

Portage

This section summarizes the current state of the portage tree.

Architectures 45
Categories 161
Packages 17342
Ebuilds 36489
Architecture Stable Testing Total % of Packages
alpha 3612 510 4122 23.77%
amd64 10703 6142 16845 97.13%
amd64-fbsd 0 1577 1577 9.09%
arm 2631 1636 4267 24.61%
hppa 3034 484 3518 20.29%
ia64 3186 575 3761 21.69%
m68k 576 88 664 3.83%
mips 4 2362 2366 13.64%
ppc 6865 2349 9214 53.13%
ppc64 4334 849 5183 29.89%
s390 1493 290 1783 10.28%
sh 1714 339 2053 11.84%
sparc 4135 877 5012 28.90%
sparc-fbsd 0 323 323 1.86%
x86 11418 5183 16601 95.73%
x86-fbsd 0 3233 3233 18.64%

gmn-portage-stats-2013-11

Security

The following GLSAs have been released by the Security Team

GLSA Package Description Bug
201403-08 dev-perl/PlRPC PlRPC: Arbitrary code execution 497692
201403-07 sys-apps/grep grep: User-assisted execution of arbitrary code 448246
201403-06 net-libs/libupnp libupnp: Arbitrary code execution 454570
201403-05 app-editors/emacs GNU Emacs: Multiple vulnerabilities 398239
201403-04 dev-qt/qtcore QtCore: Denial of Service 494728
201403-03 sys-apps/file file: Denial of Service 501574
201403-02 dev-libs/libyaml LibYAML: Arbitrary code execution 499920
201403-01 www-client/chromium Chromium-V8: Multiple vulnerabilities 486742

Package Removals/Additions

Removals

Package Developer Date
x11-misc/slimlock titanofold 10 Mar 2014
dev-libs/ido ssuominen 15 Mar 2014
dev-ruby/ruby-bdb mrueg 15 Mar 2014
www-servers/mongrel_cluster mrueg 15 Mar 2014
virtual/emacs-cedet ulm 17 Mar 2014
gnustep-libs/cddb voyageur 17 Mar 2014
app-emacs/nxml-mode ulm 17 Mar 2014
app-emacs/erc ulm 17 Mar 2014
app-emacs/cperl-mode ulm 17 Mar 2014
app-emacs/alt-font-menu ulm 17 Mar 2014
app-emacs/u-vm-color ulm 17 Mar 2014
app-emacs/eperiodic ulm 20 Mar 2014
app-emacs/view-process ulm 20 Mar 2014
media-sound/audio-entropyd angelos 22 Mar 2014
app-emacs/http-emacs ulm 23 Mar 2014
app-emacs/mairix ulm 23 Mar 2014

Additions

Package Developer Date
dev-python/pretend radhermit 01 Mar 2014
dev-python/cryptography radhermit 01 Mar 2014
dev-java/boilerpipe ercpe 01 Mar 2014
media-plugins/gst-plugins-vaapi pacho 01 Mar 2014
dev-db/derby ercpe 01 Mar 2014
net-analyzer/masscan robbat2 01 Mar 2014
mate-base/mate-desktop tomwij 02 Mar 2014
mate-extra/mate-dialogs tomwij 02 Mar 2014
mate-extra/mate-polkit tomwij 02 Mar 2014
x11-libs/libmatewnck tomwij 02 Mar 2014
dev-python/ssl-fetch dolsen 02 Mar 2014
dev-java/hamcrest-integration ercpe 02 Mar 2014
sci-libs/Fiona slis 03 Mar 2014
dev-python/ipdbplugin slis 03 Mar 2014
sci-libs/pyshp slis 03 Mar 2014
dev-util/lttng-modules dlan 04 Mar 2014
dev-util/lttng-ust dlan 04 Mar 2014
dev-util/lttng-tools dlan 04 Mar 2014
dev-util/babeltrace dlan 04 Mar 2014
games-misc/papers-please hasufell 04 Mar 2014
dev-haskell/scientific qnikst 04 Mar 2014
dev-haskell/text-stream-decode qnikst 04 Mar 2014
kde-base/kwalletmanager johu 04 Mar 2014
mate-base/mate-panel tomwij 05 Mar 2014
mate-base/mate-settings-daemon tomwij 05 Mar 2014
net-wireless/crackle zerochaos 05 Mar 2014
dev-util/appdata-tools polynomial-c 06 Mar 2014
media-libs/libepoxy mattst88 06 Mar 2014
dev-ruby/magic mrueg 06 Mar 2014
net-wireless/mate-bluetooth tomwij 07 Mar 2014
x11-themes/mate-icon-theme tomwij 07 Mar 2014
x11-wm/mate-window-manager tomwij 07 Mar 2014
dev-ruby/ruby-feedparser mrueg 07 Mar 2014
dev-java/dnsjava ercpe 07 Mar 2014
dev-haskell/abstract-deque-tests gienah 09 Mar 2014
dev-haskell/exceptions gienah 09 Mar 2014
dev-haskell/errorcall-eq-instance gienah 09 Mar 2014
dev-haskell/asn1-encoding gienah 09 Mar 2014
dev-haskell/asn1-parse gienah 09 Mar 2014
dev-haskell/chunked-data gienah 09 Mar 2014
dev-haskell/enclosed-exceptions gienah 09 Mar 2014
dev-haskell/esqueleto gienah 09 Mar 2014
dev-haskell/foldl gienah 09 Mar 2014
dev-haskell/x509 gienah 09 Mar 2014
dev-haskell/x509-store gienah 09 Mar 2014
dev-haskell/x509-system gienah 09 Mar 2014
dev-haskell/x509-validation gienah 09 Mar 2014
mate-base/mate-file-manager tomwij 09 Mar 2014
mate-extra/mate-calc tomwij 09 Mar 2014
mate-extra/mate-character-map tomwij 09 Mar 2014
mate-extra/mate-power-manager tomwij 09 Mar 2014
mate-extra/mate-screensaver tomwij 10 Mar 2014
mate-extra/mate-sensors-applet tomwij 10 Mar 2014
dev-python/ansicolor jlec 10 Mar 2014
dev-libs/liblogging ultrabug 10 Mar 2014
sys-apps/gentoo-functions williamh 10 Mar 2014
mate-extra/mate-system-monitor tomwij 10 Mar 2014
mate-extra/mate-utils tomwij 11 Mar 2014
x11-terms/mate-terminal tomwij 11 Mar 2014
x11-themes/mate-backgrounds tomwij 11 Mar 2014
x11-themes/mate-themes tomwij 11 Mar 2014
media-video/atomicparsley-wez ssuominen 11 Mar 2014
app-arch/mate-file-archiver tomwij 12 Mar 2014
app-editors/mate-text-editor tomwij 12 Mar 2014
app-text/mate-document-viewer tomwij 12 Mar 2014
games-misc/games-envd hasufell 12 Mar 2014
perl-core/Dumpvalue zlogene 12 Mar 2014
dev-python/python-caja tomwij 12 Mar 2014
dev-haskell/fingertree qnikst 12 Mar 2014
dev-haskell/reducers qnikst 12 Mar 2014
dev-haskell/monadrandom qnikst 12 Mar 2014
dev-haskell/either qnikst 12 Mar 2014
media-libs/x265 aballier 12 Mar 2014
dev-haskell/tasty-rerun qnikst 12 Mar 2014
dev-haskell/ekg qnikst 12 Mar 2014
dev-lang/lfe patrick 13 Mar 2014
dev-ml/optcomp aballier 13 Mar 2014
dev-ml/deriving aballier 13 Mar 2014
dev-python/venusian patrick 14 Mar 2014
dev-python/pyramid patrick 14 Mar 2014
kde-misc/about-distro johu 14 Mar 2014
dev-haskell/errors qnikst 14 Mar 2014
perl-core/Math-Complex zlogene 14 Mar 2014
dev-libs/ido ssuominen 15 Mar 2014
dev-python/dugong radhermit 17 Mar 2014
mate-base/mate-applets tomwij 17 Mar 2014
mate-extra/caja-dropbox tomwij 17 Mar 2014
mate-extra/mate-file-manager-image-converter tomwij 17 Mar 2014
mate-extra/mate-file-manager-open-terminal tomwij 17 Mar 2014
mate-extra/mate-file-manager-sendto tomwij 17 Mar 2014
mate-extra/mate-file-manager-share tomwij 17 Mar 2014
dev-util/emilpro zerochaos 18 Mar 2014
kde-misc/kcmsystemd johu 18 Mar 2014
media-gfx/mate-image-viewer tomwij 19 Mar 2014
x11-misc/mate-menu-editor tomwij 19 Mar 2014
net-analyzer/mate-netspeed tomwij 19 Mar 2014
x11-misc/mate-notification-daemon tomwij 19 Mar 2014
x11-themes/mate-icon-theme-faenza tomwij 19 Mar 2014
dev-ruby/rb-readline zerochaos 19 Mar 2014
dev-vcs/hg-fast-export ottxor 21 Mar 2014
sys-apps/audio-entropyd angelos 22 Mar 2014
dev-vcs/git-flow johu 22 Mar 2014
app-emacs/gnuplot-mode ulm 22 Mar 2014
app-admin/mate-system-tools tomwij 22 Mar 2014
mate-extra/mate-media tomwij 22 Mar 2014
mate-base/mate-control-center tomwij 22 Mar 2014
net-misc/portspoof zerochaos 22 Mar 2014
app-leechcraft/lc-ooronee maksbotan 23 Mar 2014
app-leechcraft/lc-cpuload maksbotan 23 Mar 2014
app-leechcraft/lc-certmgr maksbotan 23 Mar 2014
mate-extra/mate-user-share tomwij 23 Mar 2014

Bugzilla

The Gentoo community uses Bugzilla to record and track bugs, notifications, suggestions and other interactions with the development team.

Activity

The following tables and charts summarize the activity on Bugzilla between 25 February 2014 and 27 March 2014. Not fixed means bugs that were resolved as NEEDINFO, WONTFIX, CANTFIX, INVALID or UPSTREAM.
gmn-activity-2014-02

Bug Activity Number
New 1820
Closed 1307
Not fixed 177
Duplicates 159
Total 5600
Blocker 4
Critical 19
Major 65

Closed bug ranking

The developers and teams who have closed the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Python Gentoo Team 76
2 Perl Devs @ Gentoo 63
3 Gentoo KDE team 47
4 Gentoo Security 41
5 Gentoo's Team for Core System packages 41
6 Gentoo's Haskell Language team 35
7 Gentoo Linux Gnome Desktop Team 31
8 GNU Emacs Team 29
9 Default Assignee for Orphaned Packages 28
10 Others 915


gmn-activity-2014-02

Assigned bug ranking

The developers and teams who have been assigned the most bugs during this period are as follows.

Rank Team/Developer Bug Count
1 Gentoo Linux bug wranglers 119
2 Gentoo Security 95
3 Gentoo Games 75
4 Gentoo KDE team 57
5 Gentoo Linux Gnome Desktop Team 57
6 Python Gentoo Team 52
7 Gentoo's Team for Core System packages 51
8 Gentoo's Haskell Language team 41
9 GNU Emacs Team 41
10 Others 1231


gmn-activity-2014-02

Tip of the month

Gentoolkit has a little known utility called enalyze.

Enalyze analyzes the deployment information Gentoo keeps of all packages and checks this against the current settings status.

There are 2 sub-modules:
- the "analyze" module produces the reports, and
- the "rebuild" module which allows for rebuilding package.use, package.accept_keywords, and package.unmask files which can be placed in /etc/portage.

The difference between it and equery, is that equery does specific queries, while enalyze does complete reports. So, essentially it can be used as a tune up or repair kit for your gentoo system. It does not do everything for you, it does leave some of the decision making to you. But after reviewing the reports, you may want to edit your make.conf to optimize its settings. An interesting feature is that enalyze supports creation of new package.use, package.accept_keywords or package.unmask files based on the currently installed package information, your current profile and make.conf settings. Through it, enalyze can help you rebuild these files or remove obsolete entries from it.

Please note that it does not use or modify existing /etc/portage/package.* files

eg:

# enalyze analyze -v use

This produces a report of all use flags used by packages on your system as well as how they are used. It shows if a USE flag is enabled or disabled, and shows if the USE flag has a "default" setting (a summary of: a profile enabled USE flag, a global make.defaults USE flag, etc.) For each USE flag, the packages that use it are listed as well when called with the -v module option.

From that information you can edit your make.conf's USE= and remove any flags that are already defaulted. if there is a flag that has more than a few packages using that setting, you could add it to the USE= instead of relying on having that flag in package.use for those packages.

When finished the above:

# enalyze rebuild use

Will generate a new package.use file (neatly sorted) of only the entries needed to preserve the current state of the packages installed. Once you check over the file, add some custom tweaks (to your satisfaction) you can replace the existing or missing file in /etc/portage.

It also runs completely as any user in the portage group. There is no need to run it with superuser rights. Any files generated are saved in the users home directory.

Tip: It is very useful for changing profiles too. Just run them to adapt to the new profile and the new defaults.

P.S. There is room for the utility to get many more reports and rebuild options. So, submit your requests (and hopefully code).

Send us your favorite Gentoo script or tip at gmn@gentoo.org

Getting Involved?

Interested in helping out? The GMN relies on volunteers and members of the community for content every month. If you are interested in writing for the GMN or thinking of another way to contribute, please send an e-mail to gmn@gentoo.org.

31 Mar 2014 7:07pm GMT

Sven Vermeulen: Proof of concept for USE enabled policies

tl;dr: Some (-9999) policy ebuilds now have USE support for building in (or leaving out) SELinux policy statements.

One of the "problems" I have been facing since I took on the maintenance of SELinux policies within Gentoo Hardened is the (seeming) inability to make a "least privilege" policy that suits the flexibility that Gentoo offers. As a quick recap: SELinux policies describe the "acceptable behavior" of an application (well, domain to be exact), often known as the "normalized behavior" in the security world. When an application (which runs within a SELinux domain) wants to perform some action which is not part of the policy, then this action is denied.

Some applications can have very broad acceptable behavior. A web server for instance might need to connect to a database, but that is not the case if the web server only serves static information, or dynamic information that doesn't need a database. To support this, SELinux has booleans through which optional policy statements can be enabled or disabled. So far so good.

Let's look at a second example: ALSA. When ALSA enabled applications want to access the sound devices, they use IPC resources to "collaborate" around the sound subsystem (semaphores and shared memory to be exact). Semaphores inherit the type of the domain that first created the semaphore (so if mplayer creates it, then the semaphore has the mplayer_t context) whereas shared memory usually gets the tmpfs-related type (mplayer_tmpfs_t). When a second application wants to access the sound device as well, it needs access to the semaphore and shared memory. Assuming this second application is the browser, then mozilla_t needs access to semaphores by mplayer_t. And the same for chromium_t. Or java_t applications that are ALSA-enabled. And alsa_t. And all other applications that are ALSA enabled.

In Gentoo, ALSA support can be made optional through USE="alsa". If a user decides not to use ALSA, then it doesn't make sense to allow all those domains access to each others' semaphores and shared memory. And although SELinux booleans can help, this would mean that for each application domain, something like the following policy would need to be, optionally, allowed:

# For the mplayer_t domain:
optional_policy(`
  tunable_policy(`use_alsa',`
    mozilla_rw_semaphores(mplayer_t)
    mozilla_rw_shm(mplayer_t)
    mozilla_tmpfs_rw_files(mplayer_t)
  ')
')

optional_policy(`
  tunable_policy(`use_alsa',`
    chromium_rw_semaphores(mplayer_t)
    chromium_rw_shm(mplayer_t)
    chromium_tmpfs_rw_files(mplayer_t)
  ')
')

And this for all domains that are ALSA-enabled. Every time a new application is added that knows ALSA, the same code needs to be added to all policies. And this only uses a single SELinux boolean (whereas Gentoo supports USE="alsa" on per-package level), although we can create separate booleans for each domain if we want to. Not that that will make it more manageable.

One way of dealing with this would be to use attributes. Say we have a policy like so:

attribute alsadomain;
attribute alsatmpfsfile;

allow alsadomain alsadomain:sem rw_sem_perms;
allow alsadomain alsadomain:shm rw_shm_perms;
allow alsadomain alsatmpfsfile:file rw_file_perms;

By assigning the attribute to the proper domains whenever ALSA support is needed, we can toggle this more easily:

# In alsa.if
interface(`alsa_domain',`
  gen_require(`
    attribute alsadomain;
    attribute alsatmpfsfile;
  ')
  typeattribute $1 alsadomain;
  typeattribute $2 alsatmpfsfile;
')


# In mplayer.te
optional_policy(`
  tunable_policy(`use_alsa',`
    alsa_domain(mplayer_t, mplayer_tmpfs_t)
  ')
')

That would solve the problem of needlessly adding more calls in a policy for every ALSA application. And hey, we can probably live with either a global boolean (use_alsa) or per-domain one (mplayer_use_alsa) and toggle this according to our needs.

Sadly, the above is not possible: one cannot define typeattribute assignments inside a tunable_policy code: attributes are part of the non-conditional part of a SELinux policy. The solution would be to create build-time conditionals (rather than run-time):

ifdef(`use_alsa',`
  optional_policy(`
    alsa_domain(mplayer_t, mplayer_tmpfs_t)
  ')
')

This does mean that use_alsa has to be known when the policy is built. For Gentoo, that's not that bad, as policies are part of separate packages, like sec-policy/selinux-mplayer. So what I now added was USE-enabled build-time decisions that trigger this code. The selinux-mplayer package has IUSE="alsa" which will enable, if set, the use_alsa build-time conditional.

As a result, we now support a better, fine-grained privilege setting inside the SELinux policy which is triggered through the proper USE flags.

Is this a perfect solution? No, but it is manageable and known to Gentoo users. It isn't perfect, because it listens to the USE flag setting for the selinux-mplayer package (and of course globally set USE flags) but doesn't "detect" that the firefox application (for which the policy is meant) is or isn't built with USE="alsa". So users/administrators will need to keep this in mind when using package-local USE flag definitions.

Also, this will make it a bit more troublesome for myself to manage the SELinux policy for Gentoo (as upstream will not use this setup, and as such patches from upstream might need a few manual corrections before they apply to our tree). However, I gladly take that up if it means my system will have somewhat better confinement.

31 Mar 2014 4:33pm GMT

29 Mar 2014

feedPlanet Gentoo

Robin Johnson: Mail Bounces & Gmail/GApps users: The ugly truth of DMARC in open-source mailing lists

This is a slightly edited copy of an email I send to the mailing lists for my local hackspace, VHS. I run their mailing lists presently for historical reasons, but we're working on migrating them slowly.


Hi all,

Speaking as your email list administrator here. I've tried to keep the logs below as intact as possible, I've censored only one user's domain as being identifying information explicitly, and then two other recipient addresses.

There have been a lot of reports lately of bounce notices from the list, and users have correctly contacted me, wondering what's going on. The bounce messages are seen primarily by users on Gmail and hosted Google Apps, but the problems do ultimately affect everybody.

67.6% of the vhs-general list uses either gmail or google apps (347 subs of 513). For the vhs-members list it's 68.3% (both of these stats created by checking if the MX record for the user's domain points to Google).

Google deciding that a certain list message is too much like spam, because of two things:

Content:

We CAN do something about the content.

Please don't send email that has one or twos, containing a URL and a short line of text. It's really suspicious and spam-like.

Include a better description (two or three lines) with the URL.

This gets an entry in the mailserver logs like:

delivery 47198: failure:
+173.194.79.26_failed_after_I_sent_the_message./Remote_host_said:_550-5.7.1_[66.196.40.251______12]_Our_system_has_detected_that_this_message_is/550-5.7.1_likely_unsolicited_mail._To_reduce_the_amount_of_spam_sent_to_Gmail,/550-5.7.1_this_message_has_been_blocked._Please_visit/550-5.7.1_http://support.google.com/m
+ail/bin/answer.py?hl=en&answer=188131_for/550_5.7.1_more_information._mu18si1139639pab.287_-_gsmtp/

That was triggered by this email earlier in the month:

> Subject: Kano OS for RasPi
> http://kano.me/downloads
> Apparently it's faster than Rasbian

DMARC policy:

TL;DR: If you work on an open-source mailing list app, please implement DMARC support ASAP!

Google and other big mail hosters have been working on an anti-spam measure called DMARC [1].

Unlike many prior attempts, it latches onto the From header as well as the SMTP envelope sender, and this unfortunately interferes with mailing lists [2], [3].

I do applaud the concept behind DMARC, but the rollout seems to be hurting lots of the small guys.

At least person (Eric Sachs) at Google is aware of this [4]. There is no useful workaround that I can enact as a list admin right now, other than asking the one present user to tweak his mailserver if possible.

There is also no completed open source support I can find for DMARC. Per the Google post above, the Mailman project is working on it [5], [6], but it's not yet available as of the last release. Our lists run on ezmlm-idx, and I run some other very large lists using mlmmj (gentoo.org) and sympa; none of them have DMARC support.

The problem is only triggering with a few conditions so far:

Of the 115 unique domains used by subscribers on this list, here are all the DMARC policies:

_dmarc.gmail.com.       600  IN TXT "v=DMARC1\; p=none\; rua=mailto:mailauth-reports@google.com"
_dmarc.USERDOMAIN.ca.   7200 IN TXT "v=DMARC1\; p=reject\; rua=mailto:azrxfkte@ag.dmarcian.com\; ruf=mailto:azrxfkte@fr.dmarcian.com\; adkim=s\; aspf=s"
_dmarc.icloud.com.      3600 IN TXT "v=DMARC1\; p=none\; rua=mailto:dmarc_agg@auth.returnpath.net, mailto:d@rua.agari.com\; ruf=mailto:d@ruf.agari.com, mailto:dmarc_afrf@auth.returnpath.net\;rf=afrf\;pct=100"
_dmarc.mac.com.         3600 IN TXT "v=DMARC1\; p=none\; rua=mailto:d@rua.agari.com\; ruf=mailto:d@ruf.agari.com\;"
_dmarc.me.com.          3600 IN TXT "v=DMARC1\; p=none\; rua=mailto:d@rua.agari.com\; ruf=mailto:d@ruf.agari.com\;"
_dmarc.yahoo.ca.        7200 IN TXT "v=DMARC1\; p=none\; pct=100\; rua=mailto:dmarc-yahoo-rua@yahoo-inc.com\;"
_dmarc.yahoo.com.       1800 IN TXT "v=DMARC1\; p=none\; pct=100\; rua=mailto:dmarc-yahoo-rua@yahoo-inc.com\;"
_dmarc.yahoo.co.uk.     1800 IN TXT "v=DMARC1\; p=none\; pct=100\; rua=mailto:dmarc-yahoo-rua@yahoo-inc.com\;"

Only one of those includes a reject policy, but I suspect it's a matter of time until more of them will include it. I'm going to use USERDOMAIN.ca here as the rest of the example, and that user is indirectly responsible for lots of the rejects we are seeing.

Step 1.

User sends this email.

From: A User <someuser@userdomain.ca>
To: vhs-general@lists.hackspace.ca

Delivered to list server via SMTP (these two addresses form the SMTP envelope)

MAIL FROM:<someuser@userdomain.ca>
RCPT TO:<vhs-general@lists.hackspace.ca>

Step 2.

If the MAIL-FROM envelope address is on the list of list subscribers, your message is accepted.

Step 3.0.

The list adjusts the mail to outgoing, and uses SMTP VERP [7] to get the mail server to send the new message. This means it hands off a single copy of the email, as well as a list of all recipients for the mail. Envelope from address in this case will encode the name of the list and the number of the mail in the archive.

If it was delivering to me (robbat2@orbis-terrarum.net), the outgoing SMTP connection would look roughly like:

MAIL FROM:<vhs-general-return-18094-robbat2=orbis-terrarum.net@lists.hackspace.ca>
RCPT TO:<robbat2@orbis-terrarum.net>

And the mail itself still looks like:

From: A User <someuser@userdomain.ca>
To: vhs-general@lists.hackspace.ca

Step 3.1.

I got this email, and if I open it I see this telling me about the SMTP details:

Return-Path: <vhs-general-return-18094-robbat2=orbis-terrarum.net@lists.hackspace.ca>

I don't implement DMARC on my domain. If my system bounced the email, it would have gone to that address, and the list app would know that message 18094 on list vhs-general bounced to user robbat2@orbis-terrarum.net.

Step 3.2.

Google DOES implement DMARC, so lets run through that.

The key part of DMARC is that it takes the domain from the From header.

_dmarc.USERDOMAIN.ca.   7200 IN TXT "v=DMARC1\; p=reject\; rua=mailto:azrxfkte@ag.dmarcian.com\; ruf=mailto:azrxfkte@fr.dmarcian.com\; adkim=s\; aspf=s"

The relevant parts to us are:

p=reject, aspf=s

The ASPF section applies strict mode, and says the mail with a From header of someuser@USERDOMAIN.ca, must have an exact match of the MAIL FROM transaction of @USERDOMAIN.ca.

It doesn't match, as the list changed the MAIL FROM address. The p=reject says to reject the mail if this happens.

This runs counter to the design principles of mailing lists, so DMARC has a bunch of options, all of which require changing the mail in some way.

Here's the logs from the above failure:

> 2014-03-19 11:19:50.783996500 new msg 98907
> 2014-03-19 11:19:50.783998500 info msg 98907: bytes 8864 from <vhs-general-return-18094-@lists.hackspace.ca-@[]> qp 32511 uid 89
> 2014-03-19 11:19:50.785359500 starting delivery 211352: msg 98907 to remote user1@gappsdomain.com
> 2014-03-19 11:19:50.785385500 status: local 1/10 remote 1/40
> 2014-03-19 11:19:50.785450500 starting delivery 211353: msg 98907 to remote user2@gmail.com
> ...
> 2014-03-19 11:19:58.713558500 delivery 211352: failure:
+74.125.25.27_failed_after_I_sent_the_message./Remote_host_said:_550-5.7.1_Unauthenticated_email_from_USERDOMAIN.ca_is_not_accepted_due_to_domain's/550-5.7.1_DMARC_policy._Please_contact_administrator_of_USERDOMAIN.ca_domain_if/550-5.7.1_this_was_a_legitimate_mail._Please_visit/550-5.7.1__http://support.google.com
+/mail/answer/2451690_to_learn_about_DMARC/550_5.7.1_initiative._ub8si9386628pac.133_-_gsmtp/
> 2014-03-19 11:19:59.053816500 delivery 211353: failure:
+173.194.79.26_failed_after_I_sent_the_message./Remote_host_said:_550-5.7.1_Unauthenticated_email_from_USERDOMAIN.ca_is_not_accepted_due_to_domain's/550-5.7.1_DMARC_policy._Please_contact_administrator_of_USERDOMAIN.ca_domain_if/550-5.7.1_this_was_a_legitimate_mail._Please_visit/550-5.7.1__http://support.google.co
+m/mail/answer/2451690_to_learn_about_DMARC/550_5.7.1_initiative._my2si9389106pab.76_-_gsmtp/

[1] http://dmarc.org/
[2] http://dmarc.org/faq.html#s_3
[3] http://dmarc.org/faq.html#r_2
[4] https://sites.google.com/site/oauthgoog/mlistsdkim
[5] http://www.marshut.com/qskkv/adding-dmarc-support-for-mailman-3.html
[6] https://code.launchpad.net/~jimpop/mailman/dmarc-reject
[7] http://en.wikipedia.org/wiki/Variable_envelope_return_path

29 Mar 2014 10:35pm GMT

27 Mar 2014

feedPlanet Gentoo

Sven Vermeulen: Online hardened meeting of March

I'm back from the depths of the unknown, so time to pick up my usual write-up of the online Gentoo Hardened meeting.

Toolchain

GCC 4.9 is being worked on, and might be released by end of April (based on the amount of open bugs). You can find the changes online.

Speaking of GCC, pipacs asked if it is possible in the upcoming 4.8.2 ebuilds to disable the SSP protection for development purposes (such as when you're developing GCC plugins that do similar protection measures like SSP, but you don't want those to collide with each other). Recent discussion on Gentoo development mailinglist had a consensus that the SSP protection measures (-fstack-protector) can be enabled by default, but of course if people are developing new GCC plugins which might interfere with SSP, disabling it is needed. One can use -fno-stack-protector for this, or build stuff with -D__KERNEL__ (as for kernel builds the default SSP handling is disabled anyway, allowing for kernel-specific implementations).

Other than those, there is no direct method to make SSP generally unavailable.

Blueness is also working on musc-libc on Gentoo, which would give a strong incentive for hardened embedded devices. For desktops, well, don't hold your breath just yet.

Kernel grSec/PaX

It looks like kernel 3.13 will be Ubuntu's LTS kernel choice, which also makes it the kernel version that grSecurity will put the long term support for in. And with Linux 3.14 almost out, the grsec patches for it are ready as well. Of the previous LTS kernels, 3.2 will probably finish seeing grsec support somewhere this year.

The C wrapper (called install-xattr) used to preserve xattr information during Portage builds has not been integrated in Portage yet, but the development should be finished.

During the chat session, we also discussed the gold linker and how it might be used by more and more packages (so not only by users that explicitly ask for it). udev version 210 onwards is one example, but some others exist. But other than its existence there's not much to say right here.

SELinux

The 20140311 release of the reference policy is now in the Portage tree.

Also, prometheanfire caught a vulnerability (CVE-2014-1874) in SELinux which has been fixed in the latest kernels.

System Integrity

I made a few updates to the gentoo hardening guide in XCCDF/OVAL format. Nothing major, and I still need to add in a lot of other best practices (as well as automate the tests through OVAL), but I do intend to update the files (at least the gentoo one and ssh as OpenSSH 6 is now readily available) regularly in the next few weeks.

Profiles

A few minor changes have been made to hardened/uclibc to support multilib, but other than that nothing has been done (nor needed to be done) to our profiles.

That's it for this months hardened meeting write-up. See you next time!

27 Mar 2014 9:44pm GMT

26 Mar 2014

feedPlanet Gentoo

Jan Kundrát: Tagged pointers, and saving memory in Trojita

One of the improvements which were mentioned in the recent announcement of Trojitá, a fast Qt e-mail client, were substantial memory savings and speed improvements. In the rest of this post, I would like to explain what exactly we have done and how it matters. This is going to be a technical post, so if you are not interested in C++ or software engineering, you might want to skip this article.

Planting Trees

At the core of Trojitá's IMAP implementation is the TreeItem, an abstract class whose basic layout will be familiar to anyone who has worked with a custom QAbstractItemModel reimplementation. In short, the purpose of this class is to serve as a node in the tree of items which represent all the data stored on a remote IMAP server.

The structure is tree-shaped because that's what fits both the QAbstractItemModel's and the IMAP way of working. At the top, there's a list of mailboxes. Children of these mailboxes are either other, nested mailboxes, or lists of messages. Below the lists of messages, one can find individual e-mails, and within these e-mails, individual body parts as per the recursive nature of the MIME encapsulation. (This is what enables messages with pictures attached, e-mail forwarding, and similar stuff. MIME is fun.) This tree of items is used by the QAbstractItemModel for keeping track of what is where, and for issuing the QModelIndex instances which are used by the rest of the application for accessing, requesting and manipulating the data.

When a QModelIndex is used and passed to the IMAP Model, what matters most is its internalPointer(), a void * which, within Trojitá, always points to an instance of some TreeItem subclass. Everything else, like the row() and column(), are actually not important; the pointer itself is enough to determine everything about the index in question.

Each TreeItem has to store a couple of interesting properties. Besides the usual Qt-mandated stuff like pointer to the parent item and a list of children, there are also application-specific items which enable the code to, well, actually do useful things like printing e-mail subjects or downloading mail attachments. For a mailbox, this crucial information might be the mailbox name. For a message, the UID of the message along with a pointer to the mailbox is enough to uniquely identify everything which is needed.

Lazy Loading

Enter the lazy loading. Many people confirm that Trojitá is fast, and plenty of them are not afraid to say that it is blazingly fast. This speed is enabled by the fact that Trojitá will only do the smallest amount of work required to bring the data over the network (or from disk, for that matter). If you open a huge mailbox with half a million messages, perhaps the GMail's "All messages" account, or one's LKML archive, Trojitá will not start loading half a million of subjects. Instead, the in-memory TreeItem nodes are created in a special state "no data has been requested yet". Trojitá still creates half a million items in memory, but these items are rather lightweight and only contain the absolute minimum of data they need for proper operation.

Some of these "empty" nodes are, eventually, consulted and used for item display -- perhaps because a view is attached to this model, and the view wants to show the recent mail to the user. In Qt, this usually happens via the data() method of the QAbstractItemModel, but other methods like rowCount() have a very similar effect. Whenever more data are needed, the state of the tree node changes from the initial "no data have been requested" to "loading stuff", and an asynchronous request for these data is dispatched. An important part of the tale is that the request is indeed completely asynchronous, so you won't see any blocking whatsoever in the GUI. The QTreeView will show an animation while a subtree is expanded, the message viewer might display a spinner, and the mail listing shows greyed-out "Loading..." placeholder instead of the usual message subjects.

After a short while, the data arrive and the tree node is updated with the extracted contents -- be it e-mail subject, or perhaps the attached image of dancing pigs. As the requested data are now here, the status of the tree node is updated from the previous "loading stuff" into "done". At the same time, an appropriate signal, like dataChanged or rowsInserted, is emitted. Requesting the same data again via the classic MVC API will not result in network requests, but everything will be accommodated from the local cache.

What we see now is that there is just a handful of item states, yet the typical layout of the TreeItem looks roughly like this:

enum class FetchingStatus {
    INITIAL_NOTHING_REQUESTED_YET,
    LOADING,
    DONE,
    FAILED
};
class TreeItem {
    TreeItem *m_parent;
    QList<TreeItem*> m_children;
    FetchingStatus m_status;
};

On a 64bit system, this translates to at least three 64bit words being used -- one for the painter to the parent item, one (or much more) for storage of the list of children, and one more for storing the enum FetchingStatus. That's a lot of space, given we have just created half a million of these items.

Tagged Pointers

An interesting property of a modern CPU is that the data structures must be aligned properly. A very common rule is that e.g. a 32bit integer can only start at memory offset which is a multiple of four. In hex, this means that an address, or a pointer value, could end with 0x0, or 0x4, or 0x8, or 0xc. The detailed rules are platform-specific and depend on the exact data structure which we are pointing to, but the important message is that at least some of the low bits in the pointer address are always going to be zero. Perhaps we could encode some information in there?

Turns out this is exactly what pointer tagging is about. Instead of having two members, one TreeItem * and one FetchingStatus, these are squashed into a single pointer-sized value. The CPU can no longer use the pointer value directly, all accesses have to go via an inlined function which simply masks away the lowest bits which do bring a very minor performance hit, but the memory conservation is real.

For a real-world example, see this commit in Trojitá.

Using Memory Only When Needed

Back to our example of a mailbox with 500k messages. Surely a user is only going to see a small subset of them at once, right?

That is indeed the case. We still have to at least reserve space for 500k items for technical reasons, but there is certainly no need to reserve space for heavy stuff like subjects and other headers. Indeed, in Trojitá, we track the From/To/Cc/Bcc headers, the subjects, various kinds of timestamps, other envelope items and similar stuff, and this totals a couple hundred bytes per each message. A couple hundred bytes is not much (pun intended), but "a couple hundred bytes" times "half a million" is a ton of memory.

This got implemented here. One particular benchmark which tests how fast Trojitá resynchronizes a mailbox with 100k of messages showed immediate reduction in memory usage from previous 45 MB to 25 MB. The change, again, does come with a cost; one now has to follow one more pointer redirection, and one has to perform one more dynamic allocation for each message which is actually visible. That, however, proves to be negligible during typical usage.

Measure, Don't Guess

As usual with optimizing, the real results might sometimes be surprising. A careful reader and an experienced Qt programmer might have noticed the QList above and shuddered in horror. In fact, Trojitá now uses QVector in its place, but when I was changing the code, using std::vector sounded like a no-brainer. Who needs the copy-on-write semantics here anyway, so why should I pay its price in this context? These data (list of children of an item) are not copied that often, and copying a contiguous list of pointers is pretty cheap anyway (it surely is dwarfed by dynamic allocation overhead). So we should just stick with std::vector, right?

Well, not really. It turned out that plenty of these lists are empty most of the time. If we are looking at the list of messages in our huge mailbox, chances are that most of these messages were not loaded yet, and therefore the list of children, i.e. something which represents their inner MIME structure, is likely empty. This is where the QVector really shines. Instead of using three pointers per vector, like the GCC's std::vector does, QVector is happy with a single pointer pointing to a shared null instance, something which is empty.

Now, factor of three on an item which is used half a million times, this is something which is going to hurt. That's why Trojitá eventually settled on using QVector for the m_children member. The important lesson here is "don't assume, measure".

Wrapping up

Thanks to these optimization (and a couple more, see the git log), one particular test case now runs ten times faster while simultaneously using 38% less memory -- comparing the v0.4 with v0.3.96. Trojitá was pretty fast even before, but now it really flies. The sources of memory diet were described in today's blog post; the explanation on how the time was cut is something which will have to wait for another day.

26 Mar 2014 5:02pm GMT

Sven Vermeulen: Fixing the busybox build failure

Since a few months I have a build failure every time I try to generate an initial ram file system (as my current primary workstation uses a separate /usr and LVM for everything except /boot):

* busybox: >> Compiling...
* ERROR: Failed to compile the "all" target...
* 
* -- Grepping log... --
* 
*           - busybox-1.7.4-signal-hack.patch
* busybox: >> Configuring...
*COMMAND: make -j2 CC="gcc" LD="ld" AS="as"  
*  HOSTCC  scripts/basic/fixdep
*make: execvp: /var/tmp/genkernel/18562.2920.28766.17301/busybox-1.20.2/scripts/gen_build_files.sh: Permission denied
*make: *** [gen_build_files] Error 127
*make: *** Waiting for unfinished jobs....
*/bin/sh: scripts/basic/fixdep: Permission denied
*make[1]: *** [scripts/basic/fixdep] Error 1
*make: *** [scripts_basic] Error 2

I know it isn't SELinux that is causing this, as I have no denial messages and even putting SELinux in permissive mode doesn't help. Today I found the time to look at it with more fresh eyes, and noticed that it wants to execute a file (gen_build_files.sh) situated in /var/tmp somewhere. That file system however is mounted with noexec (amongst other settings) so executing anything from within that file system is not allowed.

The solution? Update /etc/genkernel.conf and have TMPDIR point to a location where executing is allowed. Of course, this being a SELinux system, the new location will need to be labeled as tmp_t as well, but that's just a simple thing to do.

~# semanage fcontext -a -t tmp_t /var/build/genkernel(/.*)?
~# restorecon -R /var/build/genkernel

The new location is not world-writable (only for root as only root builds initial ram file systems here) so not having noexec here is ok.

26 Mar 2014 12:18pm GMT

24 Mar 2014

feedPlanet Gentoo

Sven Vermeulen: Create your own SELinux Gentoo profile

Or any other profile for that matter ;-)

A month or so ago we got the question how to enable SELinux on a Gentoo profile that doesn't have a <some profilename>/selinux equivalent. Because we don't create SELinux profiles for all possible profiles out there, having a way to do this yourself is good to know.

Sadly, the most efficient way to deal with this isn't supported by Portage: creating a parent file pointing to /usr/portage/profiles/features/selinux in /etc/portage/profile, as is done for all SELinux enabled profiles. The /etc/portage/profile location (where users can do local changes to the profile settings) does not support a parent file in there.

Luckily, enabling SELinux is a matter of merging the files in /usr/portage/profiles/features/selinux into /etc/portage/profile. If you don't have any files in there, you can blindly copy over the files from features/selinux.

Edit: aballier on #gentoo-dev mentioned that you can create a /etc/portage/make.profile directory (instead of having it be a symlink managed by eselect profile) which does support parent files. In that case, just create one with two entries: one path to the profile you want, and one path to the features/selinux location.

24 Mar 2014 7:51pm GMT

Sven Vermeulen: Hidden symbols and dynamic linking

A few weeks ago, we introduced an error in the (~arch) libselinux ebuild which caused the following stacktrace to occur every time the semanage command was invoked:

~ # semanage
Traceback (most recent call last):
  File "/usr/lib/python-exec/python2.7/semanage", line 27, in 
    import seobject
  File "/usr/lib64/python2.7/site-packages/seobject.py", line 27, in 
    import sepolicy
  File "/usr/lib64/python2.7/site-packages/sepolicy/__init__.py", line 11, in 
    import sepolgen.interfaces as interfaces
  File "/usr/lib64/python2.7/site-packages/sepolgen/interfaces.py", line 24, in 
    import access
  File "/usr/lib64/python2.7/site-packages/sepolgen/access.py", line 35, in 
    from selinux import audit2why
ImportError: /usr/lib64/python2.7/site-packages/selinux/audit2why.so: undefined symbol: sepol_set_policydb

Usually this error means that a needed library (a .so file) is missing, or is not part of the /etc/ld.so.conf list of directories to scan. And when SELinux is enabled, you might want to check the permissions of that file as well (who knows). But that wasn't the case here. After trying to figure things out (which includes switching Python versions, grepping for sepol_set_policydb in libsepol.so and more) I looked at the audit2why.c code and see if/where sepol_set_policydb is needed, as well as at the libsepol sources to see where it is defined. And yes, the call is (of course) needed, but the definition made me wonder if this wasn't a bug:

int hidden sepol_set_policydb(policydb_t * p)
{
        policydb = p;
        return 0;
}

Hidden? But, that means that the function symbol is not available for dynamic linking… So if that is the case, shouldn't audit2why.c not call it? Turns out, this was due to a fix we introduced earlier on, where libsepol got linked dynamically instead of statically (i.e. using libsepol.a). Static linking of libraries still allows for the (hidden) symbols to be used, whereas dynamic linking doesn't.

So that part of the fix got reverted (and should fix the bug we introduced), and I learned a bit more about symbols (and the hidden statement).

Bonus: if you need to check what symbols are available in a binary / shared library, use nm:

~$ nm -D /path/to/binary

24 Mar 2014 7:14pm GMT