29 May 2015

feedPlanet Debian

Roland Mas: FusionForge 6.0 final release available

Hi all,

After 4 release candidates, the FusionForge community is proud to announce the new major Fusionforge 6.0 final release.

The major changes in this version are:

Here is a more detailed list of visible changes:

Standard features:

Tracker roadmap of the 6.0 release: roadmap.

Some metrics about 6.0:

FusionForge 6.0 can be downloaded in source form from our file release system. Packages will be available in some distributions soon.

Enjoy! Your feedback to fusionforge-general@lists.fusionforge.org is welcome!

For more information on FusionForge, refer to FusionForge.org.

-- The FusionForge community

29 May 2015 2:01pm GMT

Richard Hartmann: Do what I want

Sesse just gave me the most useful piece of information of this week:

To zoom in/out in Android, you double tap and then drag your finger. All of a sudden, you can use Google Maps in one-handed operation again!

A quick search turned up this gem: Touch mechanics on Android.

Neat.

29 May 2015 10:19am GMT

Mike Gabriel: DXPC retroactively re-licensed as BSD-2-clause, nx-libs(-lite) now really DFSG-compliant

We recently had an intensive phase while reconsidering the DFSG-compliancy of the nx-libs(-lite) code base.

TL;DR; In May 2015, all versions of DXPC released before version 3.8.1 (sometime in 2002) have retroactively been re-licensed by all previous maintainers of DXPC as BSD-2-clause.

This blog arcticle is a modified version of the nxcomp/README.on-retroactive-DXPC-license file [1] and gives an overview of the discussion thread that lead to the retroactive re-licensing of DXPC.

For the full discussion, see doc/DXPC_re-licensed::debbug_784565.mbox [2] in the nx-libs source project or #784565 on the Debian bug tracker [3].

[1] https://github.com/ArcticaProject/nx-libs/blob/3.6.x/nxcomp/README.on-re...
[2] https://github.com/ArcticaProject/nx-libs/blob/3.6.x/doc/DXPC_re-license...
[3] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=784565

------------------------------------------------------------------------------

STEP 1

In May 2015, a serious license issue around the nxcomp code shipped in this source project was raised and solved on the Debian bug tracker (thanks to Francesco Poli and many others): http://bugs.debian.org/784565

From: "Francesco Poli \(wintermute\)" 
To: Debian Bug Tracking System 
Date: Wed, 06 May 2015 19:35:32 +0200

I noticed that the debian/copyright states:

[...]
| Parts of this software are derived from DXPC project. These copyright
| notices apply to original DXPC code:
|

read more

29 May 2015 7:51am GMT

Steinar H. Gunderson: Offline routing in Maps

My[1] work project for the last year or so was shown on Google I/O! And the reaction was seemingly positive, also in the tech press.

[1]: Of course, I'm only one of many developers.

29 May 2015 12:21am GMT

28 May 2015

feedPlanet Debian

Sven Hoexter: RMS, free software and where I fail the goal

You might have already read this comment by RMS in the Guardian. That comment and a recent discussion about the relevance of GPL changes post GPLv2 made me think again about the battle RMS started to fight. While some think RMS should "retire", at least I still fail on my personal goal to not depend on non-free software and services. So for me this battle is far from over, and here is my personal list of "non-free debt" I've to pay off.

general purpose systems aka your computer

Looking at the increasing list of firmware blobs required to use a GPU, wireless chipsets and more and more wired NICs, the situation seems to be worse then in the late 90s. Back then the primary issue was finding supported hardware, but the driver was free. Nowadays even the open sourced firmware often requires obscure patched compilers to build. If I look at this stuff I think the OpenBSD project got that right with the more radical position.

Oh and then there is CPU microcode. I'm not yet sure what to think about it, but in the end it's software and it's not open source. So it's non-free software running on my system.

Maybe my memory is blurred due to the fact, that the seperation of firmware from the Linux kernel, and proper firmware loading got implemented only years later. I remember the discussion about the pwc driver and its removal from Linux. Maybe the situation wasn't better at that time but the firmware was just hidden inside the Linux driver code?

On my system at work I've to add the Flash plugin to the list due to my latest test with Prezi which I'll touch later.

I also own a few Humble Indie bundles. I played parts of Osmos after a recommendation by Joey Hess, I later finished to play through Limbo and I got pretty far with Machinarium on a Windows system I still had at that time. I also tried a few others but never got far or soon lost interest.

Another thing I can not really get rid of is unrar because of stuff I need to pull from xda-developer links just to keep a cell phone running. Update: Josh Triplett pointed out that there is unar available in the Debian archive. And indeed that one works on the rar file I just extracted.

Android ecosystem

I will soon get rid of a stock S3 mini and try to replace it with a moto g loaded with CyanogenMod. That leaves me with a working phone with a OS that just works because of a shitload of non-free blobs. The time and work required to get there is another story. Among others you need a new bootloader that requires a newer fastboot compared to what we have in Jessie, and later you also need the newer adb to be able to sideload the CM image. There I gave in and just downloaded the pre build SDK from Google. And there you've another binary I did not even try to build from source. Same for the CM image itself, though that's not that much different from using a GNU/Linux distribution if you ignore the trust issues.

It's hard to trust the phone I've build that way, but it's the best I can get at the moment with at least some bigger chunks of free software inside. So let's move to the applications on the phone. I do not use GooglePlay, so I rely on f-droid and freeware I can download directly from the vendor.

"Cloud" services

This category mixes a lot with the stuff listed above, most of them are not only an application, in fact Threema and Wunderlist are useless without the backend service. And Opera is just degraded to a browser - and to be replaced with Firefox - if you discount the compression proxy.

The other big addition in this category is Prezi. We tried it out at work after it got into my focus due to a post by Dave Aitel. It's kind of the poster child of non-freeness. It requires a non-free, unstable, insecure and half way deprecated browser plugin to work, you can not download your result in a useful format, you've to buy storage for your presentation at this one vendor, you've to pay if you want to keep your presentation private. It's the perfect lockin situation. But still it's very convenient, prevents a lot of common mistakes you can make when you create a presentation and they invented a new concept of presenting.

I know about impress.js(hosted on a non-free platform by the way, but at least you can export it from there) and I also know about hovercraft. I'm impressed by them, but it's still not close to the ease of use of Prezi. So here you can also very prominently see the cost of free and non-free software. Invest the time and write something cool with CSS3 and impress.js or pay Prezi to just klick yourself through. To add something about the instability - I had to use a windows laptop for presenting with Prezi because the Flash plugin on Jessie crashed in the presentation mode, I did not yet check the latest Flash update. I guess that did not make the situation worse, it already is horrible.

I also use kind of database services like duden.de and dict.cc. When I was younger you bought such things printed on dead trees but they did not update very well.

Thinking a bit further, a Certification Authority is not only questionable due to the whole trust issue, they also provide OCSP responder as kind of a web service. And I've already had the experience what the internet looks like when the OCSP systems of GlobalSign failed.

So there is still a lot to fight for and a lot of "personal non-free debt" to pay off.

28 May 2015 7:31pm GMT

Richard Hartmann: On SourceForge

You either die a hero or you live long enough to see yourself become the villain.

And yes, we all know that that SF decided to wrap crapware around Windows installers ages ago and then made it opt-in after the backlash. Doing so for stale accounts makes sense from their PoV, which makes it all the worse.

And no, I don't know how stale that account actually was, but that's irrelevant in this context either way.

28 May 2015 12:09pm GMT

Steve Kemp: A brief examination of tahoe-lafs

Continuing the theme from the last post I made, I've recently started working my way down the list of existing object-storage implementations.

tahoe-LAFS is a well-established project which looked like a good fit for my needs:

Getting the system up and running, on four nodes, was very simple. Setup a single/simple "introducer" which is a well-known node that all hosts can use to find each other, and then setup four deamons for storage.

When files are uploaded they are split into chunks, and these chunks are then distributed amongst the various nodes. There are some configuration settings which determine how many chunks files are split into (10 by default), how many chunks are required to rebuild the file (3 by default) and how many copies of the chunks will be created.

The biggest problem I have with tahoe is that there is no rebalancing support: Setup four nodes, and the space becomes full? You can add more nodes, new uploads go to the new nodes, while old ones stay on the old. Similarly if you change your replication-counts because you're suddenly more/less paranoid this doesn't affect existing nodes.

In my perfect world you'd distribute blocks around pretty optimistically, and I'd probably run more services:

The storage nodes would have the primitives "List all blocks", "Get block", "Put block", and using that you could ensure that each node had sent its data to at least N other nodes. This could be done in the background.

The indexer would be responsible for keeping track of which blocks live where, and which blocks are needed to reassemble upload N. There's probably more that it could do.

28 May 2015 12:00am GMT

27 May 2015

feedPlanet Debian

Norbert Preining: USB stick update: TAILS 1.4, GParted 0.22, SysResCD 4.5.2, Debian Jessie

I have posted a view times (here and here) about how to get a multi-boot/multi-purpose USB stick working. Now that TAILS has seens a major upgrade, and Debian 8.0 Jessie has been released, I think it is time to update the procedure to reflect the latest releases. That turned out to be a painful experience, in particular since Debian removed support for any reasonable boot method.

multi-usb-stick

So going through these explanations one will end up with a usable USB stick that can boot you into TAILS, System Rescue CD, GNU Parted Live CD, but unfortunately not anymore to boot into an installation of Debian 8.0 Jessie installation. But the USB stick will still be usable as normal media.

Let us repeat some things from the original post concerning the wishlist and the main players:

I have a long wishlist of items a boot stick should fulfill

This time I have added the GNOME/GNU Partition Editor gparted as it came in useful at times.

Requirements

A USB stick, the iso images of TAILS 1.4, SystemRescueCD 4.5.2, GParted Lice CD 0.22.0, and some tool to access iso images, for example ISOmaster (often available from your friendly Linux distribution).

I assume that you have already an USB stick prepared as described previously. If this is not the case, please go there and follow the section on preparing your usb stick.

Two types of boot options

We will employ two different approaches to boot special systems: the one is directly from an iso image, the other via extraction of the necessary kernels and images.

At the moment we have the following status with respect to boot methods:

What is a pity is that during the testing phase, booting and installation from testing images worked for Debian as documented previously. But with the final images (and my guess it has to do with systemd, wouldn't surprise me), there is no Debian CD detected as it cannot find the iso image on the USB stick. Bummer. That means that for having a Debian/USB stick you need a dedicated one.

Booting from ISO image

Grub has gained quite some time ago the ability to boot directly from an ISO image. In this case the iso image is mounted via loopback, and the kernel and initrd are specified relatively to the iso image root.

For both SystemRescueCD and GNOME Partition Live CD, just drop the iso files into /boot/iso/, in my case /boot/iso/systemrescuecd-x86-4.5.2.iso and /boot/iso/gparted-live-0.22.0-1-i586.iso.

After that, entries like the following have to be added to grub.cfg. For the full list see grub.cfg:

submenu "System Rescue CD 4.5.2 (via ISO) ---> " {
  set isofile="/boot/iso/systemrescuecd-x86-4.5.2.iso"
  menuentry "SystemRescueCd (64bit, default boot options)" {
        set gfxpayload=keep
        loopback loop (hd0,1)$isofile
        linux   (loop)/isolinux/rescue64 isoloop=$isofile
        initrd  (loop)/isolinux/initram.igz
  }
  ...
}
 
submenu "GNU/Gnome Parted Live CD 0.22.0 (via ISO) ---> " {
  set isofile="/boot/iso/gparted-live-0.22.0-1-i586.iso"
  menuentry "GParted Live (Default settings)"{
    loopback loop (hd0,1)$isofile
    linux (loop)/live/vmlinuz boot=live username=user config components quiet noswap noeject  ip=  nosplash findiso=$isofile
    initrd (loop)/live/initrd.img
  }
  ...
}

Note the added isoloop=$isofile and findiso=$isofile that helps the installer find the iso images.

Booting via extraction of kernels and images

This is a bit more tedious, but still not too bad.

Installation of TAILS files

Assuming you have access to the files on the TAILS CD via the directory ~/tails, execute the following commands:

mkdir -p /usbstick/boot/tails
cp -a ~/tails/live/* /usbstick/boot/tails/

The grub.cfg entries look now similar to the following:

submenu "TAILS Environment 1.4 ---> " {
  menuentry "Tails64 Live System" {
        linux   /boot/tails/vmlinuz2 boot=live live-media-path=/boot/tails config live-media=removable nopersistent noprompt timezone=Etc/UTC block.events_dfl_poll_msecs=1000 splash noautologin module=Tails  libata.force=noncq
        initrd  /boot/tails/initrd2.img
  }
  ...
}

The important part here is the live-media-path=/boot/tails, otherwise TAILS will not find the correct files for booting. The rest of the information was extracted from the boot setup of TAILS itself.

Current status of USB stick

Just to make sure, the usb stick should contain at the current stage the following files:

/boot/
    iso/
        gparted-live-0.22.0-1-i586.iso
        systemrescuecd-x86-4.5.2.iso
    tails/
        vmlinuz Tails.module initrd.img ....
    grub/
        fonts/
            lots of files
        locale/
            lots of files
        x86_64-efi/
            lots of files
        font.pf2
        grubenv
        grub.cfg            *this file we create in the next step!!*
/EFI
    BOOT/
        BOOTX64.EFI

The Grub config file grub.cfg

The final step is to provide a grub config file in /usbstick/boot/grub/grub.cfg. I created one by looking at the isoboot.cfg files both in the SystemRescueCD, TAILS iso images, GParted iso image, and the Debian/Jessie image, and converting them to grub syntax. Excerpts have been shown above in the various sections.

I spare you all the details, grab a copy here: grub.cfg

Conclusion

That's it. Now you can anonymously provide data about your evil government, rescue your friends computer, fix a forgotten Windows password, and above all, install a proper free operating system.

If you have any comments, improvements or suggestions, please drop me a comment. I hope this helps a few people getting a decent USB boot stick running.

Enjoy.

Postscriptum concerning Debian/Jessie

So I have first tried to boot directly into the Debian Jessie firmware ISO image by dropping the ISO into /boot/iso/firmware-8.0.0-amd64-i386-netinst.iso and adding a grub entry like:

submenu "Debian 8.0 Jessie NetInstall ---> " {
    set isofile="/boot/iso/firmware-8.0.0-amd64-i386-netinst.iso"
 
    menuentry '64 bit Install' {
        set background_color=black
        loopback loop (hd0,1)$isofile
        linux    (loop)/install.amd/vmlinuz iso-scan/ask_second_pass=true iso-scan/filename=$isofile vga=788 -- quiet 
        initrd   (loop)/install.amd/initrd.gz
    }
    ...
}

That was a no go. I have added the iso-scan/ask_second_pass=true iso-scan/filename=$isofile thingy after some research in forum and web, without any change. Of course I have also tried the official ISO image debian-8.0.0-amd64-netinst.iso, with the same effect.

Although I was sure it doesn't make any difference, I have also tried to extract the kernel and initrd, and boot directly from it, i.e., copying the files to /usbstick/boot/debian/ as follows:

mkdir -p /usbstick/boot/debian
cp -a ~/tails/install.amd /usbstick/boot/debian/
cp -a ~/tails/install.386 /usbstick/boot/debian/

In addition, copy the iso image itself into /usbstick/boot/iso (or directly into the root of the usb stick, didn't change anything), and added a grub.cfg entry as follows:

submenu "Debian 8.0 Jessie NetInstall ---> " {
  menuentry '64 bit Install' {
    set background_color=black
    linux    /boot/debian/install.amd/vmlinuz vga=788 -- quiet 
    initrd   /boot/debian/install.amd/initrd.gz
  }
  ...
}

All these were without success, always ending up in error messages like: Mounting sdb, this is not a Debian Installation CD, etc etc.

I have also submitted a bug report to the installation reports, unfortunately no answer (not that I expected one). In case someone has a better idea, please let me know!

It is very sad that these kind of support is removed. The Installation Manual lists some options to install from USB, and the only difference is that there syslinux on a FAT16 system is used. This restricts the USB stick size a lot, and does not allow for easy multi boot via grub.

As reported here, I had this actually running. Unfortunately, I stupidly removed the old iso image and replaced it with the current Jessie ISO image, assuming that there will be no regression. Wrong assumption. Now I cannot even investigate the changes by looking into the initrd. Looking at the date of the original post I see that it is more or less one year ago, so before the Systemd introduction. I don't know whether this has any effect, I doubt, as the installer is separate. But something happened in the mean time. Bad for us.

Email this to someonePrint this pageShare on FacebookShare on Google+Tweet about this on TwitterShare on LinkedInFlattr the author

27 May 2015 10:48pm GMT

Daniel Pocock: Quick start using Blender for video editing

Although it is mostly known for animation, Blender includes a non-linear video editing system that is available in all the current stable versions of Debian, Ubuntu and Fedora.

Here are some screenshots showing how to start editing a video of a talk from a conference.

In this case, there are two input files:

  • A video file from a DSLR camera, including an audio stream from a microphone on the camera
  • A separate audio file with sound captured by a lapel microphone attached to the speaker's smartphone. This is a much better quality sound and we would like this to replace the sound included in the video file.

Open Blender and choose the video editing mode

Launch Blender and choose the video sequence editor from the pull down menu at the top of the window:

Now you should see all the video sequence editor controls:

Setup the properties for your project

Click the context menu under the strip editor panel and change the panel to a Properties panel:

The video file we are playing with is 720p, so it seems reasonable to use 720p for the output too. Change that here:

The input file is 25fps so we need to use exactly the same frame rate for the output, otherwise you will either observe the video going at the wrong speed or there will be a conversion that is CPU intensive and degrades the quality:

Now specify an output filename and location:

Specify the file format:

and the video codec:

and specify the bitrate (smaller bitrate means smaller file but lower quality):

Specify the AAC audio codec:

Now your basic rendering properties are set. When you want to generate the output file, come back to this panel and use the Animation button at the top.

Editing the video

Use the context menu to change the properties panel back to the strip view panel:

Add the video file:

and then right click the video strip (the lower strip) to highlight it and then add a transform strip:

Audio waveform

Right click the audio strip to highlight it and then go to the properties on the right hand side and click to show the waveform:

Rendering length

By default, Blender assumes you want to render 250 frames of output. Looking in the properties to the right of the audio or video strip you can see the actual number of frames. Put that value in the box at the bottom of the window where it says 250:

Enable AV-sync

Also at the bottom of the window is a control to enable AV-sync. If your audio and video are not in sync when you preview, you need to set this AV-sync option and also make sure you set the frame rate correctly in the properties:

Add the other sound strip

Now add the other sound file that was recorded using the lapel microphone:

Enable the waveform display for that sound strip too, this will allow you to align the sound strips precisely:

You will need to listen to the strips to make an estimate of the time difference. Use this estimate to set the "start frame" in the properties for your audio strip, it will be a negative value if the audio strip starts before the video. You can then zoom the strip panel to show about 3 to 5 seconds of sound and try to align the peaks. An easy way to do this is to look for applause at the end of the audio strips, the applause generates a large peak that is easily visible.

Once you have synced the audio, you can play the track and you should not be able to hear any echo. You can then silence the audio track from the camera by right clicking it, look in the properties to the right and change volume to 0.

Make any transforms you require

For example, to zoom in on the speaker, right click the transform strip (3rd from the bottom) and then in the panel on the right, click to enable "Uniform Scale" and then set the scale factor as required:

Next steps

There are plenty of more comprehensive tutorials, including some videos on Youtube, explaining how to do more advanced things like fading in and out or zooming and panning dynamically at different points in the video.

If the lighting is not good (faces too dark, for example), you can right click the video strip, go to the properties panel on the right hand side and click Modifiers, Add Strip Modifier and then select "Color Balance". Use the Lift, Gamma and Gain sliders to adjust the shadows, midtones and highlights respectively.

27 May 2015 8:14pm GMT

Vincent Bernat: Live patching QEMU for VENOM mitigation

CVE-2015-3456, also known as VENOM, is a security vulnerability in QEMU virtual floppy controller:

The Floppy Disk Controller (FDC) in QEMU, as used in Xen […] and KVM, allows local guest users to cause a denial of service (out-of-bounds write and guest crash) or possibly execute arbitrary code via the FD_CMD_READ_ID, FD_CMD_DRIVE_SPECIFICATION_COMMAND, or other unspecified commands.

Even when QEMU has been configured with no floppy drive, the floppy controller code is still active. The vulnerability is easy to test1:

#define FDC_IOPORT 0x3f5
#define FD_CMD_READ_ID 0x0a

int main() {
    ioperm(FDC_IOPORT, 1, 1);
    outb(FD_CMD_READ_ID, FDC_IOPORT);
    for (size_t i = 0;; i++)
        outb(0x42, FDC_IOPORT);
    return 0;
}

Once the fix installed, all processes still have to be restarted for the upgrade to be effective. It is possible to minimize the downtime by leveraging virsh save.

Another possibility would be to patch the running processes. The Linux kernel attracted a lot of interest in this area, with solutions like Ksplice (mostly killed by Oracle), kGraft (by Red Hat) and kpatch (by Suse) and the inclusion of a common framework in the kernel. The userspace has far less out-of-the-box solutions2.

I present here a simple and self-contained way to patch a running QEMU to remove the vulnerability without requiring any sensible downtime. Here is a short demonstration:

Proof of concept

First, let's find a workaround that would be simple to implement through live patching: while modifying running code text is possible, it is easier to modify a single variable.

Concept

Looking at the code of the floppy controller and the patch, we can avoid the vulnerability by not accepting any command on the FIFO port. Each request would be answered by "Invalid command" (0x80) and a user won't be able to push more bytes to the FIFO until the answer is read and the FIFO queue reset. Of course, the floppy controller would be rendered useless in this state. But who cares?

The list of commands accepted by the controller on the FIFO port is contained in the handlers[] array:

static const struct {
    uint8_t value;
    uint8_t mask;
    const char* name;
    int parameters;
    void (*handler)(FDCtrl *fdctrl, int direction);
    int direction;
} handlers[] = {
    { FD_CMD_READ, 0x1f, "READ", 8, fdctrl_start_transfer, FD_DIR_READ },
    { FD_CMD_WRITE, 0x3f, "WRITE", 8, fdctrl_start_transfer, FD_DIR_WRITE },
    /* [...] */
    { 0, 0, "unknown", 0, fdctrl_unimplemented }, /* default handler */
};

To avoid browsing the array each time a command is received, another array is used to map each command to the appropriate handler:

/* Associate command to an index in the 'handlers' array */
static uint8_t command_to_handler[256];

static void fdctrl_realize_common(FDCtrl *fdctrl, Error **errp)
{
    int i, j;
    static int command_tables_inited = 0;

    /* Fill 'command_to_handler' lookup table */
    if (!command_tables_inited) {
        command_tables_inited = 1;
        for (i = ARRAY_SIZE(handlers) - 1; i >= 0; i--) {
            for (j = 0; j < sizeof(command_to_handler); j++) {
                if ((j & handlers[i].mask) == handlers[i].value) {
                    command_to_handler[j] = i;
                }
            }
        }
    }
    /* [...] */
}

Our workaround is to modify the command_to_handler[] array to map all commands to the fdctrl_unimplemented() handler (the last one in the handlers[] array).

Testing with gdb

To check if the workaround works as expected, we test it with gdb. Unless you have compiled QEMU yourself, you need to install a package with debug symbols. Unfortunately, on Debian, they are not available, yet3. On Ubuntu, you can install the qemu-system-x86-dbgsym package after enabling the appropriate repositories.

The following function for gdb maps every command to the unimplemented handler:

define patch
  set $handler = sizeof(handlers)/sizeof(*handlers)-1
  set $i = 0
  while ($i < 256)
   set variable command_to_handler[$i++] = $handler
  end
  printf "Done!\n"
end

Attach to the vulnerable process (with attach), call the function (with patch) and detach of the process (with detach). You can check that the exploit is not working anymore. This could be easily automated.

Limitations

Using gdb has two main limitations:

  1. It needs to be installed on each host to be patched.
  2. The debug packages need to be installed as well. Moreover, it can be difficult to fetch previous versions of those packages.

Writing a custom patcher

To overcome those limitations, we can write a customer patcher using the ptrace() system call without relying on debug symbols being present.

Finding the right memory spot

Before being able to modify the command_to_handler[] array, we need to know its location. The first clue is given by the symbol table. To query it, use readelf -s:

$ readelf -s /usr/lib/debug/.build-id/09/95121eb46e2a4c13747ac2bad982829365c694.debug | \
>   sed -n -e 1,3p -e /command_to_handler/p

Symbol table '.symtab' contains 27066 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
  8485: 00000000009f9d00   256 OBJECT  LOCAL  DEFAULT   26 command_to_handler

This table is usually stripped out of the executable to save space, like shown below:

$ file -b /usr/bin/qemu-system-x86_64 | tr , \\n
ELF 64-bit LSB shared object
 x86-64
 version 1 (SYSV)
 dynamically linked
 interpreter /lib64/ld-linux-x86-64.so.2
 for GNU/Linux 2.6.32
 BuildID[sha1]=0995121eb46e2a4c13747ac2bad982829365c694
 stripped

If your distribution provides a debug package, the debug symbols are installed in /usr/lib/debug. Most modern distributions are now relying on the build ID4 to map an executable to its debugging symbols, like the example above. Without a debug package, you need to recompile the existing package without stripping debug symbols in a clean environment5. On Debian, this can be done by setting the DEB_BUILD_OPTIONS environment variable to nostrip.

We have now two possible cases:

The easy case

On x86, here is the standard layout of a regular Linux process in memory6:

Memory layout of a regular process on x86

The random gaps (ASLR) are here to prevent an attacker from reliably jumping to a particular exploited function in memory. On x86-64, the layout is quite similar. The important point is that the base address of the executable is fixed.

The memory mapping of a process is also available through /proc/PID/maps. Here is a shortened and annotated example on x86-64:

$ cat /proc/3609/maps
00400000-00401000         r-xp 00000000 fd:04 483  not-qemu [text segment]
00601000-00602000         r--p 00001000 fd:04 483  not-qemu [data segment]
00602000-00603000         rw-p 00002000 fd:04 483  not-qemu [BSS segment]
[random gap]
02419000-0293d000         rw-p 00000000 00:00 0    [heap]
[random gap]
7f0835543000-7f08356e2000 r-xp 00000000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08356e2000-7f08358e2000 ---p 0019f000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e2000-7f08358e6000 r--p 0019f000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e6000-7f08358e8000 rw-p 001a3000 fd:01 9319 /lib/x86_64-linux-gnu/libc-2.19.so
7f08358e8000-7f08358ec000 rw-p 00000000 00:00 0
7f08358ec000-7f083590c000 r-xp 00000000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835aca000-7f0835acd000 rw-p 00000000 00:00 0
7f0835b08000-7f0835b0c000 rw-p 00000000 00:00 0
7f0835b0c000-7f0835b0d000 r--p 00020000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835b0d000-7f0835b0e000 rw-p 00021000 fd:01 5138 /lib/x86_64-linux-gnu/ld-2.19.so
7f0835b0e000-7f0835b0f000 rw-p 00000000 00:00 0
[random gap]
7ffdb0f85000-7ffdb0fa6000 rw-p 00000000 00:00 0    [stack]

With a regular executable, the value given in the symbol table is an absolute memory address:

$ readelf -s not-qemu | \
>   sed -n -e 1,3p -e /command_to_handler/p

Symbol table '.dynsym' contains 9 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    47: 0000000000602080   256 OBJECT  LOCAL  DEFAULT   25 command_to_handler

So, the address of command_to_handler[], in the above example, is just 0x602080.

The hard case

To enhance security, it is possible to load some executables at a random base address, just like a library. Such an executable is called a Position Independent Executable (PIE). An attacker won't be able to rely on a fixed address to find some helpful function. Here is the new memory layout:

Memory layout of a PIE process on x86

With a PIE process, the value in the symbol table is now an offset from the base address.

$ readelf -s not-qemu-pie | sed -n -e 1,3p -e /command_to_handler/p

Symbol table '.dynsym' contains 17 entries:
   Num:    Value          Size Type    Bind   Vis      Ndx Name
    47: 0000000000202080   256 OBJECT  LOCAL  DEFAULT   25 command_to_handler

If we look at /proc/PID/maps, we can figure out where the array is located in memory:

$ cat /proc/12593/maps
7f6c13565000-7f6c13704000 r-xp 00000000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13704000-7f6c13904000 ---p 0019f000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13904000-7f6c13908000 r--p 0019f000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c13908000-7f6c1390a000 rw-p 001a3000 fd:01 9319  /lib/x86_64-linux-gnu/libc-2.19.so
7f6c1390a000-7f6c1390e000 rw-p 00000000 00:00 0
7f6c1390e000-7f6c1392e000 r-xp 00000000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b2e000-7f6c13b2f000 r--p 00020000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b2f000-7f6c13b30000 rw-p 00021000 fd:01 5138  /lib/x86_64-linux-gnu/ld-2.19.so
7f6c13b30000-7f6c13b31000 rw-p 00000000 00:00 0
7f6c13b31000-7f6c13b33000 r-xp 00000000 fd:04 4594  not-qemu-pie [text segment]
7f6c13cf0000-7f6c13cf3000 rw-p 00000000 00:00 0
7f6c13d2e000-7f6c13d32000 rw-p 00000000 00:00 0
7f6c13d32000-7f6c13d33000 r--p 00001000 fd:04 4594  not-qemu-pie [data segment]
7f6c13d33000-7f6c13d34000 rw-p 00002000 fd:04 4594  not-qemu-pie [BSS segment]
[random gap]
7f6c15c46000-7f6c15c67000 rw-p 00000000 00:00 0     [heap]
[random gap]
7ffe823b0000-7ffe823d1000 rw-p 00000000 00:00 0     [stack]

The base address is 0x7f6c13b31000, the offset is 0x202080 and therefore, the location of the array is 0x7f6c13d33080. We can check with gdb:

$ print &command_to_handler
$1 = (uint8_t (*)[256]) 0x7f6c13d33080 <command_to_handler>

Patching a memory spot

Once we know the location of the command_to_handler[] array in memory, patching it is quite straightforward. First, we start tracing the target process:

/* Attach to the running process */
static int
patch_attach(pid_t pid)
{
    int status;

    printf("[.] Attaching to PID %d...\n", pid);
    if (ptrace(PTRACE_ATTACH, pid, NULL, NULL) == -1) {
        fprintf(stderr, "[!] Unable to attach to PID %d: %m\n", pid);
        return -1;
    }

    if (waitpid(pid, &status, 0) == -1) {
        fprintf(stderr, "[!] Error while attaching to PID %d: %m\n", pid);
        return -1;
    }
    assert(WIFSTOPPED(status)); /* Tracee may have died */

    if (ptrace(PTRACE_GETSIGINFO, pid, NULL, &si) == -1) {
        fprintf(stderr, "[!] Unable to read siginfo for PID %d: %m\n", pid);
        return -1;
    }
    assert(si.si_signo == SIGSTOP); /* Other signals may have been received */

    printf("[*] Successfully attached to PID %d\n", pid);
    return 0;
}

Then, we retrieve the command_to_handler[] array, modify it and put it back in memory7.

static int
patch_doit(pid_t pid, unsigned char *target)
{
    int ret = -1;
    unsigned char *command_to_handler = NULL;
    size_t i;

    /* Get the table */
    printf("[.] Retrieving command_to_handler table...\n");
    command_to_handler = ptrace_read(pid,
                                     target,
                                     QEMU_COMMAND_TO_HANDLER_SIZE);
    if (command_to_handler == NULL) {
        fprintf(stderr, "[!] Unable to read command_to_handler table: %m\n");
        goto out;
    }

    /* Check if the table has already been patched. */
    /* [...] */

    /* Patch it */
    printf("[.] Patching QEMU...\n");
    for (i = 0; i < QEMU_COMMAND_TO_HANDLER_SIZE; i++) {
        command_to_handler[i] = QEMU_NOT_IMPLEMENTED_HANDLER;
    }
    if (ptrace_write(pid, target, command_to_handler,
           QEMU_COMMAND_TO_HANDLER_SIZE) == -1) {
        fprintf(stderr, "[!] Unable to patch command_to_handler table: %m\n");
        goto out;
    }
    printf("[*] QEMU successfully patched!\n");
    ret = 0;

out:
    free(command_to_handler);
    return ret;
}

Since ptrace() only allows to read or write a word at a time, ptrace_read() and ptrace_write() are wrappers to read or write arbitrary large chunks of memory8. Here is the code for ptrace_read():

/* Read memory of the given process */
static void *
ptrace_read(pid_t pid, void *address, size_t size)
{
    /* Allocate the buffer */
    uword_t *buffer = malloc((size/sizeof(uword_t) + 1)*sizeof(uword_t));
    if (!buffer) return NULL;

    /* Read word by word */
    size_t readsz = 0;
    do {
        errno = 0;
        if ((buffer[readsz/sizeof(uword_t)] =
                ptrace(PTRACE_PEEKTEXT, pid,
                       (unsigned char*)address + readsz,
                       0)) && errno) {
            fprintf(stderr, "[!] Unable to peek one word at address %p: %m\n",
                    (unsigned char *)address + readsz);
            free(buffer);
            return NULL;
        }
        readsz += sizeof(uword_t);
    } while (readsz < size);
    return (unsigned char *)buffer;
}

Putting the pieces together

The patcher is provided with the following information:

The main steps are:

  1. Attach to the process with ptrace().
  2. Get the executable name from /proc/PID/exe.
  3. Parse /proc/PID/maps to find the address of the text segment (it's the first one).
  4. Do some sanity checks:
    • check there is a ELF header at this location (4-byte magic number),
    • check the executable type (ET_EXEC for regular executables, ET_DYN for PIE), and
    • get the build ID and compare with the expected one.
  5. From the base address and the provided offset, compute the location of the command_to_handler[] array.
  6. Patch it.

You can find the complete patcher on GitHub.

$ ./patch --build-id 0995121eb46e2a4c13747ac2bad982829365c694 \
>         --offset 9f9d00 \
>         --pid 16833
[.] Attaching to PID 16833...
[*] Successfully attached to PID 16833
[*] Executable name is /usr/bin/qemu-system-x86_64
[*] Base address is 0x7f7eea912000
[*] Both build IDs match
[.] Retrieving command_to_handler table...
[.] Patching QEMU...
[*] QEMU successfully patched!

  1. The complete code for this test is on GitHub.

  2. An interesting project seems to be Katana. But there are also some insightful hacking papers on the subject.

  3. Some packages come with a -dbg package with debug symbols, some others don't. Fortunately, a proposal to automatically produce debugging symbols for everything is near completion.

  4. The Fedora Wiki contains the rationale behind the build ID.

  5. If the build is incorrectly reproduced, the build ID won't match. The information provided by the debug symbols may or may not be correct. Debian currently has a reproducible builds effort to ensure that each package can be reproduced.

  6. Anatomy of a program in memory is a great blog post explaining in more details how a program lives in memory.

  7. Being an uninitialized static variable, the variable is in the BSS section. This section is mapped to a writable memory segment. If it wasn't the case, with Linux, the ptrace() system call is still allowed to write. Linux will copy the page and mark it as private.

  8. With Linux 3.2 or later, process_vm_readv() and process_vm_writev() can be used to transfer data from/to a remote process without using ptrace() at all. However, ptrace() would still be needed to reliably stop the main thread.

27 May 2015 3:52pm GMT

Jonathan Carter: Of course I support Jonathan

riddel-support-banner-300x82

Spending yesterday mostly away from the computer screen, I was shocked this morning when I read about the Ubuntu Community Council's request for Jonathan Ridell to step down from the Kubuntu Council. I knew that things have been rough lately and honestly there were some situations that Jonathan could have handled better, but I didn't expect anything as drastic and sudden as this without seeing any warning signs.

Looking at the mails that Scott Kitterman posted sent by the Kubuntu Council, it seems like it's been a surprise to KC as well.

I'm disappointed in the way the Ubuntu Community Council has handled this and I think the way they treated Jonathan is appalling, even taking into account that he could've communicated his grievances better. I'm also unconvinced that the Ubuntu Community Council is as beneficial to the Ubuntu community in its current form as it could be. The way it is structured and reports to the SABDFL makes that it will always favour Canonical when there's a conflict of interest. I brought this up with two different CC members last year who both provided shruggy answers in the vein of "Sorry, but we have a framework that's set up on how we can work in here and there's just so much we can do about it." - they seem to fear the leadership too much to question it, and it's a pity, because everyone makes mistakes.

This request to step down is probably going to sour the Ubuntu project's relationship with Jonathan Ridell even more, which is especially sad because he's one of the really good community guys left that keeps both the CoC and the original Ubuntu manifesto ethos in high regard while striving for technical excellence. On top of that, it seems like it may result in at least another such person leaving.

I hope that the CC also takes this opportunity to take a step back and re-avaluate it's structure and purpose, instead of just shrugging it off with a corporate-sounding statement. I'd also urge them to retract their statement to Jonathan Ridell and attempt to find a more amicable solution.

27 May 2015 3:41pm GMT

Dirk Eddelbuettel: drat 0.0.4: Yet more features and documentation

A new version, now at 0.0.4, of the drat package arrived on CRAN yesterday. Its name stands for drat R Archive Template, and it helps with easy-to-create and easy-to-use repositories for R packages, and is finding increasing by other projects.

Version 0.0.4 brings both new code and more documentation:

Courtesy of CRANberries, there is a comparison to the previous release. More detailed information is on the drat page.

This post by Dirk Eddelbuettel originated on his Thinking inside the box blog. Please report excessive re-aggregation in third-party for-profit settings.

27 May 2015 12:06pm GMT

Matthew Garrett: This is not the UEFI backdoor you are looking for

This is currently the top story on the Linux subreddit. It links to this Tweet which demonstrates using a System Management Mode backdoor to perform privilege escalation under Linux. This is not a story.

But first, some background. System Management Mode (SMM) is a feature in most x86 processors since the 386SL back in 1990. It allows for certain events to cause the CPU to stop executing the OS, jump to an area of hidden RAM and execute code there instead, and then hand off back to the OS without the OS knowing what just happened. This allows you to do things like hardware emulation (SMM is used to make USB keyboards look like PS/2 keyboards before the OS loads a USB driver), fan control (SMM will run even if the OS has crashed and lets you avoid the cost of an additional chip to turn the fan on and off) or even more complicated power management (some server vendors use SMM to read performance counters in the CPU and adjust the memory and CPU clocks without the OS interfering).

In summary, SMM is a way to run a bunch of non-free code that probably does a worse job than your OS does in most cases, but is occasionally helpful (it's how your laptop prevents random userspace from overwriting your firmware, for instance). And since the RAM that contains the SMM code is hidden from the OS, there's no way to audit what it does. Unsurprisingly, it's an interesting vector to insert malware into - you could configure it so that a process can trigger SMM and then have the resulting SMM code find that process's credentials structure and change it so it's running as root.

And that's what Dmytro has done - he's written code that sits in that hidden area of RAM and can be triggered to modify the state of the running OS. But he's modified his own firmware in order to do that, which isn't something that's possible without finding an existing vulnerability in either the OS or (or more recently, and) the firmware. It's an excellent demonstration that what we knew to be theoretically possible is practically possible, but it's not evidence of such a backdoor being widely deployed.

What would that evidence look like? It's more difficult to analyse binary code than source, but it would still be possible to trace firmware to observe everything that's dropped into the SMM RAM area and pull it apart. Sufficiently subtle backdoors would still be hard to find, but enough effort would probably uncover them. A PC motherboard vendor managed to leave the source code to their firmware on an open FTP server and copies leaked into the wild - if there's a ubiquitous backdoor, we'd expect to see it there.

But still, the fact that system firmware is mostly entirely closed is still a problem in engendering trust - the means to inspect large quantities binary code for vulnerabilities is still beyond the vast majority of skilled developers, let alone the average user. Free firmware such as Coreboot gets part way to solving this but still doesn't solve the case of the pre-flashed firmware being backdoored and then installing the backdoor into any new firmware you flash.

This specific case may be based on a misunderstanding of Dmytro's work, but figuring out ways to make it easier for users to trust that their firmware is tamper free is going to be increasingly important over the next few years. I have some ideas in that area and I hope to have them working in the near future.

comment count unavailable comments

27 May 2015 6:38am GMT

26 May 2015

feedPlanet Debian

Lisandro Damin Nicanor Prez Meyer: The last planned Qt 4 release is here: Qt 4.8.7. Is your app runnning with Qt5?

Qt 4.8.7 has been released today. Quoting from the blog post (emphasis is mine):

Many users have already moved their active projects to Qt 5 and we encourage also others to do so. With a high degree of source compatibility, we have ensured that switching to Qt 5 is smooth and straightforward. It should be noted that Qt 4.8.7 provides only the basic functionality to run Qt based applications on Mac OS X 10.10, full support is in Qt 5.

Qt 4.8.7 is planned to be the last patch release of the Qt 4 series. Standard support is available until December 2015, after which extended support will be available. We recommend all active projects to migrate to Qt 5, as new operating systems and compilers with Qt 4.8 will not be supported. If you have challenges migrating to Qt 5, please contact us or some of our service partners for assistance


Have you started to port your project?

26 May 2015 4:36pm GMT

Lunar: Reproducible builds: week 4 in Stretch cycle

What happened about the reproducible builds effort for this week:

Toolchain fixes

Lunar rebased our custom dpkg on the new release, removing a now unneeded patch identified by Guillem Jover. An extra sort in the buildinfo generator prevented a stable order and was quickly fixed once identified.

Mattia Rizzolo also rebased our custom debhelper on the latest release.

Packages fixed

The following 30 packages became reproducible due to changes in their build dependencies: animal-sniffer, asciidoctor, autodock-vina, camping, cookie-monster, downthemall, flashblock, gamera, httpcomponents-core, https-finder, icedove-l10n, istack-commons, jdeb, libmodule-build-perl, libur-perl, livehttpheaders, maven-dependency-plugin, maven-ejb-plugin, mozilla-noscript, nosquint, requestpolicy, ruby-benchmark-ips, ruby-benchmark-suite, ruby-expression-parser, ruby-github-markup, ruby-http-connection, ruby-settingslogic, ruby-uuidtools, webkit2gtk, wot.

The following packages became reproducible after getting fixed:

Some uploads fixed some reproducibility issues but not all of them:

Patches submitted which did not make their way to the archive yet:

Also, the following bugs have been reported:

reproducible.debian.net

Holger Levsen made several small bug fixes and a few more visible changes:

strip-nondeterminism

Version 0.007-1 of strip-nondeterminism-the tool to post-process various file formats to normalize them-has been uploaded by Holger Levsen. Version 0.006-1 was already in the reproducible repository, the new version mainly improve the detection of Maven's pom.properties files.

debbindiff development

At the request of Emmanuel Bourg, Reiner Herrmann added a comparator for Java .class files.

Documentation update

Christoph Berg created a new page for the timestamps in manpages created by Doxygen.

Package reviews

93 obsolete reviews have been removed, 76 added and 43 updated this week.

New identified issues: timestamps in manpages generated by Doxygen, modification time differences in files extracted by unzip, tstamp task used in Ant build.xml, timestamps in documentation generated by ASDocGen. The description for build id related issues has been clarified.

Meetings

Holger Levsen announced a first meeting on Wednesday, June 3rd, 2015, 19:00 UTC. The agenda is amendable on the wiki.

Misc.

Lunar worked on a proof-of-concept script to import the build environment found in .buildinfo files to UDD. Lucas Nussbaum has positively reviewed the proposed schema.

Holger Levsen cleaned up various experimental toolchain repositories, marking merged brances as such.

26 May 2015 12:19pm GMT

Ricardo Mones: Downgrading to stable

This weekend I had to downgrade my home desktop to stable thanks to a strange Xorg bug which I've been unable to identify among the current ones. Both testing and sid versions seem affected and all you can see after booting is this:


The system works fine otherwise and can be accessed via ssh, but restarting kdm doesn't help to fix it, it just changes the pattern. Anyway, as explaining a toddler he cannot watch his favourite youtube cartoons because suddenly the computer screen has become an abstract art work is not easy I quickly decided to downgrade.

Downgrading went fine, using APT pinning to fix stable and apt-get update/upgrade/dist-upgrade after that, but today I noticed libreoffice stopped working with this message:

Warning: failed to launch javaldx - java may not function correctly
/usr/lib/libreoffice/program/soffice.bin: error while loading shared libraries: libreglo.so: cannot open shared object file: No such file or directory



All I found related to that is a post on forums, which didn't help much (neither the original poster nor me). But just found the library was not missing, it was installed:

# locate libreglo.so
/usr/lib/ure/lib/libreglo.so



But that was not part of any ldconfig conf file, hence the fix was easy:

# echo '/usr/lib/ure/lib' > /etc/ld.so.conf.d/libreoffice-ure.conf
# ldconfig



And presto! libreoffice is working again :-)

26 May 2015 10:11am GMT