Writing a book is a big undertaking. You have to think about what you will actually write, the content, its organization, the examples you want to show, illustrations, etc.

When publishing with the help of a regular editor, your job stops there at writing - and that's already a big and hard enough task. Your editor will handle the publishing process, leaving you free of the printing task. Though they might have their own set of requirements, such as making you work with a word processing tool (think LibreOffice Writer or Microsoft Word).

The Hacker's Guide to Python on my Kindle

When you self-publish like I did with The Hacker's Guide to Python, none of that happens. You have to deal yourself with getting your work out there, released and available in a viable format for your readership.

Most of the time, you need to render your book in different formats. You will have to make sure it works correctly on different devices and that the formatting and content disposition is correct.

I knew exactly what I wanted exactly when writing my book. I wanted to have the book published in at least PDF (for computer reading) and ePub (for e-readers). I also knew, as an Emacs user, that I did not want to spend hours writing a book in LibreOffice. It's not for me.

When I wrote about the making of The Hacker's Guide to Python, I briefly mentioned which tools I used to build the book and that I picked AsciiDoc as the input format. It makes it easy to write your book inside your favorite text editor, and AsciiDoc has plenty of output format. Customizing these formats to my liking and requirements was another challenge.

It took me hours and hours of work to have all the nitty-gritty details right. Today I am happy to announce that I can save you a few hours of work if you also want to publish a book.

I've published a new project on my GitHub called asciidoc-book-toolchain. It is the actual toolchain that I use to build The Hacker's Guide to Python. It should be easy to use and is able to render any book in HTML, PDF, PDF (printable 6"×9" format), ePub and MOBI.

Conversion workflow of the asciidoc-book-toolchain

So feel free to use it, hack it, pull-request it, or whatever. You don't have any good excuse to not write a book now! 😇 And if you want to self-publish a book and need some help getting started, let me know, I would be glad giving you a few hints!

20 Sep 2016 10:00am GMT

First a definition: a trackstick is also called trackpoint, pointing stick, or "that red knob between G, H, and B". I'll be using trackstick here, because why not.

This post is the continuation of libinput and the Lenovo T450 and T460 series touchpads where we focused on a stalling pointer when moving the finger really slowly. Turns out the T460s at least, possibly others in the *60 series have another bug that caused a behaviour that is much worse but we didn't notice for ages as we were focusing on the high-precision cursor movement. Specifically, the pointer would just randomly stop moving for a short while (spoiler alert: 300ms), regardless of the movement speed.

libinput has built-in palm detection and one of the things it does is to disable the touchpad when the trackstick is in use. It's not uncommon to rest the hand near or on the touchpad while using the trackstick and any detected touch would cause interference with the pointer motion. So events from the touchpad are ignored whenever the trackpoint sends events. [1]

On (some of) the T460s the trackpoint sends spurious events. In the recording I have we have random events at 9s, then again 3.5s later, then 14s later, then 2s later, etc. Each time, our palm detection could would assume the trackpoint was in use and disable the touchpad for 300ms. If you were using the touchpad while this was happening, the touchpad would suddenly stop moving for 300ms and then continue as normal. Depending on how often these spurious events come in and the user's current caffeination state, this was somewhere between odd, annoying and infuriating.

The good news is: this is fixed in libinput now. libinput 1.5 and the upcoming 1.4.3 releases will have a fix that ignores these spurious events and makes the touchpad stalls a footnote of history. Hooray.

[1] we still allow touchpad physical button presses, and trackpoint button clicks won't disable the touchpad

20 Sep 2016 6:43am GMT

This post explains how the evdev protocol works. After reading this post you should understand what evdev is and how to interpret evdev event dumps to understand what your device is doing. The post is aimed mainly at users having to debug a device, I will thus leave out or simplify some of the technical details. I'll be using the output from evemu-record as example because that is the primary debugging tool for evdev.

What is evdev?

evdev is a Linux-only generic protocol that the kernel uses to forward information and events about input devices to userspace. It's not just for mice and keyboards but any device that has any sort of axis, key or button, including things like webcams and remote controls. Each device is represented as a device node in the form of /dev/input/event0, with the trailing number increasing as you add more devices. The node numbers are re-used after you unplug a device, so don't hardcode the device node into a script. The device nodes are also only readable by root, thus you need to run any debugging tools as root too.

evdev is the primary way to talk to input devices on Linux. All X.Org drivers on Linux use evdev as protocol and libinput as well. Note that "evdev" is also the shortcut used for xf86-input-evdev, the X.Org driver to handle generic evdev devices, so watch out for context when you read "evdev" on a mailing list.

Communicating with evdev devices

Communicating with a device is simple: open the device node and read from it. Any data coming out is a struct input_event, defined in /usr/include/linux/input.h:

struct input_event {
 struct timeval time;
 __u16 type;
 __u16 code;
 __s32 value;
};

I'll describe the contents later, but you can see that it's a very simple struct.

Static information about the device such as its name and capabilities can be queried with a set of ioctls. Note that you should always use libevdevto interact with a device, it blunts the few sharp edges evdev has. See the libevdev documentation for usage examples.

evemu-record, our primary debugging tool for anything evdev is very simple. It reads the static information about the device, prints it and then simply reads and prints all events as they come in. The output is in machine-readable format but it's annotated with human-readable comments (starting with #). You can always ignore the non-comment bits. There's a second command, evemu-describe, that only prints the description and exits without waiting for events.

Relative devices and keyboards

The top part of an evemu-record output is the device description. This is a list of static properties that tells us what the device is capable of. For example, the USB mouse I have plugged in here prints:

# Input device name: "PIXART USB OPTICAL MOUSE"
# Input device ID: bus 0x03 vendor 0x93a product 0x2510 version 0x110
# Supported events:
#   Event type 0 (EV_SYN)
#     Event code 0 (SYN_REPORT)
#     Event code 1 (SYN_CONFIG)
#     Event code 2 (SYN_MT_REPORT)
#     Event code 3 (SYN_DROPPED)
#     Event code 4 ((null))
#     Event code 5 ((null))
#     Event code 6 ((null))
#     Event code 7 ((null))
#     Event code 8 ((null))
#     Event code 9 ((null))
#     Event code 10 ((null))
#     Event code 11 ((null))
#     Event code 12 ((null))
#     Event code 13 ((null))
#     Event code 14 ((null))
#   Event type 1 (EV_KEY)
#     Event code 272 (BTN_LEFT)
#     Event code 273 (BTN_RIGHT)
#     Event code 274 (BTN_MIDDLE)
#   Event type 2 (EV_REL)
#     Event code 0 (REL_X)
#     Event code 1 (REL_Y)
#     Event code 8 (REL_WHEEL)
#   Event type 4 (EV_MSC)
#     Event code 4 (MSC_SCAN)
# Properties:

The device name is the one (usually) set by the manufacturer and so are the vendor and product IDs. The bus is one of the "BUS_USB" and similar constants defined in /usr/include/linux/input.h. The version is often quite arbitrary, only a few devices have something meaningful here.

We also have a set of supported events, categorised by "event type" and "event code" (note how type and code are also part of the struct input_event). The type is a general category, and /usr/include/linux/input-event-codes.h defines quite a few of those. The most important types are EV_KEY (keys and buttons), EV_REL (relative axes) and EV_ABS (absolute axes). In the output above we can see that we have EV_KEY and EV_REL set.

As a subitem of each type we have the event code. The event codes for this device are self-explanatory: BTN_LEFT, BTN_RIGHT and BTN_MIDDLE are the left, right and middle button. The axes are a relative x axis, a relative y axis and a wheel axis (i.e. a mouse wheel). EV_MSC/MSC_SCAN is used for raw scancodes and you can usually ignore it. And finally we have the EV_SYN bits but let's ignore those, they are always set for all devices.

Note that an event code cannot be on its own, it must be a tuple of (type, code). For example, REL_X and ABS_X have the same numerical value and without the type you won't know which one is which.

That's pretty much it. A keyboard will have a lot of EV_KEY bits set and the EV_REL axes are obviously missing (but not always...). Instead of BTN_LEFT, a keyboard would have e.g. KEY_ESC, KEY_A, KEY_B, etc. 90% of device debugging is looking at the event codes and figuring out which ones are missing or shouldn't be there.

Exercise: You should now be able to read a evemu-record description from any mouse or keyboard device connected to your computer and understand what it means. This also applies to most special devices such as remotes - the only thing that changes are the names for the keys/buttons. Just run sudo evemu-describe and pick any device in the list.

The events from relative devices and keyboards

evdev is a serialised protocol. It sends a series of events and then a synchronisation event to notify us that the preceeding events all belong together. This synchronisation event is EV_SYN SYN_REPORT, is generated by the kernel, not the device and hence all EV_SYN codes are always available on all devices.

Let's have a look at a mouse movement. As explained above, half the line is machine-readable but we can ignore that bit and look at the human-readable output on the right.

E: 0.335996 0002 0000 0001      # EV_REL / REL_X                1
E: 0.335996 0002 0001 -002      # EV_REL / REL_Y                -2
E: 0.335996 0000 0000 0000      # ------------ SYN_REPORT (0) ----------

This means that within one hardware event, we've moved 1 device unit to the right (x axis) and two device units up (y axis). Note how all events have the same timestamp (0.335996).

Let's have a look at a button press:

E: 0.656004 0004 0004 589825    # EV_MSC / MSC_SCAN             589825
E: 0.656004 0001 0110 0001      # EV_KEY / BTN_LEFT             1
E: 0.656004 0000 0000 0000      # ------------ SYN_REPORT (0) ----------
E: 0.727002 0004 0004 589825    # EV_MSC / MSC_SCAN             589825
E: 0.727002 0001 0110 0000      # EV_KEY / BTN_LEFT             0
E: 0.727002 0000 0000 0000      # ------------ SYN_REPORT (0) ----------

For button events, the value 1 signals button pressed, button 0 signals button released.

And key events look like this:

E: 0.000000 0004 0004 458792    # EV_MSC / MSC_SCAN             458792
E: 0.000000 0001 001c 0000      # EV_KEY / KEY_ENTER            0
E: 0.000000 0000 0000 0000      # ------------ SYN_REPORT (0) ----------
E: 0.560004 0004 0004 458976    # EV_MSC / MSC_SCAN             458976
E: 0.560004 0001 001d 0001      # EV_KEY / KEY_LEFTCTRL         1
E: 0.560004 0000 0000 0000      # ------------ SYN_REPORT (0) ----------
[....]
E: 1.172732 0001 001d 0002      # EV_KEY / KEY_LEFTCTRL         2
E: 1.172732 0000 0000 0001      # ------------ SYN_REPORT (1) ----------
E: 1.200004 0004 0004 458758    # EV_MSC / MSC_SCAN             458758
E: 1.200004 0001 002e 0001      # EV_KEY / KEY_C                1
E: 1.200004 0000 0000 0000      # ------------ SYN_REPORT (0) ----------

Mostly the same as button events. But wait, there is one difference: we have a value of 2 as well. For key events, a value 2 means "key repeat". If you're on the tty, then this is what generates repeat keys for you. In X and Wayland we ignore these repeat events and instead use XKB-based key repeat.

Now look at the keyboard events again and see if you can make sense of the sequence. We have an Enter release (but no press), then ctrl down (and repeat), followed by a 'c' press - but no release. The explanation is simple - as soon as I hit enter in the terminal, evemu-record started recording so it captured the enter release too. And it stopped recording as soon as ctrl+c was down because that's when it was cancelled by the terminal. One important takeaway here: the evdev protocol is not guaranteed to be balanced. You may see a release for a key you've never seen the press for, and you may be missing a release for a key/button you've seen the press for (this happens when you stop recording). Oh, and there's one danger: if you record your keyboard and you type your password, the keys will show up in the output. Security experts generally reocmmend not publishing event logs with your password in it.

Exercise: You should now be able to read a evemu-record events list from any mouse or keyboard device connected to your computer and understand the event sequence.This also applies to most special devices such as remotes - the only thing that changes are the names for the keys/buttons. Just run sudo evemu-record and pick any device listed.

Absolute devices

Things get a bit more complicated when we look at absolute input devices like a touchscreen or a touchpad. Yes, touchpads are absolute devices in hardware and the conversion to relative events is done in userspace by e.g. libinput. The output of my touchpad is below. Note that I've manually removed a few bits to make it easier to grasp, they will appear later in the multitouch discussion.

# Input device name: "SynPS/2 Synaptics TouchPad"
# Input device ID: bus 0x11 vendor 0x02 product 0x07 version 0x1b1
# Supported events:
#   Event type 0 (EV_SYN)
#     Event code 0 (SYN_REPORT)
#     Event code 1 (SYN_CONFIG)
#     Event code 2 (SYN_MT_REPORT)
#     Event code 3 (SYN_DROPPED)
#     Event code 4 ((null))
#     Event code 5 ((null))
#     Event code 6 ((null))
#     Event code 7 ((null))
#     Event code 8 ((null))
#     Event code 9 ((null))
#     Event code 10 ((null))
#     Event code 11 ((null))
#     Event code 12 ((null))
#     Event code 13 ((null))
#     Event code 14 ((null))
#   Event type 1 (EV_KEY)
#     Event code 272 (BTN_LEFT)
#     Event code 325 (BTN_TOOL_FINGER)
#     Event code 328 (BTN_TOOL_QUINTTAP)
#     Event code 330 (BTN_TOUCH)
#     Event code 333 (BTN_TOOL_DOUBLETAP)
#     Event code 334 (BTN_TOOL_TRIPLETAP)
#     Event code 335 (BTN_TOOL_QUADTAP)
#   Event type 3 (EV_ABS)
#     Event code 0 (ABS_X)
#       Value   2919
#       Min     1024
#       Max     5112
#       Fuzz       0
#       Flat       0
#       Resolution 42
#     Event code 1 (ABS_Y)
#       Value   3711
#       Min     2024
#       Max     4832
#       Fuzz       0
#       Flat       0
#       Resolution 42
#     Event code 24 (ABS_PRESSURE)
#       Value      0
#       Min        0
#       Max      255
#       Fuzz       0
#       Flat       0
#       Resolution 0
#     Event code 28 (ABS_TOOL_WIDTH)
#       Value      0
#       Min        0
#       Max       15
#       Fuzz       0
#       Flat       0
#       Resolution 0
# Properties:
#   Property  type 0 (INPUT_PROP_POINTER)
#   Property  type 2 (INPUT_PROP_BUTTONPAD)
#   Property  type 4 (INPUT_PROP_TOPBUTTONPAD)

We have a BTN_LEFT again and a set of other buttons that I'll explain in a second. But first we look at the EV_ABS output. We have the same naming system as above. ABS_X and ABS_Y are the x and y axis on the device, ABS_PRESSURE is an (arbitrary) ranged pressure value.

Absolute axes have a bit more state than just a simple bit. Specifically, they have a minimum and maximum (not all hardware has the top-left sensor position on 0/0, it can be an arbitrary position, specified by the minimum). Notable here is that the axis ranges are simply the ones announced by the device - there is no guarantee that the values fall within this range and indeed a lot of touchpad devices tend to send values slightly outside that range. Fuzz and flat can be safely ignored, but resolution is interesting. It is given in units per millimeter and thus tells us the size of the device. in the above case: (5112 - 1024)/42 means the device is 97mm wide. The resolution is quite commonly wrong, a lot of axis overrides need the resolution changed to the correct value.

The axis description also has a current value listed. The kernel only sends events when the value changes, so even if the actual hardware keeps sending events, you may never see them in the output if the value remains the same. In other words, holding a finger perfectly still on a touchpad creates plenty of hardware events, but you won't see anything coming out of the event node.

Finally, we have properties on this device. These are used to indicate general information about the device that's not otherwise obvious. In this case INPUT_PROP_POINTER tells us that we need a pointer for this device (it is a touchpad after all, a touchscreen would instead have INPUT_PROP_DIRECT set). INPUT_PROP_BUTTONPAD means that this is a so-called clickpad, it does not have separate physical buttons but instead the whole touchpad clicks. Ignore INPUT_PROP_TOPBUTTONPAD because it only applies to the Lenovo *40 series of devices.

Ok, back to the buttons: aside from BTN_LEFT, we have BTN_TOUCH. This one signals that the user is touching the surface of the touchpad (with some in-kernel defined minimum pressure value). It's not just for finger-touches, it's also used for graphics tablet stylus touchpes (so really, it's more "contact" than "touch" but meh).

The BTN_TOOL_FINGER event tells us that a finger is in detectable range. This gives us two bits of information: first, we have a finger (a tablet would have e.g. BTN_TOOL_PEN) and second, we may have a finger in proximity without touching. On many touchpads, BTN_TOOL_FINGER and BTN_TOUCH come in the same event, but others can detect a finger hovering over the touchpad too (in which case you'd also hope for ABS_DISTANCE being available on the touchpad).

Finally, the BTN_TOOL_DOUBLETAP up to BTN_TOOL_QUINTTAP tell us whether the device can detect 2 through to 5 fingers on the touchpad. This doesn't actually track the fingers, it merely tells you "3 fingers down" in the case of BTN_TOOL_TRIPLETAP.

Exercise: Look at your touchpad's description and figure out if the size of the touchpad is correct based on the axis information [1]. Check how many fingers your touchpad can detect and whether it can do pressure or distance detection.

The events from absolute devices

Events from absolute axes are not really any different than events from relative devices which we already covered. The same type/code combination with a value and a timestamp, all framed by EV_SYN SYN_REPORT events. Here's an example of me touching the touchpad:

E: 0.000001 0001 014a 0001      # EV_KEY / BTN_TOUCH            1
E: 0.000001 0003 0000 3335      # EV_ABS / ABS_X                3335
E: 0.000001 0003 0001 3308      # EV_ABS / ABS_Y                3308
E: 0.000001 0003 0018 0069      # EV_ABS / ABS_PRESSURE         69
E: 0.000001 0001 0145 0001      # EV_KEY / BTN_TOOL_FINGER      1
E: 0.000001 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +0ms
E: 0.021751 0003 0018 0070      # EV_ABS / ABS_PRESSURE         70
E: 0.021751 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +21ms
E: 0.043908 0003 0000 3334      # EV_ABS / ABS_X                3334
E: 0.043908 0003 0001 3309      # EV_ABS / ABS_Y                3309
E: 0.043908 0003 0018 0065      # EV_ABS / ABS_PRESSURE         65
E: 0.043908 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +22ms
E: 0.052469 0001 014a 0000      # EV_KEY / BTN_TOUCH            0
E: 0.052469 0003 0018 0000      # EV_ABS / ABS_PRESSURE         0
E: 0.052469 0001 0145 0000      # EV_KEY / BTN_TOOL_FINGER      0
E: 0.052469 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +9ms

In the first event you see BTN_TOOL_FINGER and BTN_TOUCH set (this touchpad doesn't detect hovering fingers). An x/y coordinate pair and a pressure value. The pressure changes in the second event, the third event changes pressure and location. Finally, we have BTN_TOOL_FINGER and BTN_TOUCH released on finger up, and the pressure value goes back to 0. Notice how the second event didn't contain any x/y coordinates? As I said above, the kernel only sends updates on absolute axes when the value changed.

Ok, let's look at a three-finger tap (again, minus the ABS_MT_ bits):

E: 0.000001 0001 014a 0001      # EV_KEY / BTN_TOUCH            1
E: 0.000001 0003 0000 2149      # EV_ABS / ABS_X                2149
E: 0.000001 0003 0001 3747      # EV_ABS / ABS_Y                3747
E: 0.000001 0003 0018 0066      # EV_ABS / ABS_PRESSURE         66
E: 0.000001 0001 014e 0001      # EV_KEY / BTN_TOOL_TRIPLETAP   1
E: 0.000001 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +0ms
E: 0.034209 0003 0000 2148      # EV_ABS / ABS_X                2148
E: 0.034209 0003 0018 0064      # EV_ABS / ABS_PRESSURE         64
E: 0.034209 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +34ms
[...]
E: 0.138510 0003 0000 4286      # EV_ABS / ABS_X                4286
E: 0.138510 0003 0001 3350      # EV_ABS / ABS_Y                3350
E: 0.138510 0003 0018 0055      # EV_ABS / ABS_PRESSURE         55
E: 0.138510 0001 0145 0001      # EV_KEY / BTN_TOOL_FINGER      1
E: 0.138510 0001 014e 0000      # EV_KEY / BTN_TOOL_TRIPLETAP   0
E: 0.138510 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +23ms
E: 0.147834 0003 0000 4287      # EV_ABS / ABS_X                4287
E: 0.147834 0003 0001 3351      # EV_ABS / ABS_Y                3351
E: 0.147834 0003 0018 0037      # EV_ABS / ABS_PRESSURE         37
E: 0.147834 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +9ms
E: 0.157151 0001 014a 0000      # EV_KEY / BTN_TOUCH            0
E: 0.157151 0003 0018 0000      # EV_ABS / ABS_PRESSURE         0
E: 0.157151 0001 0145 0000      # EV_KEY / BTN_TOOL_FINGER      0
E: 0.157151 0000 0000 0000      # ------------ SYN_REPORT (0) ---------- +10ms

In the first event, the touchpad detected all three fingers at the same time. So get BTN_TOUCH, x/y/pressure and BTN_TOOL_TRIPLETAP set. Note that the various BTN_TOOL_* bits are mutually exclusive. BTN_TOOL_FINGER means "exactly 1 finger down" and you can't have exactly 1 finger down when you have three fingers down. In the second event x and pressure update (y has no event, it stayed the same).

In the event after the break, we switch from three fingers to one finger. BTN_TOOL_TRIPLETAP is released, BTN_TOOL_FINGER is set. That's very common. Humans aren't robots, you can't release all fingers at exactly the same time, so depending on the hardware scanout rate you have intermediate states where one finger has left already, others are still down. In this case I released two fingers between scanouts, one was still down. It's not uncommon to see a full cycle from BTN_TOOL_FINGER to BTN_TOOL_DOUBLETAP to BTN_TOOL_TRIPLETAP on finger down or the reverse on finger up.

Exercise: test out the pressure values on your touchpad and see how close you can get to the actual announced range. Check how accurate the multifinger detection is by tapping with two, three, four and five fingers. (In both cases, you'll likely find that it's very much hit and miss).

Multitouch and slots

Now we're at the most complicated topic regarding evdev devices. In the case of multitouch devices, we need to send multiple touches on the same axes. So we need an additional dimension and that is called multitouch slots (there is another, older multitouch protocol that doesn't use slots but it is so rare now that you don't need to bother).

First: all axes that are multitouch-capable are repeated as ABS_MT_foo axis. So if you have ABS_X, you also get ABS_MT_POSITION_X and both axes have the same axis ranges and resolutions. The reason here is backwards-compatibility: if a device only sends multitouch events, older programs only listening to the ABS_X etc. events won't work. Some axes may only be available for single-touch (ABS_MT_TOOL_WIDTH in this case).

Let's have a look at my touchpad, this time without the axes removed:

# Input device name: "SynPS/2 Synaptics TouchPad"
# Input device ID: bus 0x11 vendor 0x02 product 0x07 version 0x1b1
# Supported events:
#   Event type 0 (EV_SYN)
#     Event code 0 (SYN_REPORT)
#     Event code 1 (SYN_CONFIG)
#     Event code 2 (SYN_MT_REPORT)
#     Event code 3 (SYN_DROPPED)
#     Event code 4 ((null))
#     Event code 5 ((null))
#     Event code 6 ((null))
#     Event code 7 ((null))
#     Event code 8 ((null))
#     Event code 9 ((null))
#     Event code 10 ((null))
#     Event code 11 ((null))
#     Event code 12 ((null))
#     Event code 13 ((null))
#     Event code 14 ((null))
#   Event type 1 (EV_KEY)
#     Event code 272 (BTN_LEFT)
#     Event code 325 (BTN_TOOL_FINGER)
#     Event code 328 (BTN_TOOL_QUINTTAP)
#     Event code 330 (BTN_TOUCH)
#     Event code 333 (BTN_TOOL_DOUBLETAP)
#     Event code 334 (BTN_TOOL_TRIPLETAP)
#     Event code 335 (BTN_TOOL_QUADTAP)
#   Event type 3 (EV_ABS)
#     Event code 0 (ABS_X)
#       Value   5112
#       Min     1024
#       Max     5112
#       Fuzz       0
#       Flat       0
#       Resolution 41
#     Event code 1 (ABS_Y)
#       Value   2930
#       Min     2024
#       Max     4832
#       Fuzz       0
#       Flat       0
#       Resolution 37
#     Event code 24 (ABS_PRESSURE)
#       Value      0
#       Min        0
#       Max      255
#       Fuzz       0
#       Flat       0
#       Resolution 0
#     Event code 28 (ABS_TOOL_WIDTH)
#       Value      0
#       Min        0
#       Max       15
#       Fuzz       0
#       Flat       0
#       Resolution 0
#     Event code 47 (ABS_MT_SLOT)
#       Value      0
#       Min        0
#       Max        1
#       Fuzz       0
#       Flat       0
#       Resolution 0
#     Event code 53 (ABS_MT_POSITION_X)
#       Value      0
#       Min     1024
#       Max     5112
#       Fuzz       8
#       Flat       0
#       Resolution 41
#     Event code 54 (ABS_MT_POSITION_Y)
#       Value      0
#       Min     2024
#       Max     4832
#       Fuzz       8
#       Flat       0
#       Resolution 37
#     Event code 57 (ABS_MT_TRACKING_ID)
#       Value      0
#       Min        0
#       Max    65535
#       Fuzz       0
#       Flat       0
#       Resolution 0
#     Event code 58 (ABS_MT_PRESSURE)
#       Value      0
#       Min        0
#       Max      255
#       Fuzz       0
#       Flat       0
#       Resolution 0
# Properties:
#   Property  type 0 (INPUT_PROP_POINTER)
#   Property  type 2 (INPUT_PROP_BUTTONPAD)
#   Property  type 4 (INPUT_PROP_TOPBUTTONPAD)

We have an x and y position for multitouch as well as a pressure axis. There are also two special multitouch axes that aren't really axes. ABS_MT_SLOT and ABS_MT_TRACKING_ID. The former specifies which slot is currently active, the latter is used to track touch points.

Slots are a static property of a device. My touchpad, as you can see above ony supports 2 slots (min 0, max 1) and thus can track 2 fingers at a time. Whenever the first finger is set down it's coordinates will be tracked in slot 0, the second finger will be tracked in slot 1. When the finger in slot 0 is lifted, the second finger continues to be tracked in slot 1, and if a new finger is set down, it will be tracked in slot 0. Sounds more complicated than it is, think of it as an array of possible touchpoints.

The tracking ID is an incrementing number that lets us tell touch points apart and also tells us when a touch starts and when it ends. The two values are either -1 or a positive number. Any positive number means "new touch" and -1 means "touch ended". So when you put two fingers down and lift them again, you'll get a tracking ID of 1 in slot 0, a tracking ID of 2 in slot 1, then a tracking ID of -1 in both slots to signal they ended. The tracking ID value itself is meaningless, it simply increases as touches are created.

Let's look at a single tap:

E: 0.000001 0003 0039 0387 # EV_ABS / ABS_MT_TRACKING_ID   387
E: 0.000001 0003 0035 2560 # EV_ABS / ABS_MT_POSITION_X    2560
E: 0.000001 0003 0036 2905 # EV_ABS / ABS_MT_POSITION_Y    2905
E: 0.000001 0003 003a 0059 # EV_ABS / ABS_MT_PRESSURE      59
E: 0.000001 0001 014a 0001 # EV_KEY / BTN_TOUCH            1
E: 0.000001 0003 0000 2560 # EV_ABS / ABS_X                2560
E: 0.000001 0003 0001 2905 # EV_ABS / ABS_Y                2905
E: 0.000001 0003 0018 0059 # EV_ABS / ABS_PRESSURE         59
E: 0.000001 0001 0145 0001 # EV_KEY / BTN_TOOL_FINGER      1
E: 0.000001 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +0ms
E: 0.021690 0003 003a 0067 # EV_ABS / ABS_MT_PRESSURE      67
E: 0.021690 0003 0018 0067 # EV_ABS / ABS_PRESSURE         67
E: 0.021690 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +21ms
E: 0.033482 0003 003a 0068 # EV_ABS / ABS_MT_PRESSURE      68
E: 0.033482 0003 0018 0068 # EV_ABS / ABS_PRESSURE         68
E: 0.033482 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +12ms
E: 0.044268 0003 0035 2561 # EV_ABS / ABS_MT_POSITION_X    2561
E: 0.044268 0003 0000 2561 # EV_ABS / ABS_X                2561
E: 0.044268 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +11ms
E: 0.054093 0003 0035 2562 # EV_ABS / ABS_MT_POSITION_X    2562
E: 0.054093 0003 003a 0067 # EV_ABS / ABS_MT_PRESSURE      67
E: 0.054093 0003 0000 2562 # EV_ABS / ABS_X                2562
E: 0.054093 0003 0018 0067 # EV_ABS / ABS_PRESSURE         67
E: 0.054093 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +10ms
E: 0.064891 0003 0035 2569 # EV_ABS / ABS_MT_POSITION_X    2569
E: 0.064891 0003 0036 2903 # EV_ABS / ABS_MT_POSITION_Y    2903
E: 0.064891 0003 003a 0059 # EV_ABS / ABS_MT_PRESSURE      59
E: 0.064891 0003 0000 2569 # EV_ABS / ABS_X                2569
E: 0.064891 0003 0001 2903 # EV_ABS / ABS_Y                2903
E: 0.064891 0003 0018 0059 # EV_ABS / ABS_PRESSURE         59
E: 0.064891 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +10ms
E: 0.073634 0003 0039 -001 # EV_ABS / ABS_MT_TRACKING_ID   -1
E: 0.073634 0001 014a 0000 # EV_KEY / BTN_TOUCH            0
E: 0.073634 0003 0018 0000 # EV_ABS / ABS_PRESSURE         0
E: 0.073634 0001 0145 0000 # EV_KEY / BTN_TOOL_FINGER      0
E: 0.073634 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +9ms

We have a tracking ID (387) signalling finger down, as well as a position plus pressure. then some updates and eventually a tracking ID of -1 (signalling finger up). Notice how there is no ABS_MT_SLOT here - the kernel buffers those too so while you stay in the same slot (0 in this case) you don't see any events for it. Also notice how you get both single-finger as well as multitouch in the same event stream. This is for backwards compatibility [2]

Ok, time for a two-finger tap:

E: 0.000001 0003 0039 0496 # EV_ABS / ABS_MT_TRACKING_ID   496
E: 0.000001 0003 0035 2609 # EV_ABS / ABS_MT_POSITION_X    2609
E: 0.000001 0003 0036 3791 # EV_ABS / ABS_MT_POSITION_Y    3791
E: 0.000001 0003 003a 0054 # EV_ABS / ABS_MT_PRESSURE      54
E: 0.000001 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.000001 0003 0039 0497 # EV_ABS / ABS_MT_TRACKING_ID   497
E: 0.000001 0003 0035 3012 # EV_ABS / ABS_MT_POSITION_X    3012
E: 0.000001 0003 0036 3088 # EV_ABS / ABS_MT_POSITION_Y    3088
E: 0.000001 0003 003a 0056 # EV_ABS / ABS_MT_PRESSURE      56
E: 0.000001 0001 014a 0001 # EV_KEY / BTN_TOUCH            1
E: 0.000001 0003 0000 2609 # EV_ABS / ABS_X                2609
E: 0.000001 0003 0001 3791 # EV_ABS / ABS_Y                3791
E: 0.000001 0003 0018 0054 # EV_ABS / ABS_PRESSURE         54
E: 0.000001 0001 014d 0001 # EV_KEY / BTN_TOOL_DOUBLETAP   1
E: 0.000001 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +0ms
E: 0.012909 0003 002f 0000 # EV_ABS / ABS_MT_SLOT          0
E: 0.012909 0003 0039 -001 # EV_ABS / ABS_MT_TRACKING_ID   -1
E: 0.012909 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.012909 0003 0039 -001 # EV_ABS / ABS_MT_TRACKING_ID   -1
E: 0.012909 0001 014a 0000 # EV_KEY / BTN_TOUCH            0
E: 0.012909 0003 0018 0000 # EV_ABS / ABS_PRESSURE         0
E: 0.012909 0001 014d 0000 # EV_KEY / BTN_TOOL_DOUBLETAP   0
E: 0.012909 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +12ms

This was a really quick two-finger tap that illustrates the tracking IDs nicely. In the first event we get a touch down, then an ABS_MT_SLOT event. This tells us that subsequent events belong to the other slot, so it's the other finger. There too we get a tracking ID + position. In the next event we get an ABS_MT_SLOT to switch back to slot 0. Tracking ID of -1 means that touch ended, and then we see the touch in slot 1 ended too.

Time for a two-finger scroll:

E: 0.000001 0003 0039 0557 # EV_ABS / ABS_MT_TRACKING_ID   557
E: 0.000001 0003 0035 2589 # EV_ABS / ABS_MT_POSITION_X    2589
E: 0.000001 0003 0036 3363 # EV_ABS / ABS_MT_POSITION_Y    3363
E: 0.000001 0003 003a 0048 # EV_ABS / ABS_MT_PRESSURE      48
E: 0.000001 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.000001 0003 0039 0558 # EV_ABS / ABS_MT_TRACKING_ID   558
E: 0.000001 0003 0035 3512 # EV_ABS / ABS_MT_POSITION_X    3512
E: 0.000001 0003 0036 3028 # EV_ABS / ABS_MT_POSITION_Y    3028
E: 0.000001 0003 003a 0044 # EV_ABS / ABS_MT_PRESSURE      44
E: 0.000001 0001 014a 0001 # EV_KEY / BTN_TOUCH            1
E: 0.000001 0003 0000 2589 # EV_ABS / ABS_X                2589
E: 0.000001 0003 0001 3363 # EV_ABS / ABS_Y                3363
E: 0.000001 0003 0018 0048 # EV_ABS / ABS_PRESSURE         48
E: 0.000001 0001 014d 0001 # EV_KEY / BTN_TOOL_DOUBLETAP   1
E: 0.000001 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +0ms
E: 0.027960 0003 002f 0000 # EV_ABS / ABS_MT_SLOT          0
E: 0.027960 0003 0035 2590 # EV_ABS / ABS_MT_POSITION_X    2590
E: 0.027960 0003 0036 3395 # EV_ABS / ABS_MT_POSITION_Y    3395
E: 0.027960 0003 003a 0046 # EV_ABS / ABS_MT_PRESSURE      46
E: 0.027960 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.027960 0003 0035 3511 # EV_ABS / ABS_MT_POSITION_X    3511
E: 0.027960 0003 0036 3052 # EV_ABS / ABS_MT_POSITION_Y    3052
E: 0.027960 0003 0000 2590 # EV_ABS / ABS_X                2590
E: 0.027960 0003 0001 3395 # EV_ABS / ABS_Y                3395
E: 0.027960 0003 0018 0046 # EV_ABS / ABS_PRESSURE         46
E: 0.027960 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +27ms
E: 0.051720 0003 002f 0000 # EV_ABS / ABS_MT_SLOT          0
E: 0.051720 0003 0035 2609 # EV_ABS / ABS_MT_POSITION_X    2609
E: 0.051720 0003 0036 3447 # EV_ABS / ABS_MT_POSITION_Y    3447
E: 0.051720 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.051720 0003 0036 3080 # EV_ABS / ABS_MT_POSITION_Y    3080
E: 0.051720 0003 0000 2609 # EV_ABS / ABS_X                2609
E: 0.051720 0003 0001 3447 # EV_ABS / ABS_Y                3447
E: 0.051720 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +24ms
[...]
E: 0.272034 0003 002f 0000 # EV_ABS / ABS_MT_SLOT          0
E: 0.272034 0003 0039 -001 # EV_ABS / ABS_MT_TRACKING_ID   -1
E: 0.272034 0003 002f 0001 # EV_ABS / ABS_MT_SLOT          1
E: 0.272034 0003 0039 -001 # EV_ABS / ABS_MT_TRACKING_ID   -1
E: 0.272034 0001 014a 0000 # EV_KEY / BTN_TOUCH            0
E: 0.272034 0003 0018 0000 # EV_ABS / ABS_PRESSURE         0
E: 0.272034 0001 014d 0000 # EV_KEY / BTN_TOOL_DOUBLETAP   0
E: 0.272034 0000 0000 0000 # ------------ SYN_REPORT (0) ---------- +30ms

Note that "scroll" is something handled in userspace, so what you see here is just a two-finger move. Everything in there i something we've already seen, but pay attention to the two middle events: as updates come in for each finger, the ABS_MT_SLOT changes before the upates are sent. The kernel filter for identical events is still in effect, so in the third event we don't get an update for the X position on slot 1. The filtering is per-touchpoint, so in this case this means that slot 1 position x is still on 3511, just as it was in the previous event.

That's all you have to remember, really. If you think of evdev as a serialised way of sending an array of touchpoints, with the slots as the indices then it should be fairly clear. The rest is then just about actually looking at the touch positions and making sense of them.

Exercise: do a pinch gesture on your touchpad. See if you can track the two fingers moving closer together. Then do the same but only move one finger. See how the non-moving finger gets less updates.

That's it. There are a few more details to evdev but much of that is just more event types and codes. The few details you really have to worry about when processing events are either documented in libevdev or abstracted away completely. The above should be enough to understand what your device does, and what goes wrong when your device isn't working. Good luck.

[1] If not, file a bug against systemd's hwdb and CC me so we can put corrections in
[2] We treat some MT-capable touchpads as single-touch devices in libinput because the MT data is garbage

19 Sep 2016 1:33pm GMT

While stuck on an airplane, I put together a repository for apitraces with confirmed good images for driver output. Combined with the piglit patches I pushed, I now have regression testing of actual apps on vc4 (particularly relevant given that I'm working on optimizing one of those apps!)

The flight was to visit the Raspberry Pi Foundation, with the goal of getting something usable for their distro to switch to the open 3D stack. There's still a giant pile of KMS work to do (HDMI audio, DSI power management, SDTV support, etc.), and waiting for all of that to be regression-free will be a long time. The question is: what could we do that would get us 3D, even if KMS isn't ready?

So, I put together a quick branch to expose the firmware's display stack as the KMS display pipeline. It's a filthy hack, and loses us a lot of the important new features that the open stack was going to bring (changing video modes in X, vblank timestamping, power management), but it gets us much closer to the featureset of the previous stack. Hopefully they'll be switching to it as the default in new installs soon.

In debugging while I was here, Simon found that on his HDMI display the color ramps didn't quite match between closed and open drivers. After a bit of worrying about gamma ramp behavior, I figured out that it was actually that the monitor was using a CEA mode that requires limited range RGB input. A patch is now on the list.

19 Sep 2016 11:50am GMT

Tickets for systemd 2016 Workshop day still available!

We still have a number of ticket for the workshop day of systemd.conf 2016 available. If you are a newcomer to systemd, and would like to learn about various systemd facilities, or if you already know your way around, but would like to know more: this is the best chance to do so. The workshop day is the 28th of September, one day before the main conference, at the betahaus in Berlin, Germany. The schedule for the day is available here. There are five interesting, extensive sessions, run by the systemd hackers themselves. Who better to learn systemd from, than the folks who wrote it?

Note that the workshop day and the main conference days require different tickets. (Also note: there are still a few tickets available for the main conference!).

Buy a ticket here.

See you in Berlin!

17 Sep 2016 10:00pm GMT

libinput's touchpad acceleration is the cause for a few bugs and outcry from a quite vocal (maj|in)ority. A common suggestion is "make it like the synaptics driver". So I spent a few hours going through the pointer acceleration code to figure out what xf86-input-synaptics actually does (I don't think anyone knows at this point) [1].

If you just want the TLDR: synaptics doesn't use physical distances but works in device units coupled with a few magic factors, also based on device units. That pretty much tells you all that's needed.

Also a disclaimer: the last time some serious work was done on acceleration was in 2008/2009. A lot of things have changed since and since the server is effectively un-testable, we ended up with the mess below that seems to make little sense. It probably made sense 8 years ago and given that most or all of the patches have my signed-off-by it must've made sense to me back then. But now we live in the glorious future and holy cow it's awful and confusing.

Synaptics has three options to configure speed: MinSpeed, MaxSpeed and AccelFactor. The first two are not explained beyond "speed factor" but given how accel usually works let's assume they all somewhoe should work as a multiplication on the delta (so a factor of 2 on a delta of dx/dy gives you 2dx/2dy). AccelFactor is documented as "acceleration factor for normal pointer movements", so clearly the documentation isn't going to help clear any confusion.

I'll skip the fact that synaptics also has a pressure-based motion factor with four configuration options because oh my god what have we done. Also, that one is disabled by default and has no effect unless set by the user. And I'll also only handle default values here, I'm not going to get into examples with configured values.

Also note: synaptics has a device-specific acceleration profile (the only driver that does) and thus the acceleration handling is split between the server and the driver.

Ok, let's get started. MinSpeed and MaxSpeed default to 0.4 and 0.7. The MinSpeed is used to set constant acceleration (1/min_speed) so we always apply a 2.5 constant acceleration multiplier to deltas from the touchpad. Of course, if you set constant acceleration in the xorg.conf, then it overwrites the calculated one.

MinSpeed and MaxSpeed are mangled during setup so that MaxSpeed is actually MaxSpeed/MinSpeed and MinSpeed is always 1.0. I'm not 100% why but the later clipping to the min/max speed range ensures that we never go below a 1.0 acceleration factor (and thus never decelerate).

The AccelFactor default is 200/diagonal-in-device-coordinates. On my T440s it's thus 0.04 (and will be roughly the same for most PS/2 Synaptics touchpads). But on a Cyapa with a different axis range it is 0.125. On a T450s it's 0.035 when booted into PS2 and 0.09 when booted into RMI4. Admittedly, the resolution halfs under RMI4 so this possibly maybe makes sense. Doesn't quite make as much sense when you consider the x220t which also has a factor of 0.04 but the touchpad is only half the size of the T440s.

There's also a magic constant "corr_mul" which is set as:

/* synaptics seems to report 80 packet/s, but dix scales for
 * 100 packet/s by default. */
pVel->corr_mul = 12.5f; /*1000[ms]/80[/s] = 12.5 */

It's correct that the frequency is roughly 80Hz but I honestly don't know what the 100packet/s reference refers to. Either way, it means that we always apply a factor of 12.5, regardless of the timing of the events. Ironically, this one is hardcoded and not configurable unless you happen to know that it's the X server option VelocityScale or ExpectedRate (both of them set the same variable).

Ok, so we have three factors. 2.5 as a function of MaxSpeed, 12.5 because of 80Hz (??) and 0.04 for the diagonal.

When the synaptics driver calculates a delta, it does so in device coordinates and ignores the device resolution (because this code pre-dates devices having resolutions). That's great until you have a device with uneven resolutions like the x220t. That one has 75 and 129 units/mm for x and y, so for any physical movement you're going to get almost twice as many units for y than for x. Which means that if you move 5mm to the right you end up with a different motion vector (and thus acceleration) than when you move 5mm south.

The core X protocol actually defines who acceleration is supposed to be handled. Look up the man page for XChangePointerControl(), it sets a threshold and an accel factor:

The XChangePointerControl function defines how the pointing device moves. The acceleration, expressed as a fraction, is a multiplier for movement. For example, specifying 3/1 means the pointer moves three times as fast as normal. The fraction may be rounded arbitrarily by the X server. Acceleration only takes effect if the pointer moves more than threshold pixels at once and only applies to the amount beyond the value in the threshold argument.

Of course, "at once" is a bit of a blurry definition outside of maybe theoretical physics. Consider the definition of "at once" for a gaming mouse with 500Hz sampling rate vs. a touchpad with 80Hz (let us fondly remember the 12.5 multiplier here) and the above description quickly dissolves into ambiguity.

Anyway, moving on. Let's say the server just received a delta from the synaptics driver. The pointer accel code in the server calculates the velocity over time, basically by doing a hypot(dx, dy)/dtime-to-last-event. Time in the server is always in ms, so our velocity is thus in device-units/ms (not adjusted for device resolution).

Side-note: the velocity is calculated across several delta events so it gets more accurate. There are some checks though so we don't calculate across random movements: anything older than 300ms is discarded, anything not in the same octant of movement is discarded (so we don't get a velocity of 0 for moving back/forth). And there's two calculations to make sure we only calculate while the velocity is roughly the same and don't average between fast and slow movements. I have my doubts about these, but until I have some more concrete data let's just say this is accurate (altough since the whole lot is in device units, it probably isn't).

Anyway. The velocity is multiplied with the constant acceleration (2.5, see above) and our 12.5 magic value. I'm starting to think that this is just broken and would only make sense if we used a delta of "event count" rather than milliseconds.

It is then passed to the synaptics driver for the actual acceleration profile. The first thing the driver does is remove the constant acceleration again, so our velocity is now just v * 12.5. According to the comment this brings it back into "device-coordinate based velocity" but this seems wrong or misguided since we never changed into any other coordinate system.

The driver applies the accel factor (0.04, see above) and then clips the whole lot into the MinSpeed/MaxSpeed range (which is adjusted to move MinSpeed to 1.0 and scale up MaxSpeed accordingly, remember?). After the clipping, the pressure motion factor is calculated and applied. I skipped this above but it's basically: the harder you press the higher the acceleration factor. Based on some config options. Amusingly, pressure motion has the potential to exceed the MinSpeed/MaxSpeed options. Who knows what the reason for that is...

Oh, and btw: the clipping is actually done based on the accel factor set by XChangePointerControl into the acceleration function here. The code is

double acc = factor from XChangePointerControl();
double factor = the magic 0.04 based on the diagonal;

accel_factor = velocity * accel_factor;
if (accel_factor > MaxSpeed * acc)
    accel_factor = MaxSpeed * acc;

So we have a factor set by XChangePointerControl() but it's only used to determine the maximum factor we may have, and then we clip to that. I'm missing some cross-dependency here because this is what the GUI acceleration config bits hook into. Somewhere this sets things and changes the acceleration by some amount but it wasn't obvious to me.

Alrighty. We have a factor now that's returned to the server and we're back in normal pointer acceleration land (i.e. not synaptics-specific). Woohoo. That factor is averaged across 4 events using the simpson's rule to smooth out aprupt changes. Not sure this really does much, I don't think we've ever done any evaluation on that. But it looks good on paper (we have that in libinput as well).

Now the constant accel factor is applied to the deltas. So far we've added the factor, removed it (in synaptics), and now we're adding it again. Which also makes me wonder whether we're applying the factor twice to all other devices but right now I'm past the point where I really want to find out . With all the above, our acceleration factor is, more or less:

        f = units/ms * 12.5 * (200/diagonal) * (1.0/MinSpeed)

and the deltas we end up using in the server are

        (dx, dy) = f * (dx, dy)

But remember, we're still in device units here (not adjusted for resolution).

Anyway. You think we're finished? Oh no, the real fun bits start now. And if you haven't headdesked in a while, now is a good time.

After acceleration, the server does some scaling because synaptics is an absolute device (with axis ranges) in relative mode [2]. Absolute devices are mapped into the whole screen by default but when they're sending relative events, you still want a 45 degree line on the device to map into 45 degree cursor movement on the screen. The server does this by adjusting dy in-line with the device-to-screen-ratio (taking device resolution into account too). On my T440s this means:

    touchpad x:y is 1:1.45 (16:11)
    screen is 1920:1080 is 1:177 (16:9)

    dy scaling is thus: (16:11)/(16:9) = 9:11 -> y * 11/9

dx is left as-is. Now you have the delta that's actually applied to the cursor. Except that we're in device coordinates, so we map the current cursor position to device coordinates, then apply the delta, then map back into screen coordinates (i.e. pixels). You may have spotted the flaw here: when the screen size changes, the dy scaling changes and thus the pointer feel. Plug in another monitor, and touchpad acceleration changes. Also: the same touchpad feels different on laptops when their screen hardware differs.

Ok, let's wrap this up. Figuring out what the synaptics driver does is... "tricky". It seems much like a glorified random number scheme. I'm not planning to implement "exactly the same acceleration as synaptics" in libinput because this would be insane and despite my best efforts, I'm not that yet. Collecting data from synaptics users is almost meaningless, because no two devices really employ the same acceleration profile (touchpad axis ranges + screen size) and besides, there are 11 configuration options that all influence each other.

What I do plan though is collect more motion data from a variety of touchpads and see if I can augment the server enough that I can get a clear picture of how motion maps to the velocity. If nothing else, this should give us some picture on how different the various touchpads actually behave.

But regardless, please don't ask me to "just copy the synaptics code".

[1] fwiw, I had this really great idea of trying to get behind all this, with diagrams and everything. But then I was printing json data from the X server into the journal to be scooped up by sed and python script to print velocity data. And I questioned some of my life choices.
[2] why the hell do we do this? because synaptics at some point became a device that announce the axis ranges (seemed to make sense at the time, 2008) and then other things started depending on it and with all the fixes to the server to handle absolute devices in relative mode (for tablets) we painted ourselves into a corner. Synaptics should switch back to being a relative device, but last I tried it breaks pointer acceleration and that a) makes the internets upset and b) restoring the "correct" behaviour is, well, you read the article so far, right?

16 Sep 2016 6:00am GMT

Last week I spent working on the glmark2 performance issues. I now have a NIR patch out for the pathological conditionals test (it's now faster than on the old driver), and a branch for job shuffling (+17% and +27% on the two desktop tests).

Here's the basic idea of job shuffling:

We're a tiled renderer, and tiled renderers get their wins from having a Clear at the start of the frame (indicating we don't need to load any previous contents into the tile buffer). When your frame is done, we flush each tile out to memory. If you do your clear, start rendering some primitives, and then switch to some other FBO (because you're rendering to a texture that you're planning on texturing from in your next draw to the main FBO), we have to flush out all of those tiles, start rendering to the new FBO, and flush its rendering, and then when you come back to the main FBO and we have to reload your old cleared-and-a-few-draws tiles.

Job shuffling deals with this by separating the single GL command stream into separate jobs per FBO. When you switch to your temporary FBO, we don't flush the old job, we just set it aside. To make this work we have to add tracking for which buffers have jobs writing into them (so that if you try to read those from another job, we can go flush the job that wrote it), and which buffers have jobs reading from them (so that if you try to write to them, they can get flushed so that they don't get incorrectly updated contents).

This felt like it should have been harder than it was, and there's a spot where I'm using a really bad data structure I had laying around, but that data structure has been bad news since the driver was imported and it hasn't been high in any profiles yet. The other tests don't seem to have any problem with the possible increased CPU overhead.

The shuffling branch also unearthed a few bugs related to clearing and blitting in the multisample tests. Some of the piglit cases involved are fixed, but some will be reporting new piglit "regressions" because the tests are now closer to working correctly (sigh, reftests).

I also started writing documentation for updating the system's X and Mesa stack on Raspbian for one of the Foundation's developers. It's not polished, and if I was rewriting it I would use modular's build.sh instead of some of what I did there. But it's there for reference.

14 Sep 2016 9:35am GMT

A great new feature has been merged during this 1.19 X server development cycle: we're now using threads for input [1]. Previously, there were two options for how an input driver would pass on events to the X server: polling or from within the signal handler. Polling simply adds all input devices' file descriptors to a select(2) loop that is processed in the mainloop of the server. The downside here is that if the server is busy rendering something, your input is delayed until that rendering is complete. Historically, polling was primarily used by the keyboard driver because it just doesn't matter much when key strokes are delayed. Both because you need the client to render them anyway (which it can't when it's busy) and possibly also because we're just so bloody used to typing delays.

The signal handler approach circumvented the delays by installing a SIGIO handler for each input device fd and calling that when any input occurs. This effectively interrupts the process until the signal handler completes, regardless of what the server is currently busy with. A great solution to provide immediate visible cursor movement (hence it is used by evdev, synaptics, wacom, and most of the now-retired legacy drivers) but it comes with a few side effects. First of all, because the main process is interrupted, the bit where we read the events must be completely separate to the bit where we process the events. That's easy enough, we've had an input event queue in the server for as long as I've been involved with X.Org development (~2006). The drivers push events into the queue during the signal handler, in the main loop the server reads them and processes them. In a busy server that may be several seconds after the pointer motion was performed on the screen but hey, it still feels responsive.

The bigger issue with the use of a signal handler is: you can't use malloc [2]. Or anything else useful. Look at the man page for signal(7), it literally has a list of allowed functions. This leads to two weird side-effects: one is that you have to pre-allocate everything you may ever need for event processing, the other is that you need to re-implement any function that is not currently async signal safe. The server actually has its own implementation of printf for this reason (for error logging). Let's just say this is ... suboptimal. Coincidentally, libevdev is mostly async signal safe for that reason too. It also means you can't use any libraries, because no-one [3] is insane enough to make libraries async signal-safe.

We were still mostly "happy" with it until libinput came along. libinput is a full input stack and expecting it to work within a signal handler is the somewhere between optimistic, masochistic and sadistic. The xf86-input-libinput driver doesn't use the signal handler and the side effect of this is that a desktop with libinput didn't feel as responsive when the server was busy rendering.

Keith Packard stepped in and switched the server from the signal handler to using input threads. Or more specifically: one input thread on top of the main thread. That thread controls all the input device's file descriptors and continuously reads events off them. It otherwise provides the same functionality the signal handler did before: visible pointer movement and shoving events into the event queue for the main thread to process them later. But of course, once you switch to threads, problems have 2 you now. A signal handler is "threading light", only one code path can be interrupted and you know you continue where you left off. So synchronisation primitives are easier than in threads where both code paths continue independently. Keith replaced the previous xf86BlockSIGIO() calls with corresponding input_lock() and input_unlock() calls and all the main drivers have been switched over. But some interesting race conditions kept happening. But as of today, we think most of these are solved.

The best test we have at this point is libinput's internal test suite. It creates roughly 5000 devices within about 4 minutes and thus triggers most code paths to do with device addition and removal, especially the overlaps between devices sending events before/during/after they get added and/or removed. This is the largest source of possible errors as these are the code paths with the most amount of actual simultaneous access to the input devices by both threads. But what the test suite can't test is normal everyday use. So until we get some more code maturity, expect the occasional crash and please do file bug reports. They'll be hard to reproduce and detect, but don't expect us to run into the same race conditions by accident.

[1] Yes, your calendar is right, it is indeed 2016, not the 90s or so
[2] Historical note: we actually mostly ignored this until about 2010 or so when glibc changed the malloc implementation and the server was just randomly hanging whenever we tried to malloc from within the signal handler. Users claimed this was bad UX, but I think it's right up there with motif.
[3] yeah, yeah, I know, there's always exceptions.

09 Sep 2016 7:21am GMT

When working with timestamps, one question that often arises is the precision of those timestamps. Most software is good enough with a precision up to the second, and that's easy. But in some cases, like working on metering, a finer precision is required.

I don't know exactly why¹, and it makes me suffer every day, but OpenStack is really tied to MySQL (and its clones). It hurts because MySQL is a very poor solution if you want to leverage your database to actually solve problems. But that's how life is, unfair. And in the context of the projects I work on, that boils down to that we can't afford to not support MySQL.

So here we are, needing to work with MySQL and at the same time requiring timestamp with a finer precision than just seconds. And guess what: MySQL did not support that until 2011.

No microseconds in MySQL? No problem: DECIMAL!

MySQL 5.6.4 (released in 2011), a beta version of MySQL 5.6 (hello MySQL, ever heard of Semantic Versioning?), brought microsecond precision to timestamps. But the first stable version supporting that, MySQL 5.6.10, was only released in 2013. So for a long time, there was a problem without any solution.

The obvious workaround, in this case, is to reassess your choices in technologies, discover that PostgreSQL supports microsecond precision for at least a decade and problem solved.

This is not what happened in our case, and in order to support MySQL, one had to find a workaround. And so did they in our Ceilometer project, using a DECIMAL type instead of DATETIME.

The DECIMAL type takes 2 arguments: the total number of digits you need to store, and how many in that total will be used for the fractional part. Knowing that the internal storage of MySQL uses 1 byte for 2 digits, 2 bytes for 4 digits, 3 bytes for 6 digits and 4 bytes for 9 digits, and that each part is stored independently, in order to maximize your storage space, you want to pick a number of digits that fits that correctly.

This is why Ceilometer picked 14 for the integer part (9 digits on 4 bytes and 5 digits on 3 bytes) and 6 for the decimal part (3 bytes).

Wait. It's stupid because:

• DECIMAL(20, 6) implies that you uses 14 digits for the integer part, which using epoch as a reference makes you able to encode timestamp (10^14) - 1 which is year 3170843. I am certain Ceilometer won't last that far.
• 14 digits is 9 + 5 digits in MySQL which is 7 bytes, the same size that is used for 9 + 6 digits. So if you could have DECIMAL(21, 6) for the same storage space (and go up to year 31690708 which is a nice bonus, right?)

Well, I guess the original author of the patch did not read the documentation entirely (DECIMAL(20, 6) being on the MySQL documentation page as an example, I imagine it just has been copy-pasted blindly?).

The best choice for this use case would have been DECIMAL(17, 6) which would allow storing 11 digits for integer (5 bytes), supporting timestamp up to (2^11)-1 (year 5138), and 6 digits for decimal part (3 bytes), using only 8 bytes in total per timestamp.

Nonetheless, this workaround has been implemented using a SQLAlchemy custom type and works as expected:

class PreciseTimestamp(sqlalchemy.types.TypeDecorator):
    """Represents a timestamp precise to the microsecond."""
 
    impl = sqlalchemy.DateTime
 
    def load_dialect_impl(self, dialect):
        if dialect.name == 'mysql':
            return sqlalchemy.dialect.type_descriptor(
                sqlalchemy.types.DECIMAL(precision=20,
                                         scale=6,
                                         asdecimal=True))
        return sqlalchemy.dialect.type_descriptor(self.impl)

Microseconds in MySQL? Damn, migration!

As I said, MySQL 5.6.4 brought microseconds precision to the table (pun intended). Therefore, it's a great time to migrate away from this hackish format to the brand new one.

First, be aware that the default DATETIME type has no microseconds precision: you have to specify how many digits you want as an argument. To support microseconds, you should therefore use DATETIME(6).

If we were using a great RDBMS, let's say, hum, PostgreSQL, we could do that very easily, see:

postgres=# CREATE TABLE foo (mytime decimal);
CREATE TABLE
postgres=# \d foo
      Table "public.foo"
 Column │  Type   │ Modifiers
────────┼─────────┼───────────
 mytime │ numeric │
postgres=# INSERT INTO foo (mytime) VALUES (1473254401.234);
INSERT 0 1
postgres=# ALTER TABLE foo ALTER COLUMN mytime SET DATA TYPE timestamp with time zone USING to_timestamp(mytime);
ALTER TABLE
postgres=# \d foo
              Table "public.foo"
 Column │           Type           │ Modifiers
────────┼──────────────────────────┼───────────
 mytime │ timestamp with time zone │
 
postgres=# select * from foo;
           mytime
────────────────────────────
 2016-09-07 13:20:01.234+00
(1 row)

And since this is a pretty common use case, it's even an example in the PostgreSQL documentation. The version from the documentation uses a calculation based on epoch, whereas my example here leverages the to_timestamp() function. That's my personal touch.

Obviously, doing this conversion in a single line is not possible with MySQL: it does not implement the USING keyword on ALTER TABLE … ALTER COLUMN. So what's the solution gonna be? Well, it's a 4 steps job:

1. Create a new column of type DATETIME(6)
2. Copy data from the old column to the new column, converting them to the new format
3. Delete the old column
4. Rename the new column to the old column name.

But I know what you're thinking: there are 4 steps, but that's not a problem, we'll just use a transaction and embed these operations inside.

MySQL does not support transactions on data definition language (DDL). So if any of those steps fails, you'll be unable rollback steps 1, 3 and 4. Who knew that using MySQL was like living on the edge, right?

Doing this in Python with our friend Alembic

I like Alembic. It's a Python library based on SQLAlchemy that handles schema migration for your favorite RDBMS.

Once you created a new alembic migration script using alembic revision, it's time to edit it and write something along those lines:

from alembic import op
import sqlalchemy as sa
from sqlalchemy.sql import func
 
class Timestamp(sa.types.TypeDecorator):
    """Represents a timestamp precise to the microsecond."""
 
    impl = sqlalchemy.DateTime
 
    def load_dialect_impl(self, dialect):
        if dialect.name == 'mysql':
            return dialect.type_descriptor(mysql.DATETIME(fsp=6))
        return self.impl
 
def upgrade():
    bind = op.get_bind()
    if bind and bind.engine.name == "mysql":
        existing_type = sa.types.DECIMAL(
            precision=20, scale=6, asdecimal=True)
        existing_col = sa.Column("mytime", existing_type, nullable=False)
        temp_col = sa.Column("mytime_ts", Timestamp(), nullable=False)
        # Step 1: ALTER TABLE mytable ADD COLUMN mytime_ts DATETIME(6)
        op.add_column("mytable", temp_col)
        t = sa.sql.table("mytable", existing_col, temp_col)
        # Step 2: UPDATE mytable SET mytime_ts=from_unixtime(mytime)
        op.execute(t.update().values(mytime_ts=func.from_unixtime(existing_col)}))
        # Step 3: ALTER TABLE mytable DROP COLUMN mytime
        op.drop_column("mytable", "mytime")
        # Step 4: ALTER TABLE mytable CHANGE mytime_ts mytime
        # Note: MySQL needs to have all the old/new information to just rename a column…
        op.alter_column("mytable",
                        "mytime_ts",
                        nullable=False,
                        type_=Timestamp(),
                        existing_nullable=False,
                        existing_type=existing_type,
                        new_column_name="mytime")

In MySQL, the function to convert a float to a UNIX timestamp is from_unixtime(), so the script leverages it to convert the data. As said, you'll notice we don't bother using any kind of transaction, so if anything goes wrong, there's no rollback, and it won't be possible to re-run the migration without a manual intervention.

TimestampUTC is a custom class that implements sqlalchemy.DateTime using a DATETIME(6) type for MySQL, and a regular sqlalchemy.DateTime type for other back-ends. It is used by the rest of the code (e.g. ORM model) but I've pasted it in this example for a better understanding.

Once written, you can easily test your migration using pifpaf to run a temporary database:

$ pifpaf run mysql $SHELL
$ alembic -c alembic/alembic.ini upgrade 1c98ac614015 # upgrade to the initial revision
$ mysql -S $PIFPAF_MYSQL_SOCKET pifpaf
mysql> INSERT INTO mytable (mytime) VALUES (1325419200.213000);
Query OK, 1 row affected (0.00 sec)
 
mysql> SELECT * FROM mytable;
+-------------------+
| mytime            |
+-------------------+
| 1325419200.213000 |
+-------------------+
1 row in set (0.00 sec)
 
$ alembic -c alembic/alembic.ini upgrade head
 
$ mysql -S $PIFPAF_MYSQL_SOCKET pifpaf
mysql> SELECT * FROM mytable;
+----------------------------+
| mytime                     |
+----------------------------+
| 2012-01-01 13:00:00.213000 |
+----------------------------+
1 row in set (0.00 sec)

And voilà, we just migrated unsafely our data to a new fancy format. Thank you Alembic for solving a problem we would not have without MySQL. 😊

08 Sep 2016 12:02pm GMT

Last week I was tasked with making performance comparisons between vc4 and the closed driver possible. I decided to take glmark2 and port it to dispmanx, and submitted a pull request upstream. It already worked on X11 on the vc4 driver, and I fixed the drm backend to work as well (though the drm backend has performance problems specific to the glmark2 code).

Looking at glmark2, vc4 has a few bugs. Terrain has some rendering bugs. The driver on master took a big performance hit on one of the conditionals tests since the loops support was added, because NIR isn't aggressive enough in flattening if statements. Some of the tests require that we shuffle rendering jobs to avoid extra frame store/loads. Finally, we need to use the multithreaded fragment shader mode to hide texture fetching latency on a bunch of tests. Despite the bugs, results looked good.

(Note: I won't be posting comparisons here. Comparisons will be left up to the reader on their particular hardware and software stacks).

I'm expecting to get to do some glamor work on vc4 again soon, so I spent some of the time while I was waiting for Raspberry Pi builds working on the X Server's testing infrastructure. I've previously talked about Travis CI, but for it to be really useful it needs to run our integration tests. I fixed up the piglit XTS test wrapper to not spuriously fail, made the X Test suite spuriously fail less, and worked with Adam Jackson at Red Hat to fix build issues in XTS. Finally, I wrote scripts that will, if you have an XTS tree and a piglit tree and build Xvfb, actually run the XTS rendering tests at xserver make check time.

Next steps for xserver testing are to test glamor in a similar fashion, and to integrate this testing into travis-ci and land the travis-ci branch.

Finally, I submitted pull requests to the upstream kernel. 4.8 got some fixes for VC4 3D (merged by Dave), and 4.9 got interlaced vblank timing patches from the Mario Kleiner (not yet merged by Dave) and Raspberry Pi Zero (merged by Florian).

07 Sep 2016 12:04am GMT

On Fedora, if you have mate-desktop or cinnamon-desktop installed, your GNOME touchpad configuration panel won't work (see Bug 1338585). Both packages install a symlink to assign the synaptics driver to the touchpad. But GNOME's control-center does not support synaptics anymore, so no touchpad is detected. Note that the issue occurs regardless of whether you use MATE/Cinnamon, merely installing it is enough.

Unfortunately, there is no good solution to this issue. Long-term both MATE and Cinnamon should support libinput but someone needs to step up and implement it. We don't support run-time driver selection in the X server, so an xorg.conf.d snippet is the only way to assign a touchpad driver. And this means that you have to decide whether GNOME's or MATE/Cinnamon's panel is broken at X start-up time.

If you need the packages installed but you're not actually using Mate/Cinnamon itself, remove the following symlinks (whichever is present on your system):

# rm /etc/X11/xorg.conf.d/99-synaptics-mate.conf
# rm /etc/X11/xorg.conf.d/99-synaptics-cinnamon.conf
# rm /usr/share/X11/xorg.conf.d/99-synaptics-mate.conf
# rm /usr/share/X11/xorg.conf.d/99-synaptics-cinnamon.conf

The /usr/share paths are the old ones and have been replaced with the /etc/ symlinks in cinnamon-desktop-3.0.2-2.fc25 and mate-desktop-1.15.1-4.fc25 and their F24 equivalents.

06 Sep 2016 6:44am GMT

I'm using T450 and T460 as reference but this affects all laptops from the Lenovo *50 and *60 series. The Lenovo T450 and T460 have the same touchpad hardware, but unfortunately it suffers from what is probably a firmware issue. On really slow movements, the pointer has a halting motion. That effect disappears when the finger moves faster.

The observable effect is that of a pointer stalling, then jumping by 20 or so pixels. We have had a quirk for this in libinput since March 2016 (see commit a608d9) and detect this at runtime for selected models. In particular, what we do is look for a sequence of events that only update the pressure values but not the x/y position of the finger. This is a good indication that the bug triggers. While it's possible to trigger pressure changes alone, triggering several in a row without a change in the x/y coordinates is extremely unlikely. Remember that these touchpads have a resolution of ~40 units per mm - you cannot hold your finger that still while changing pressure [1]. Once we see those pressure changes only we reset the motion history we keep for each touch. The next event with an x/y coordinate will thus not calculate the delta to the previous position and not trigger a move. The event after that is handled normally again. This avoids the extreme jumps but there isn't anything we can do about the stalling - we never get the event from the kernel. [2]

Anyway. This bug popped up again elsewhere so this time I figured I'll analyse the data more closely. Specifically, I wrote a script that collected all x/y coordinates of a touchpad recording [3] and produced a black and white image of all device coordinates sent. This produces a graphic that's interesting but not overly useful:

Roughly 37000 touchpad events. You'll have to zoom in to see the actual pixels.

I modified the script to assume a white background and colour any x/y coordinate that was never hit black. So an x coordinate of 50 would now produce a vertical 1 pixel line at 50, a y coordinate of 70 a horizontal line at 70, etc. Any pixel that remains white is a coordinate that is hit at some point, anything black was unreachable. This produced more interesting results. Below is the graphic of a short, slow movement right to left.

A single short slow finger movement

You can clearly see the missing x coordinates. More specifically, there are some events, then a large gap, then events again. That gap is the stalling cursor where we didn't get any x coordinates. My first assumption was that it may be a sensor issue and that some areas on the touchpad just don't trigger. So what I did was move my finger around the whole touchpad to try to capture as many x and y coordinates as possible.

Let's have look at the recording from a T440 first because it doesn't suffer from this issue:

Sporadic black lines indicating unused coordinates but the center is purely white, indicating every device unit was hit at some point

Ok, looks roughly ok. The black areas are irregular, on the edges and likely caused by me just not covering those areas correctly. In the center it's white almost everywhere, that's where the most events were generated. And now let's compare this to a T450:

A visible grid of unreachable device units

The difference is quite noticeable, especially if you consider that the T440 recording had under 15000 events, the T450 recording had almost 37000. The T450 has a patterned grid of unreachable positions. But why? We currently use the PS/2 protocol to talk to the device but we should be using RMI4 over SMBus instead (which is what Windows has done for a while and luckily the RMI4 patches are on track for kernel 4.9). Once we talk to the device in its native protocol we see a resolution of ~20 units/mm and it looks like the T440 output:

With RMI4, the grid disappears

Ok, so the problem is not missing coordinates in the sensor and besides, at the resolution the touchpad has a single 'pixel' not triggering shouldn't be much of a problem anyway.

Maybe the issue had to do with horizontal movements or something? The next approach was for me to move my finger slowly from one side to the left. That's actually hard to do consistently when you're not a robot, so the results are bound to be slightly different. On the T440:

The x coordinates are sporadic with many missing ones, but the y coordinates are all covered

You can clearly see where the finger moved left to right. The big black gaps on the x coordinates mostly reflect me moving too fast but you can see how the distance narrows, indicating slower movements. Most importantly: vertically, the strip is uniformly white, meaning that within that range I hit every y coordinate at least once. And the recording from the T450:

Only one gap in the y range, sporadic gaps in the x range

Well, still looks mostly the same, so what is happening here? Ok, last test: This time an extremely slow motion left to right. It took me 87 seconds to cover the touchpad. In theory this should render the whole strip white if all x coordinates are hit. But look at this:

An extremely slow finger movement

Ok, now we see the problem. This motion was slow enough that almost every x coordinate should have been hit at least once. But there are large gaps and most notably: larger gaps than in the recording above that was a faster finger movement. So what we have here is not an actual hardware sensor issue but that the firmware is working against us here, filtering things out. Unfortunately, that's also the worst result because while hardware issues can usually be worked around, firmware issues are a lot more subtle and less predictable. We've also verified that newer firmware versions don't fix this and trying out some tweaks in the firmware didn't change anything either.

Windows is affected by this too and so is the synaptics driver. But it's not really noticeable on either and all reports so far were against libinput, with some even claiming that it doesn't manifest with synaptics. But each time we investigated in more detail it turns out that the issue is still there (synaptics uses the same kernel data after all) but because of different acceleration methods users just don't trigger it. So my current plan is to change the pointer acceleration to match something closer to what synaptics does on these devices. That's hard because synaptics is mostly black magic (e.g. synaptics' pointer acceleration depends on screen resolution) and hard to reproduce. Either way, until that is sorted at least this post serves as a link to point people to.

Many thanks to Andrew Duggan from Synaptics and Benjamin Tissoires for helping out with the analysis and testing of all this.

[1] Because pressing down on a touchpad flattens your finger and thus changes the shape slightly. While you can hold a finger still, you cannot control that shape
[2] Yes, predictive movement would be possible but it's very hard to get this right
[3] These are events as provided by the kernel and unaffected by anything in the userspace stack

06 Sep 2016 2:00am GMT

A few weeks ago, I recorded an interview with Krishnan Raghuram about what was discussed for this development cycle for OpenStack Telemetry at the Austin summit.

It's interesting to look back at this video more than 3 months after recording it, and see what actually happened to Telemetry. It turns out that some of the things that I think were going to happen did not happen yet. As the first release candidate version is approaching, it's very unlikely they happen.

And on the other side, some new fancy features arrived suddenly without me having a clue about them.

As far as Ceilometer is concerned, here's the list of what really happened in terms of user features:

• Added full support for SNMP v3 USM model
• Added support for batch measurement in Gnocchi dispatcher
• Set ended_at timestamp in Gnocchi dispatcher
• Allow Swift pollster to specify regions
• Add L3 cache usage and memory bandwidth meters
• Split out the event code (REST API and storage) to a new Panko project

And a few other minor things. I planned none of them except Panko (which I was responsible for), and the ones we planned (documentation update, pipeline rework and polling enhancement) did not happen yet.

For Aodh, we expected to rework the documentation entirely too, and that did not happen either. What we did instead:

• Deprecate and disable combination alarms
• Add pagination support in REST API
• Deprecated all non-SQL database store and provide a tool to migrate
• Support batch notification for aodh-notifier

It's definitely a good list of new features for Aodh, still small, but simplifying it, removing technical debt and continuing building momentum around it.

For Gnocchi, we really had no plan, except maybe a few small features (they're usually tracked in the Launchpad bug list). It turned out we had some fancy new idea with Gordon Chung on how to boost our storage engine, so we work on that. That kept us busy a few weeks in the end, though the preliminary results look tremendous - so it was definitely worth it. We also have a AWS S3 storage driver on its way.

I find this exercise interesting, as it really emphasizes how you can't really control what's happening in any open source project, where your contributors come and go and work on their own agenda.

That does not mean we're dropping the themes and ideas I've laid out in that video. We're still pushing our "documentation is mandatory" policy and improving our "work by default" scenario. It's just a longer road that we expected.

05 Sep 2016 1:20pm GMT

Like is usual in GPUs, Radeon executes shaders in waves that execute the same program for many threads or work-items simultaneously in lock-step. Given a single program counter for up to 64 items (e.g. pixels being processed by a pixel shader), branch statements must be lowered to manipulation of the exec mask (unless the compiler can prove the branch condition to be uniform across all items). The exec mask is simply a bit-field that contains a 1 for every thread that is currently active, so code like this:

    if (i != 0) {
        ... some code ...
    }

gets lowered to something like this:

    v_cmp_ne_i32_e32 vcc, 0, v1
    s_and_saveexec_b64 s[0:1], vcc
    s_xor_b64 s[0:1], exec, s[0:1]

if_block:
    ... some code ...

join:
    s_or_b64 exec, exec, s[0:1]

(The saveexec assembly instructions apply a bit-wise operation to the exec register, storing the original value of exec in their destination register. Also, we can introduce branches to skip the if-block entirely if the condition happens to be uniformly false, )

This is quite different from CPUs, and so a generic compiler framework like LLVM tends to get confused. For example, the fast register allocator in LLVM is a very simple allocator that just spills all live registers at the end of a basic block before the so-called terminators. Usually, those are just branch instructions, so in the example above it would spill registers after the s_xor_b64.

This is bad because the exec mask has already been reduced by the if-condition at that point, and so vector registers end up being spilled only partially.

Until recently, these issues were hidden by the fact that we lowered the control flow instructions into their final form only at the very end of the compilation process. However, previous optimization passes including register allocation can benefit from seeing the precise shape of the GPU-style control flow earlier. But then, some of the subtleties of the exec masks need to be taken account by those earlier optimization passes as well.

A related problem arises with another GPU-specific specialty, the "whole quad mode". We want to be able to compute screen-space derivatives in pixel shaders - mip-mapping would not be possible without it - and the way this is done in GPUs is to always run pixel shaders on 2x2 blocks of pixels at once and approximate the derivatives by taking differences between the values for neighboring pixels. This means that the exec mask needs to be turned on for pixels that are not really covered by whatever primitive is currently being rendered. Those are called helper pixels.

However, there are times when helper pixels absolutely must be disabled in the exec mask, for example when storing to an image. A separate pass deals with the enabling and disabling of helper pixels. Ideally, this pass should run after instruction scheduling, since we want to be able to rearrange memory loads and stores freely, which can only be done before adding the corresponding exec-instructions. The instructions added by this pass look like this:

    s_mov_b64 s[2:3], exec
    s_wqm_b64 exec, exec

    ... code with helper pixels enabled goes here ...

    s_and_b64 exec, exec, s[2:3]

    ... code with helper pixels disabled goes here ...

Naturally, adding the bit-wise AND of the exec mask must happen in a way that doesn't conflict with any of the exec manipulations for control flow. So some careful coordination needs to take place.

My suggestion is to allow arbitrary instructions at the beginning and end of basic blocks to be marked as "initiators" and "terminators", as opposed to the current situation, where there is no notion of initiators, and whether an instruction is a terminator is a property of the opcode. An alternative, that Matt Arsenault is working on, adds aliases for certain exec-instructions which act as terminators. This may well be sufficient, I'm looking forward to seeing the result.

03 Sep 2016 6:18pm GMT

In the X server, the input driver assignment is handled by xorg.conf.d snippets. Each driver assigns itself to the type of devices it can handle and the driver that actually loaded is simply the one that sorts last. Historically, we've had the evdev driver sort low and assign itself to everything. synaptics, wacom and the other few drivers that matter sorted higher than evdev and thus assigned themselves to the respective device.

When xf86-input-libinput first came out 2 years ago, we used a higher sort order than all other drivers to assign it to (almost) all devices. This was of course intentional because we believe that libinput is the best input stack around, the odd bug non-withstanding. Now it has matured a fair bit and we had a lot more exposure to various types of hardware. We've been quirking and fixing things like crazy and libinput is much better for it.

Two things were an issue with this approach though. First, overriding xf86-input-libinput required manual intervention, usually either copying or symlinking an xorg.conf.d snippet. Second, even though we were overriding the default drivers, we still had them installed everywhere. Now it's time to start properly retiring the old drivers.

The upstream approach for this is fairly simple: the xf86-input-libinput xorg.conf.d snippet will drop in sort order to sit above evdev. evdev remains as the fallback driver for miscellaneous devices where evdev's blind "forward everything" approach is sufficient. All other drivers will sort higher than xf86-input-libinput and will thus override the xf86-input-libinput assignment. The new sort order is thus:

• evdev
• libinput
• synaptics, wacom, vmmouse, joystick

evdev and libinput are generic drivers, the others are for specific devices or use-cases. To use a specific driver other than xf86-input-libinput, you now only have to install it. To fall back to xf86-inputlibinput, you uninstall it. No more manual xorg.conf.d snippets symlinking.

This has an impact on distributions and users. Distributions should ensure that other drivers are never installed by default unless requested by the user (or some software). And users need to be aware that having a driver other than xf86-input-libinput installed may break things. For example, recent GNOME does not support the synaptics driver anymore, installing it will cause the control panel's touchpad bits to stop working. So there'll be a messy transition period but once things are settled, the solution to most input-related driver bugs will be "install/remove driver $foo" as opposed to the current symlink/copy/write an xorg.conf.d snippet.

31 Aug 2016 12:01am GMT

I spent a day or so last week cleaning up @jonasarrow's demo patch for derivatives on vc4. It had been hanging around on the github issue waiting for a rework due to feedback, and I decided to finally just go and do it. It unfortunately involved totally rewriting their patches (which I dislike doing, it's always more awesome to have the original submitter get credit), but we now have dFdx()/dFdy() on Mesa master.

I also landed a fix for GPU hangs with 16 vertex attributes (4 or more vec4s, aka glsl-routing in piglit). I'd been debugging this one for a while, and finally came up with an idea ("what if this FIFO here is a bad idea to use and we should be synchronous with this external unit?"), it worked, and a hardware developer confirmed that the fix was correct. This one got a huge explanation comment. I also fixed discards inside of if/loop statements -- generally discards get lowered out of ifs, but if it's in a non-unrolled loop we were doing discards ignoring whether the channel was in the loop.

Thanks to review from Rhys, I landed Mesa's Travis build fixes. Rhys then used Travis to test out a couple of fixes to i915 and r600. This is pretty cool, but it just makes me really want to get piglit into Travis so that we can get some actual integration testing in this process.

I got xserver's Travis to the point of running the unit tests, and one of them crashes on CI but not locally. That's interesting.

The last GPU hang I have in piglit is in glsl-vs-loops. This week I figured out what's going on, and I hope I'll be able to write about a fix next week.

Finally, I landed Stefan Wahren's Raspberry Pi Zero devicetree for upstream. If nothing goes wrong, the Zero should be supported in 4.9.

29 Aug 2016 6:55pm GMT