fidzu : freedesktop

Linux.conf.au 2020

I just got back from linux.conf.au 2020 on Saturday and am still adjusting to being home again. I had the opportunity to give three presentations during the conference and wanted to provide links to the slides and videos.

Picolibc

My first presentation was part of the Open ISA miniconf on Monday. I summarized the work I've been doing on a fork of Newlib called Picolibc which targets 32- and 64- bit embedded processors.

• Video

• Slides

Snek

Wednesday morning, I presented on my snek language, which is a small Python designed for introducing programming in an embedded environment. I've been using this for the last year or more in a middle-school environment (grades 5-7) as a part of a LEGO robotics class.

• Video

• Slides

• Snek Language Reference

X History and Politics

Bradley Kuhn has been encouraging me to talk about the early politics of X and how that has shaped my views on the benefits of copyleft licenses in building strong communities, especially in driving corporate cooperation and collaboration. I would have loved to also give this talk as a part of the Copyleft Conference being held in Brussels after FOSDEM, but I won't be at that event. This talk spans the early years of X, covering events up through 1992 or so.

• Video

• Slides

21 Jan 2020 11:02pm GMT

A while ago as a spin-off of my project to improve support for Logitech wireless keyboards and mice I have also done some work on improving support for (Gaming) keyboards with a builtin LCD panel.

Specifically if you have a Logitech MX5000, G15, G15 v2 or G510 and you want the LCD panel to show something somewhat useful then on Fedora 31 you can now install the lcdproc package and it will automatically recognize the keyboard and show "top" like information on it. No need to manually write an LCDd.conf or anything, this works fully plug and play:

sudo dnf install lcdproc
sudo udevadm trigger

If you have a MX5000 and you do not want the LCD panel to show "top" like info, you may still want to install the mx5000tools package, this will automatically send the system time to the keyboard, after which it will display the time.

Once the 5.5 kernel becomes available as an update for Fedora you will also be able to use the keys surrounding the LCD panel to control the lcdproc menu-s on the LCD panel. The 5.5 kernel will also export key backlight brightness control through the standardized /sys/class/leds API, so that you can control it from e.g. the GNOME control-center's power-settings and you get a nice OSD when toggling the brightnesslevel using the key on the keyboard.

The 5.5 kernel will also make the "G" keys send standard input-events (evdev events), once userspace support for the new key-codes these send has landed, this will allow e.g. binding them to actions in GNOME control-center's keyboard-settings. But only under Wayland as the new keycodes are > 255 and X11 does not support this.

17 Jan 2020 1:39pm GMT

So continuing with the news, here is a fairly recent one: as the tile states, I am happy to announce that the Raspberry Pi 4 is now an OpenGL ES 3.1 conformant product!. This means that the Mesa V3D driver has successfully passed a whole lot of tests designed to validate the OpenGL ES 3.1 feature set, which should be a good sign of driver quality and correctness.

It should be noted that the Raspberry Pi 4 shipped with a V3D driver exposing OpenGL ES 3.0, so this also means that on top of all the bugfixes that we implemented for conformance, the driver has also gained new functionality! Particularly, we merged Eric's previous work to enable Compute Shaders.

All this work has been in Mesa master since December (I believe there is only one fix missing waiting for us to address review feedback), and will hopefully make it to Raspberry Pi 4 users soon.

17 Jan 2020 10:02am GMT

I actually landed this in Mesa back in December but never got to announce it anywhere. The implementation passes all the tests available in the Khronos Conformance Tests Suite (CTS). If you give this a try and find any bugs, please report them here with the V3D tag.

This is also the first large feature I land in V3D! Hopefully there will be more coming in the future.

17 Jan 2020 9:45am GMT

Yeah… this blog post is well overdue, but better late than never! So yes, I am currently working on progressing the Raspberry Pi 4 Mesa driver stack, together with my Igalian colleagues Piñeiro and Chema, continuing the fantastic work started by Eric Anholt on the Mesa V3D driver.

The Raspberry Pi 4 sports a Video Core VI GPU that is capable of OpenGL ES 3.2, so it is a big update from the Raspberry Pi 3, which could only do OpenGL ES 2.0. Another big change with the Raspberry Pi 4 is that the Mesa v3d driver is the driver used by default with Raspbian. Because both GPUs are quite different, Eric had to write an all new driver for the Raspberry Pi 4, and that is why there are two drivers in Mesa: the VC4 driver is for the Raspberry Pi 3, while the V3D driver targets the Raspberry Pi 4.

As for what we have been working on exactly, I wrote a long post on the Raspberry Pi blog some months ago with a lot of the details, but for those looking for the quick summary:

• Shader compiler optimizations.
• Significant Transform Feedback fixes and improvements.
• Implemented OpenGL Logic Operations.
• A bunch of bugfixes for Piglit test failures.
• Set up a Continuous Integration system to identify regressions.
• Rebased and merge Eric's work on Compute Shaders.
• Many bug fixes targeting the Khronos OpenGL ES Conformance Test Suite (CTS).

So that's it for the late news. I hope to do a better job keeping this blog updated with the news this year, and to start with that I will be writing a couple of additional posts to highlight a few significant development milestones we achieved recently, so stay tuned for more!

17 Jan 2020 9:31am GMT

Downloads

If you're curious about the slides, you can download the PDF or the ODP.

Diagrams

Overview

Kernel

Userspace

Mesa overview

Mesa Gallium

Mesa Winsys

Mesa Compiler

License

All of the material you can find on this page is licensed under the MIT license, and you are free to use, modify and re-distribute these materials however you like.

Thanks

I would like to thank the wonderful organizers of Linux Conf Au for hosting the event. This was my first LCA, and I think it may be the best Linux conference out of all of them.

I would also like to thank the organizers for paying for my flights, without it I would not have been able to give this talk.

14 Jan 2020 11:00pm GMT

TL;DR

Use GMemoryMonitor in glib 2.63.3 and newer in your applications to lower overall memory usage, and detect low memory conditions.

low-memory-monitor

To start with, let's come back to low-memory-monitor, announced at the end of August.

It's not really a "low memory monitor". I know, the name is deceiving, but it actually monitors memory pressure stalls, and how hard it is for the kernel to allocate memory when applications need it. The longer it takes to allocate memory, the longer the kernel takes to allocate it, usually because it needs to move memory around to make room for a big allocation, when an application starts up for example, or prepares an in-memory buffer for saving.

It is not a daemon that will kill programs on low memory. It's not a user-space out-of-memory killer, and does not take those policy decisions. It can however be configured to ask the kernel to do that. The kernel doesn't really know what it's doing though, and user-space isn't helping either, so best disable that for now...

As listed in low-memory-monitor's README (and in the announcement post), there were a number of similar projects around, but none that would offer everything we needed, eg.:

• Has a D-Bus interface to propagate low memory conditions
• Requires Linux 5.2's kernel memory pressure stalls information (Android's lowmemorykiller daemon has loads of code to get the same information from the kernel for older versions, and it really is quite a lot of code)
• Written in a compiled language to save on startup/memory usage costs (around 500 lines of C code, as counted by sloccount)
• Built-in policy, based upon values used in Android and Endless OS

GMemoryMonitor

Next up, in our effort to limit memory usage, we'll need some help from applications. That's where GMemoryMonitor comes in. It's simple enough, listen to the low-memory-warning signal and free some image thumbnails, index caches, or dump some data to disk, when you receive a signal.

The signal also gives you a "warning level", with 255 being when low-memory-monitor would trigger the kernel's OOM killer, and lower values different levels of "try to be a good citizen".

The more astute amongst you will have noticed that low-memory-monitor runs as root, on the system bus, and wonder how those new fangled (5 years old today!) sandboxed applications would receive those signals. Fear not! Support for a portal version of GMemoryMonitor landed in xdg-desktop-portal on the same day as in glib. Everything tied together with installed tests that use the real xdg-desktop-portal to test the portal and unsandboxed versions.

How about an OOM killer?

By using memory pressure stall information, we receive information about the state of the kernel before getting into swapping that'd cause the machine to become unusable. This also means that, as our threshold for keeping everything ticking is low, if we were to kill high memory consumers, we'd get a butter smooth desktop, but, based on my personal experience, your browser and your mail client would take it in turns disappearing from your desktop in a way that you wouldn't even notice.

We'll definitely need to think about our next step in application state management, and changing our running applications paradigm.

Distributions should definitely disable the OOM killer for now, and possibly try their hands at upstream some systemd OOMPolicy and OOMScoreAdjust options for system daemons.

Conclusion

Creating low-memory-monitor was easy enough, getting everything else in place was decidedly more complicated. In addition to requiring changes to glib, xdg-desktop-portal and python-dbusmock, it also required a lot of work on the glib CI to save me from having to write integration tests in C that would have required a lot of scaffolding. So thanks to all involved in particular Philip Withnall for his patience reviewing my changes.

17 Dec 2019 11:53pm GMT

If you remember, back in 2016, I did the work to get a "Launch on Discrete GPU" menu item added to application in gnome-shell.

This cycle I worked on adding support for the NVIDIA proprietary driver, so that the menu item shows up, and the right environment variables are used to launch applications on that device.

Behind the scenes

There were a number of problems with the old detection code in switcheroo-control:
- it required the graphics card to use vga_switcheroo in the kernel, which the NVIDIA driver didn't do
- it could support more than 2 GPUs
- and it didn't really actually know which GPU was going to be the "main" one

And, on top of all that, gnome-shell expected the Mesa OpenGL stack to be used, so it only knew the right environment variables to do that, and only for one secondary GPU.

So we've extended switcheroo-control and its API to do all this.

(As a side note, commenters asked me about the KDE support, and how it would integrate, and it turns out that KDE's code just checks for the presence of a file in /sys, which is only present when vga_switcheroo is used. So I would encourage KDE to adopt the switcheroo-control D-Bus API for this)

Closing

All this will be available in Fedora 32, using GNOME 3.36 and switcheroo-control 2.0. We might backport this to Fedora 31 after it's been tested, and if there is enough interest.

13 Dec 2019 4:13pm GMT

Unlike the tradition of my past few talks at Linux Plumbers or Kernel conferences, this time around in Lisboa I did not start out with a rant proposing to change everything. Instead I celebrated roughly 10 years of upstream graphics progress and finally achieving paradise. But that was all just prelude to a few bait-and-switches later fulfill expectations on what's broken this time around in upstream, totally, and what needs to be fixed and changed, maybe.

The LPC video recording is now released, slides are uploaded. If neither of that is to your taste, read below the break for the written summary.

Mission Accomplished

10 or so years ago upstream graphics was essentially a proof of concept for the promised to come. Kernel display modeset just landed, finally bringing a somewhat modern display driver userspace API to linux. And GEM, the graphics execution manager landed, bringing proper GPU memory management and multi client rendering. Realistically a lot needed to be done still, from rendering drivers for all the various SoC, to an atomic display API that can expose all the features, not just what was needed to light up a linux desktop back in the days. And lots of work to improve the codebase and make it much easier and quicker to write drivers.

There's obviously still a lot to do, but I think we've achieved that - for full details, check out my ELCE talk about everything great for upstream graphics.

Now despite all this justified celebrating, there is one sticking point still:

NVIDIA

The trouble with team green from an open source perspective - for them it's a great boon - is that they own the GPU software stack in two crucial ways:

• NVIDIA defines how desktop GL works. Not so relevant anymore, and at least the core profile is a solid spec and has fully open source test suite from Khronos by now. But the compatibility profile, which didn't throw out all the legacy features from the GL1.x days in the 90s, does not have any of the interactions with all the new features specced out and covered with tests - NVIDIA's binary driver is that standard, and that since roughly 20 years.

• More relevant today is CUDA, not quite as long as desktop GL, but for a market that's growing at a rather brisk pace. CUDA is the undisputed king of the general purpose GPU compute hill. Anything and everything that matters runs on top of it, often exclusively.

Together these create a huge software moat around the high margin hardware business. All an open stack would achieve is filling in that moat and inviting competition to eat the nice surplus. In other words, stupid to even attempt, vendor lock-in just pays too well.

Now of course the reverse engineered nouveau driver still exists. But if you have to pay for reverse engineering already, then you might as well go with someone else's hardware, since you're not going to get any of the CUDA/GL goodies.

And the business case for open source drivers indeed exists so much that even paying for reverse engineering a full stack is no problem. The result is a vibrant community of hardware vendors, customers, distros and consulting shops who pay the bills for all the open driver work that's being done. And in userspace even "upstream first" works - releases happen quickly and often enough, with sufficiently smooth merge process that having a vendor tree is simply not needed. Plus customer's willingness to upgrade if necessary, because it's usually a well-contained component to enable new hardware support.

In short without a solid business case behind open graphics drivers, they're just not going to happen, viz. NVIDIA.

Not Shipping Upstream

Unfortunately the business case for "upstream first" on the kernel side is completely broken. Not for open source, and not for any fundamental reasons, but simply because the kernel moves too slowly, is too big, drivers aren't well contained enough and therefore customer will not or even can not upgrade. For some hardware upstreaming early enough is possible, but graphics simply moves too fast: By the time the upstreamed driver is actually in shipping distros, it's already one hardware generation behind. And missing almost a year of tuning and performance improvements. Worse it's not just new hardware, but also GL and Vulkan versions that won't work on older kernels due to missing features, fragementing the ecosystem further.

This is entirely unlike the userspace side, where refactoring and code sharing in a cross-vendor shared upstream project actually pays off. Even in the short term.

There's a lot of approaches trying to paper over this rift with the linux kernel:

• Stable kernel ABI for driver modules, so that you can upgrade the core kernel and drivers independently. Google Android is very much aiming this solution at their huge vendor tree problem. Traditionally enterprise distros do the same. This works, safe that stable kernel-internal ABI is not a notion that's very popular with kernel maintainers …

• If you go with an "upstream first" approach to shipping graphics drivers you first need to polish your driver, refactor out common components, and push it to upstream. Only to then pay a second team to re-add all the crap so you can ship your driver on all the old kernels, where all the helpers and new common code don't exist.

• Pay your distro or OS vendor to just backport the new helpers before they even have landed in an upstream release. Which means instead of a backporting team for the driver on your payroll you now pay for backporting the entire subsystem - which in many cases is cheaper, but an even harder sell to beancounters. And sometimes not possible because other driver teams from competitors might not be on board and insist on not breaking the stable driver ABI for a given distro release kernel.

Also, there just isn't a single LTS kernel. Even upstream has multiple, plus every distro has their own flavour, plus customers love to grow their own variety trees too. Often they're not even coordinated on the same upstream release. Cheapest way to support this entire madness is to completely ignore upstream and just write your own subsystem. Or at least not use any of the helper libraries provided by kernel subsystems, completely defeating the supposed benefit of upstreaming code.

No matter the strategy, they all boil down to paying twice - if you want to upstream your code. And there's no added return for the doubled bill. In conclusion, upstream first needs a business case, like the open source graphics stack in general. And that business case is very much real, except for upstreaming, it's only real in userspace.

In the kernel, "upstream first" is a sham, at least for graphics drivers.

Thanks to Alex Deucher for reading and commenting on drafts of this text.

10 Dec 2019 12:00am GMT

At ELC Europe in Lyon I held a nice little presentation about the state of upstream graphics drivers, and how absolutely awesome it all is. Of course with a big focus on SoC and embedded drivers. Slides and the video recording

Key takeaways for the busy:

• The upstream DRM graphics subsystem really scales down to tiny drivers now, with the smallest driver coming in at just around 250 lines (including comments and whitespace), 10'000x less than the biggest!

• Batteries all included - there's modular helpers for everything. As a rule of thumb even minimal legacy fbdev drivers ported to DRM shrink by a factor of 2-4 thanks to these helpers taking care of anything that's remotely standardized in displays and GPUs.

• For shipping userspace drivers go with a dual-stack: Open source GL and Vulkan drivers for those who want that, and for getting the kernel driver merged into upstream. Closed source for everyone else, running on the same userspace API and kernel driver.

• And for customer support backport the entire subsystem, try to avoid backporting an individual driver.

In other words, world domination is assured and progressing according to plan.

03 Dec 2019 12:00am GMT

I got contacted by a user with a HP X2 10 p018wm 2-in-1 about the device waking up 10-60 seconds after suspend. I have access to a HP X2 10 p002nd myself which in essence is the same HW and I managed to reproduce the problem there. This is when the fun started:

1. There were a whole bunch of ACPI related errors in dmesg. It turns out that these affect almost all HP laptop models and we have a multiple bugs open for this. Debugging these pointed to the hp-wmi driver. I wrote 2 patches fixes 2 different kind of errors and submitted these upstream. Unfortunately this does not help with the suspend/resume issue, but it does fix all those errors people have been complaining about :)

2. I noticed some weird messages in dmesg with look like a PCI bus re-enumeration is started during suspend when suspending by closing the lid and then the re-enumeration continues after resume. This turns out to be triggered by this piece of buggy AML code which
is used for monitor hotplug notification on gfx state changes (the i915 driver ACPI opregion also tracks the lid state for some reason):

Method (GNOT, 2, NotSerialized)
{
...
CEVT = Arg0
CSTS = 0x03
If (((CHPD == Zero) && (Arg1 == Zero)))
{
If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))
{
Notify (PCI0, Arg1)
}
Else
{
Notify (GFX0, Arg1)
}
}
...
}

Notice how "If (((OSYS > 0x07D0) || (OSYS < 0x07D6)))" is always true, the condition is broken the "||" clearly should have been a "&&" this is causing the code to send a hotplug notify to the PCI root instead of to the gfx card, triggering a re-enumeration. Doing a grep for this on my personal DSDT collection shows that 55 of the 93 DSDTs in my collection have this issue!

Luckily this can be easily fixed by setting CHPD to 1 in the i915 driver, which is something which we should do anyways according to the
opregion documentation. So I wrote a patch doing this and submitted it upstream. Unfortunately this also does not help with the suspend/resume issue.

3. So the actual spurious wakeups are caused by HP using an external embedded controller (EC) on the "legacy-free" platform which they use for these laptops. Since these are not designed to use an external EC they lack the standard interface for this, so HP has hooked the EC up over I2C and using an ACPI GPIO event handler as EC interrupt.

These devices use suspend2idle (s2idle) instead of good old firmware handled S3, so the EC stays active during suspend. It does some housekeeping work which involves a round-trip through the AML code every minute. Normally EC wakeups are ignored durin s2idle by some special handling in the kernel, but this is only done for ECs using the standardized ACPI EC interface, not for this bolted on the
side model. I've started a discussion on maybe extending our ACPI event handling to deal with this special case.

For now as a workaround I ended up writing 2 more patches to allow blacklisting wakeup by ACPI GPIO event handlers on select models. This breaks wakeup by opening the LID, the user needs to wake the laptop with the powerbutton. But at least the laptop will stay suspended now.

27 Nov 2019 3:06pm GMT

There have been questions about the Fedora 30 Flicker Free Boot Change in various places, here is a FAQ which hopefully answers most questions:

1) I get a black screen for a couple of seconds during boot?

1.1) If you have an AMD or Nvidia GPU driving your screen, then this is normal. The graphics drivers for AMD and Nvidia GPUs reset the hardware when loading, this will cause the display to temporarily go black. There is nothing which can be done about this.

1b) If you have a somewhat older Intel GPU (your CPU is pre Skylake) then the i915 driver's support to skip the mode-reset is disabled by default (for now) to fix this add "i915.fastboot=1" to your kernel commandline. For more info on modifying the kernel cmdline, see question 7.

1c) Do "ls /sys/firmware/efi/efivars" if you get a "No such file or directory" error then your system is booting in classic BIOS mode instead of UEFI mode, to fix this you need to re-install and boot the livecd/installer in UEFI mode when installing. Alternatively you can try to convert your existing install, note this is quite tricky, make backups first!

1d) Your system may be using the classic VGA BIOS during boot despite running in UEFI mode. Often you can select BIOS mode compatility in your BIOS settings aka the CSM setting. If you can select this on a per component level, set the VIDEO/VGA option to "UEFI only" or "UEFI first", alternatively you can try completely disabling the CSM mode. On some systems you can disable the classic VGA BIOS by disabling / unselecting the "Legacy Option ROMs" option.

2) The vendor-logo/firmware-splash looks squashed or has the wrong size?

Your system may be using the classic VGA BIOS during boot despite running in UEFI mode, see answer 1d.

3) I get a black background instead of the firmware splash while Fedora is booting?

Do "ls /sys/firmware/acpi/bgrt" if you get a "No such file or directory" error then try answers 1c and 1d . If you do have a /sys/firmware/acpi/bgrt directory, but you are still getting the Fedora logo + spinner on a black background instead of on top of the firmware-splash, please file a bug about this and drop me a mail with a link to the bug.

4) Getting rid of the vendor-logo/firmware-splash being shown while Fedora is booting?

If you don't want the firmware-splash to be used as background during boot, you can switch plymouth to the spinner theme, which is identical to the new bgrt theme, except that it does not use the firmware-splash as background, to do this execute the following command from a terminal: "sudo plymouth-set-default-theme -R spinner"

Note that the kernel will restore the vendor-logo early on at boot in case it got damaged by e.g. option ROM messages. If you are switching to the spinner theme you may also want to add "video=efifb:nobgrt" to your kernel commandline. See 7 below for how to edit the kernel commandline.

5) Keeping the firmware-splash as background while unlocking the disk?

If you prefer this, it is possible to keep the firmware-splash as background while the diskcrypt password is shown. To do this do the following:

1. 
   
2. "sudo mkdir /usr/share/plymouth/themes/mybgrt"
   
3. "sudo cp /usr/share/plymouth/themes/bgrt/bgrt.plymouth /usr/share/plymouth/themes/mybgrt/mybgrt.plymouth"
   
4. edit /usr/share/plymouth/themes/mybgrt/mybgrt.plymouth, change DialogClearsFirmwareBackground=true to DialogClearsFirmwareBackground=false, change DialogVerticalAlignment=.382 to DialogVerticalAlignment=.6
   
5. "sudo plymouth-set-default-theme -R mybgrt"
   

Note if you do this the disk-passphrase entry dialog may be partially drawn over the vendor-logo part of the firmware-splash, if this happens then try increasing DialogVerticalAlignment to e.g. 0.7 .

6) Get detailed boot progress instead of the boot-splash ?

To get detailed boot progress info press ESC during boot.

7) Always get detailed boot progress instead of the boot-splash ?

To always get detailed boot progress instead of the boot-splash, remove "rhgb" from your kernel commandline:

Edit /etc/default/grub and remove rhgb from GRUB_CMDLINE_LINUX and then if you are booting using UEFI (see 1c) run:
"grub2-mkconfig -o /etc/grub2-efi.cfg"
else (if you are booting using classic BIOS boot) run:
"grub2-mkconfig -o /etc/grub2.cfg".

27 Nov 2019 10:02am GMT

The Lenovo Thinkpad 8 and also the Asus T100CHI both have an USB3 micro-B connector, but using a standard USB3 OTG (USB 3 micro-B to USB3-A receptacle) cable results in only USB2 devices working. USB3 devices are not recognized.

Searching the internet learns that many people have this problem and that the solution is to find a USB3 micro-A to USB3-A receptacle cable. This sounds like nonsense to me as micro-B really is micro-AB and is supposed to use the ID pin to automatically switch between modes dependent on the used cable.; and this does work for the USB-2 parts of the micro-B connector on the Thinkpad. Yet people do claim success with such cables (with a more square micro-A connector, instead of the trapezoid micro-B connector). The only problem is such cables are not for sale anywhere.

So I guessed that this means is that they have swapped the Rx and Tx superspeed pairs on the USB3 only part of the micro-B connector, and I decided to cut open one of my USB3 micro-A to USB3-A receptacle cables and swap the superspeed pairs. Here is what the cable looks like when it it cut open:



If you are going to make such a cable yourself, to get this picture I first removed the outer plastic isolation (part of it is still there on the right side in this picture). Then I folded away the shield wires till they are all on one side (wires at the top of the picture). After this I removed the metal foil underneath the shield wires.

Having removed the first 2 layers of shielding this reveals 4 wires in the standard USB2 colors: red, black, green and white. and 2 separately shielded cable pairs. On the picture above the separately shielded pairs have been cut, giving us 4 pairs, 2 on each end of cable; and the shielding has been removed from 3 of the 4 pairs, you can still see the shielding on the 4th pair.

A standard USB3 cable uses the following color codes:

• 
  
• Red: Vbus / 5 volt
• 
  
• White: USB 2.0 Data -
• 
  
• Green: USB 2.0 Data +
• 
  
• Black: Ground
• 
  
• Purple: Superspeed RX -
• 
  
• Orange: Superspeed RX +
• 
  
• Blue: Superspeed TX -
• 
  
• Yellow: Superspeed TX -
• 
  

So to swap RX and TX we need to connect purple to blue / blue to purple and orange to yellow / yellow to orange, resulting in:



Note the wires are just braided together here, not soldered yet. This is a good moment to carefully test the cable. Also note that the superspeed wire pairs must be length matched, so you need to cut and strip all 8 cables at the same length! If everything works you can put some solder on those braided together wires, re-test after soldering, and then cover them with some heat-shrink-tube:



And then cover the entire junction with a bigger heat-shrink-tube:



And you have a superspeed capable cable even though no one will sell you one.

Note that the Thinkpad 8 supports ACA mode so if you get an ACA capable "Y" cable or an ACA charging HUB then you can charge and use the Thinkpad 8 USB port at the same time. Typically ACA "Y" cables or hubs are USB2 only. So the superspeed mod from this blogpost will not help with those. The Asus T100CHI has a separate USB2 micro-B just for charging, so you do not need anything special there to charge + connect an USB device.

15 Nov 2019 8:10pm GMT

It has been a while since my last post, but there is a simple reason for that: on August 5th, I had to move from Brazil to Canada. Why did I move? Thanks to Harry Wentland recommendation, I got an interview for a software engineering position at AMD (Markham), and I got hired to work on the display team. From now on, I suppose that I'll be around the DRM subsystem for a long time :). Even though I'm now employed by AMD this post reflect my personal thoughts only and should not be construed to represent AMD in any way.

I have a few updates about my work with the community since I have been busy with my relocation and adaptation. However, my main updates came from XDC 2019 [1] and I want to share it here.

XDC 2019 - Montréal (Concordia University Conference)

This year I had the great luck joining XDC again. However, at this time, I was with Harry Wentland, Nicholas Kazlauskas, and Leo Li (we worked together at AMD). We put effort into learning from other people's experiences, and we tried to know what the compositor developers wanted to see in our driver. We also used this opportunity to try to explain a little bit more about our hardware features. In particular, we had conversations about Freesync, MST, DKMS, and so forth. Thinking of that, I'll share my view of the most exciting moments that we had.

VKMS

As usual, I tried my best to understand what people want to see in VKMS soon or later. For example, from the XDC 2018, I focused on fixing some bugs but mainly in add writeback support cause it could provide a visual output (this work is almost done, see [2]). This year I collected feedback from multiple people (special thanks to Marten, Lyude, Hiler, and Harry), and from these conversations I tend to work in the following tasks:

1. Finish the writeback feature and enable visual output;
2. Add support for adaptive refresh rate;
3. Add support for "dynamic connectors", which can enable the MST test.

Additionally, Martin Peres gave a talk that he shared his views for the CI and test. In his presentation, he suggested using VKMS to validate the API, and I have to admit that I'm really excited about this idea. I hope that I can help with this.

Freesync

The amdgpu drivers support a technology named Freesync [3]. In a few words, this feature allows the dynamic change of the refreshes rate, which can bring benefits for games and for power saving. Harry Wayland talked about that feature and you can see it here:

After Harry's presentation, many people asked interesting questions related to this subject, this situation caught my attention, and for this reason, I added the VRR to my roadmap in the VKMS. Roman Gilg, from KDE, was one of the developers that asked for a protocol extension in Wayland for support Freesync; additionally, compositor developers asked for mechanisms that enable them to know in advance if the experience of a specific panel will be good or not. Finally, there were some discussions about the use of Freesync for power saving and in an application that requires time-sensitive.

IGT and CI

This year I kept my tradition of asking thousands of questions to Hiler with the goal of learning more about IGT, and as usual, he was extremely kind and gentle with my questions (thanks Hiler). One of the concepts that Hiler explained to me, it is the use of podman (https://podman.io/) for prepare IGT image, for example, after a few minutes of code pair with him I could run IGT on my machine after executing the following commands:

sudo su
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master \
                        igt_runner -t core_auth
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master \
                        igt_runner -t core_auth /tmp
podman run --privileged -v /tmp/results:/results \
  registry.freedesktop.org/drm/igt-gpu-tools/igt:master igt_runner -t core_auth /results

We also had a chance to discuss CI with Martin Peres, and he explained his work for improving the way that the CI keep track of bugs. In particular, he introduced a fantastic tool named cibuglog, which is responsible for keep tracking of test failures and using this data for building a database. Cibuglog has many helpful filters that enable us to see test problems associated with a specific machine and bugs in the Bugzilla. The huge insights from cibuglog it is the idea of using data for helping with bug tracking. Thanks Martin, for showing us this amazing tool.

Updates

I just want to finish this post with brief updates from my work with free software, starting with kw and finish with VKMS.

Kernel Workflow (kw)

When I started to work with VKMS, I wrote a tool named kworkflow, or simply kw, for helping me with basic tasks related to Kernel development. These days kw reborn to me since I was looking for a way to automate my work with amdgpu; as a result, I implemented the following features:

• Kernel deploy in a target machine (any machine reachable via IP);
• Module deploy;
• Capture .config file from a target machine;

Unfortunately, the code is not ready for merging in the main branch, I'm working on it; I think that in a couple of weeks, I can release a new version with these features. If you want to know a little bit more about kw take a look at https://siqueira.tech/doc/kw/

VKMS

I was not working in VKMS due to my change of country; however, now I am reworking part of the IGT test related to writeback, and as soon as I finish it, I will try to upstream it again. I hope that I can also have the VKMS writeback merged into the drm-misc-next by the end of this month. Finally, I merged the prime supported implemented by Oleg Vasilev (huge thanks!).

References

[1] "First discussion in the Shayenne's patch about the CRC problem". URL: https://xdc2019.x.org. ⤴

[2] "Introduces writeback support". URL: https://patchwork.freedesktop.org/series/61738/. ⤴

[3] "FreeSync". URL: https://en.wikipedia.org/wiki/FreeSync. ⤴

10 Nov 2019 2:00am GMT

As mentioned in an earlier blogpost, I have been working on fixing many games showing a small image centered on a black background when they are run fullscreen under Wayland. In that blogpost I was moslty looking at how to solve this for native Wayland games. But for various reasons almost all games still use X11, so instead I've ended up focussing on fixing this for games using Xwayland.

Xwayland now has support for emulating resolution changes requested by an app through the randr or vidmode extensions. If a client makes a resolution change requests this is remembered and if the client then creates a window located at the monitor's origin and sized to exactly that resolution, then Xwayland will ask the compositor to scale it to fill the entire monitor.

For apps which use _NET_WM_FULLLSCREEN (e.g. SDL2, SFML or OGRE based apps) to go fullscreen some help from the compositor is necessary. This is currently implemented in mutter. If you are a developer of another compositor and have questions about this, please drop me an email.

I failed to get this all upstream in time for Fedora 31 final. But now it is all upstream, so 've backported the changes and created an update with the changes. This update is currently in updates-testing, to install this update run the following command:

sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2019-103a594d07

05 Nov 2019 8:41am GMT

I recently went to XDC 2019, where I gave yet another talk about Zink. I kinda forgot to write a blog-post about it, so here's me trying to make up for it… or something like that. I'll also go into some more recent developments as well.

My presentation was somewhat similar to the talk I did at SIGGRAPH this year, but with a bit more emphasis on the technical aspect, as the XDC audience is more familiar with Mesa internals.

If you're interested, you can find the slides for the talk here. The talk goes through the motivation and basic approach. I don't think I need to go through this again, as I've already covered that before.

As for the status, Zink currently supports OpenGL 2.1 and OpenGL ES 2.0. And there's no immediate plans on working on OpenGL 3.0 and OpenGL ES 3.0 until Zink is upstream.

Which gets us to the more interesting bit; that I started working on upstreaming Zink. So let's talk about that for a bit.

Upstreaming Zink

So, my current goal is to get Zink upstream in Mesa. The plan outline in my XDC talk is slightly outdated by now, so here I'll instead say what's actually happened so far, and what I hope will follow.

Before I could add the driver itself, I decided to send all changes outside of the driver as a set of merge-requests:

• glBitmap R8 textures
• loader/dri3: do not blit outside old/new buffers
• gallium/u_blitter: set a more sane viewport-state
• nir: initialize uses_discard to false
• lowering passes from Zink

The last one is probably the most interesting one, as it moves a lot of fixed-function operations into the state-tracker, so individual drivers won't have to deal with them. Unless they choose to, of course.

But all of these has already been merged, and there's just one final merge-request left:

• Introduce Zink: OpenGL on Vulkan

This merge-request adds the driver in its current state. It consists of 163 commits at the time of writing, so it's not a thing of beauty. But new drivers usually aren't, so I'm not too worried.

When this is merged, Zink will finally be a "normal" part of Mesa… Well, sort of anyway. I don't think we'll enable Zink to be built by default for a while. But that'll just be a matter of adding zink to the -Dgallium-drivers meson-option.

Testing on CI

The current branch only adds building of Zink to the CI. There's no testing being done yet. The reasons for this is two-fold:

1. We need to get a running environment on CI. Rather of bringing up some hardware-enabled test-runner, I intend to try to set up SwiftShader as a software rasterizer instead, as that supports Vulkan 1.1 these days.
2. We need some tests to run. Zink currently only supports OpenGL 2.1, and sadly the conformance test suite doesn't have any tests for OpenGL versions older than 3.0. Piglit has some, but a full piglit-run takes significantly more time, which makes it tricky for CI. So right now, it seems the OpenGL ES 2.0 conformance tests are our best bet. We'll of course add more test-suites as we add more features.

So, there's some work to be done here, but it seems like we should be able to get something working without too much hassle.

Next steps

After Zink is upstream, it will be maintained similarly to other Mesa drivers. Practically speaking, this means that patches are sent to the upstream repo rather than my fork. It shouldn't make a huge difference for most users.

The good thing is that if I go away on vacation, or are for some other reason unavailable, other people can still merge patches, so we'd slightly reduce the technical bus-factor.

I'm not stopping developing Zink at all, but I have other things going on in my life that means I might be busy with other things at times. As is the case for everyone! :wink:

In fact, I'm very excited to start working on OpenGL 3.x and 4.x level features; we still have a few patches for some 3.x features in some older branches.

The future is bright! :sunny:

24 Oct 2019 2:22pm GMT