fidzu : gnome

This is an extract of an email I just sent internally at Red Hat that I wanted to share with the wider GNOME and Freedesktop community.

After 6+ wonderful years at Red Hat, I've decided to hang the fedora to go and try new things. For a while I've been craving for a new challenge and I've felt the urge to try other things outside of the scope of Red Hat so with great hesitation I've finally made the jump.

I am extremely proud of the work done by the teams I have had the honour to run as engineering manager, I met wonderful people, I've worked with extremely talented engineers and learned lots. I am particularly proud of the achievements of my latest team from increasing the bootloader team and improving our relationship with GRUB upstream, to our wins at teaching Lenovo how to do upstream hardware support to improvements in Thunderbolt, Miracast, Fedora/RHEL VirtualBox guest compatibility… the list goes on and credit goes mostly to my amazing team.

Thanks to this job I have been able to reach out to other upstreams beyond GNOME, like Fedora, LibreOffice, the Linux Kernel, Rust, GRUB… it has been an amazing ride and I've met wonderful people in each one of them.

I would also like to make a special mention to my manager, Christian Schaller, who has supported me all the way in several ways both professionally and personally. There is this thing people say: "people do not leave companies, they leave managers". Well this is certainly not the case, in Christian I have found not only a great manager but a true friend.

As for my experience at Red Hat, I have never lasted more than 2 years in the same spot before, I truly found my place there, deep in my heart I know I will always be a Red Hatter, but there are some things I want to try and learn elsewhere. This job switch has been the hardest departure I ever had and in many ways it breaks my heart to leave. If you are considering joining Red Hat, do not hesitate, there is no better place to write and advocate for Free Software.

I will announce what I will be doing next once I start in December.

20 Nov 2019 6:22pm GMT

I was lucky enough to be sponsored by the GNOME Foundation to attend the 2019 Linux Application Summit, hosted in Barcelona between November 12th and 15th 2019.

It was a great conference with a diverse crew of people who all care about making apps on Linux better. I particularly enjoyed Frank's keynote on Linux apps from the perspective of Nextcloud, an Actual ISV. Also worth your time is Rob's talk on how Flathub would like to help more developers earn money from their work; Adrien on GTK and scalable UIs for phones; Robin on tone of voice and copywriting; Emel on Product Management in the context of GNOME Recipes and Paul Brown on direct language and better communication. There were also great lightning talks including a starring turn by one of my former colleagues Martin Abente Lahaye who showed off the work he's been doing to make the Sugar educational applications more widely available with Flatpak. After a bit of review and some polish in the cafe they're now starting to appear on Flathub. All of these videos are available to watch in the YouTube livestream playback, and I'm sure individually soon when appropriately processed.

I gave a talk entitled Product Management In Open Source. Astute readers will recognise the title from the similar talk I gave last year at GUADEC, however the content is actually fairly different. Emel's talk that I mentioned above covered quite a lot of the basic material so I concentrated more on how individual app developers could use Product Management techniques to make their own practice a bit more deliberate and help them guide and prioritize their work.

If you'd like to watch the video then you're more than welcome, although I must warn you that I managed to lose my voice a couple of days before so my delivery isn't exactly what I would wish for. Still, I think you'll find it worthwhile.

19 Nov 2019 10:25pm GMT

There is a big discussion ongoing in the C++ world about ABI stability. People want to make a release of the standard that does a big ABI break, so a lot of old cruft can be removed and made better. This is a big and arduous task, which has a lot of "fun" and interesting edge, corner and hypercorner cases. It might be interesting to look at some of the lesser known ones (this post is not exhaustive, not by a long shot). All information here is specific to Linux, but other OSs should be roughly similar.

The first surprising thing to note is that nobody really cares about ABI stability. Even the people who defend stable ABIs in the committee do not care about ABI stability as such. What they do care about is that existing programs keep on working. A stable ABI is just a tool in making that happen. For many problems it is seemingly the only tool. Nevertheless, ABI stability is not the end goal. If the same outcome can be achieved via some other mechanism, then it can be used instead. Thinking about this for a while leads us to the following idea:

Since "C++ness" is just linking against libstdc++.so, could we not create a new one, say libstdc++2.so, that has a completely different ABI (and even API), build new apps against that and keep the old one around for running old apps?

The answer to this questions turns out to be yes. Even better, you can already do this today on any recent Debian based distribution (and probably most other distros too, but I have not tested). By default the Clang C++ compiler shipped by the distros uses the GNU C++ standard library. However you can install the libc++ stdlib via system packages and use it with the -stdlib=libc++ command line argument. If you go even deeper, you find that the GNU standard library's name is libstdc++.so.6, meaning that it has already had five ABI breaking updates.

So … problem solved then? No, not really.

Problem #1: the ABI boundary

Suppose you have a shared library built against the old ABI that exports a function that looks like this:

void do_something(const std::unordered_map<int, int> &m);

If you build code with the new ABI and call this function, the bit representation of the unordered map causes problems. The caller has a pointer to a bunch of bits in the new representation whereas the callee expects bits in the old representation. This code compiles and links but will invoke UB at runtime when called and, at best, crash your app.

Problem #2: the hidden symbols

This one is a bit complicated and needs some background information. Suppose we have a shared library foo that is implemented in C++ but exposes a plain C API. Internally it makes calls to the C++ standard library. We also have a main program that uses said library. So far, so good, everything works.

Let's add a second shared library called bar that also implemented in C++ and exposes a C API. We can link the main app against both these libraries and call them and everything works.

Now comes the twist. Let's compile the bar library against a new C++ ABI. The result looks like this:

A project mimicing this setup can be obtained from this Github repo. In it the abi1 and abi2 libraries both export a function with the same name that returns an int that is either 1 or 2. Libraries foo and bar check the return value and print a message saying whether they got the value they were expecting. It should be reiterated that the use of the abi libraries is fully internal. Nothing about them leaks to the exposed interface. But when we compile and run the program that calls both libraries, we get the following output snippet:

Foo invoked the correct ABI function.
Bar invoked the wrong ABI function.

What has happened is that both libraries invoked the function from abi1, which means that in the real world bar would have crashed in the same way as in problem #1. Alternatively both libraries could have called abi2, which would have broken foo. Determining when this happens is left as an exercise to the reader.

The reason this happens is that the functions in abi1 and abi2 have the same mangled name and the fact that symbol lookup is global to a process. Once any given name is determined, all usages anywhere in the same process will point to the same entity. This will happen even for non-weak symbols.

Can this be solved?

19 Nov 2019 9:05pm GMT

The Lenovo Thinkpad 8 and also the Asus T100CHI both have an USB3 micro-B connector, but using a standard USB3 OTG (USB 3 micro-B to USB3-A receptacle) cable results in only USB2 devices working. USB3 devices are not recognized.

Searching the internet learns that many people have this problem and that the solution is to find a USB3 micro-A to USB3-A receptacle cable. This sounds like nonsense to me as micro-B really is micro-AB and is supposed to use the ID pin to automatically switch between modes dependent on the used cable.; and this does work for the USB-2 parts of the micro-B connector on the Thinkpad. Yet people do claim success with such cables (with a more square micro-A connector, instead of the trapezoid micro-B connector). The only problem is such cables are not for sale anywhere.

So I guessed that this means is that they have swapped the Rx and Tx superspeed pairs on the USB3 only part of the micro-B connector, and I decided to cut open one of my USB3 micro-A to USB3-A receptacle cables and swap the superspeed pairs. Here is what the cable looks like when it it cut open:



If you are going to make such a cable yourself, to get this picture I first removed the outer plastic isolation (part of it is still there on the right side in this picture). Then I folded away the shield wires till they are all on one side (wires at the top of the picture). After this I removed the metal foil underneath the shield wires.

Having removed the first 2 layers of shielding this reveals 4 wires in the standard USB2 colors: red, black, green and white. and 2 separately shielded cable pairs. On the picture above the separately shielded pairs have been cut, giving us 4 pairs, 2 on each end of cable; and the shielding has been removed from 3 of the 4 pairs, you can still see the shielding on the 4th pair.

A standard USB3 cable uses the following color codes:

• 
  
• Red: Vbus / 5 volt
• 
  
• White: USB 2.0 Data -
• 
  
• Green: USB 2.0 Data +
• 
  
• Black: Ground
• 
  
• Purple: Superspeed RX -
• 
  
• Orange: Superspeed RX +
• 
  
• Blue: Superspeed TX -
• 
  
• Yellow: Superspeed TX -
• 
  

So to swap RX and TX we need to connect purple to blue / blue to purple and orange to yellow / yellow to orange, resulting in:



Note the wires are just braided together here, not soldered yet. This is a good moment to carefully test the cable. Also note that the superspeed wire pairs must be length matched, so you need to cut and strip all 8 cables at the same length! If everything works you can put some solder on those braided together wires, re-test after soldering, and then cover them with some heat-shrink-tube:



And then cover the entire junction with a bigger heat-shrink-tube:



And you have a superspeed capable cable even though no one will sell you one.

Note that the Thinkpad 8 supports ACA mode so if you get an ACA capable "Y" cable or an ACA charging HUB then you can charge and use the Thinkpad 8 USB port at the same time. Typically ACA "Y" cables or hubs are USB2 only. So the superspeed mod from this blogpost will not help with those. The Asus T100CHI has a separate USB2 micro-B just for charging, so you do not need anything special there to charge + connect an USB device.

15 Nov 2019 8:10pm GMT

It has been a while since my last post, but there is a simple reason for that: on August 5th, I had to move from Brazil to Canada. Why did I move? Thanks to Harry Wentland recommendation, I got an interview for a software engineering position at AMD (Markham), and I got hired to work on the display team. From now on, I suppose that I'll be around the DRM subsystem for a long time :). Even though I'm now employed by AMD this post reflect my personal thoughts only and should not be construed to represent AMD in any way.

I have a few updates about my work with the community since I have been busy with my relocation and adaptation. However, my main updates came from XDC 2019 [1] and I want to share it here.

XDC 2019 - Montréal (Concordia University Conference)

This year I had the great luck joining XDC again. However, at this time, I was with Harry Wentland, Nicholas Kazlauskas, and Leo Li (we worked together at AMD). We put effort into learning from other people's experiences, and we tried to know what the compositor developers wanted to see in our driver. We also used this opportunity to try to explain a little bit more about our hardware features. In particular, we had conversations about Freesync, MST, DKMS, and so forth. Thinking of that, I'll share my view of the most exciting moments that we had.

VKMS

As usual, I tried my best to understand what people want to see in VKMS soon or later. For example, from the XDC 2018, I focused on fixing some bugs but mainly in add writeback support cause it could provide a visual output (this work is almost done, see [2]). This year I collected feedback from multiple people (special thanks to Marten, Lyude, Hiler, and Harry), and from these conversations I tend to work in the following tasks:

1. Finish the writeback feature and enable visual output;
2. Add support for adaptive refresh rate;
3. Add support for "dynamic connectors", which can enable the MST test.

Additionally, Martin Peres gave a talk that he shared his views for the CI and test. In his presentation, he suggested using VKMS to validate the API, and I have to admit that I'm really excited about this idea. I hope that I can help with this.

Freesync

The amdgpu drivers support a technology named Freesync [3]. In a few words, this feature allows the dynamic change of the refreshes rate, which can bring benefits for games and for power saving. Harry Wayland talked about that feature and you can see it here:

After Harry's presentation, many people asked interesting questions related to this subject, this situation caught my attention, and for this reason, I added the VRR to my roadmap in the VKMS. Roman Gilg, from KDE, was one of the developers that asked for a protocol extension in Wayland for support Freesync; additionally, compositor developers asked for mechanisms that enable them to know in advance if the experience of a specific panel will be good or not. Finally, there were some discussions about the use of Freesync for power saving and in an application that requires time-sensitive.

IGT and CI

This year I kept my tradition of asking thousands of questions to Hiler with the goal of learning more about IGT, and as usual, he was extremely kind and gentle with my questions (thanks Hiler). One of the concepts that Hiler explained to me, it is the use of podman (https://podman.io/) for prepare IGT image, for example, after a few minutes of code pair with him I could run IGT on my machine after executing the following commands:

sudo su
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master \
                        igt_runner -t core_auth
podman run --privileged registry.freedesktop.org/drm/igt-gpu-tools/igt:master \
                        igt_runner -t core_auth /tmp
podman run --privileged -v /tmp/results:/results \
  registry.freedesktop.org/drm/igt-gpu-tools/igt:master igt_runner -t core_auth /results

We also had a chance to discuss CI with Martin Peres, and he explained his work for improving the way that the CI keep track of bugs. In particular, he introduced a fantastic tool named cibuglog, which is responsible for keep tracking of test failures and using this data for building a database. Cibuglog has many helpful filters that enable us to see test problems associated with a specific machine and bugs in the Bugzilla. The huge insights from cibuglog it is the idea of using data for helping with bug tracking. Thanks Martin, for showing us this amazing tool.

Updates

I just want to finish this post with brief updates from my work with free software, starting with kw and finish with VKMS.

Kernel Workflow (kw)

When I started to work with VKMS, I wrote a tool named kworkflow, or simply kw, for helping me with basic tasks related to Kernel development. These days kw reborn to me since I was looking for a way to automate my work with amdgpu; as a result, I implemented the following features:

• Kernel deploy in a target machine (any machine reachable via IP);
• Module deploy;
• Capture .config file from a target machine;

Unfortunately, the code is not ready for merging in the main branch, I'm working on it; I think that in a couple of weeks, I can release a new version with these features. If you want to know a little bit more about kw take a look at https://siqueira.tech/doc/kw/

VKMS

I was not working in VKMS due to my change of country; however, now I am reworking part of the IGT test related to writeback, and as soon as I finish it, I will try to upstream it again. I hope that I can also have the VKMS writeback merged into the drm-misc-next by the end of this month. Finally, I merged the prime supported implemented by Oleg Vasilev (huge thanks!).

References

[1] "First discussion in the Shayenne's patch about the CRC problem". URL: https://xdc2019.x.org. ⤴

[2] "Introduces writeback support". URL: https://patchwork.freedesktop.org/series/61738/. ⤴

[3] "FreeSync". URL: https://en.wikipedia.org/wiki/FreeSync. ⤴

10 Nov 2019 2:00am GMT

As mentioned in an earlier blogpost, I have been working on fixing many games showing a small image centered on a black background when they are run fullscreen under Wayland. In that blogpost I was moslty looking at how to solve this for native Wayland games. But for various reasons almost all games still use X11, so instead I've ended up focussing on fixing this for games using Xwayland.

Xwayland now has support for emulating resolution changes requested by an app through the randr or vidmode extensions. If a client makes a resolution change requests this is remembered and if the client then creates a window located at the monitor's origin and sized to exactly that resolution, then Xwayland will ask the compositor to scale it to fill the entire monitor.

For apps which use _NET_WM_FULLLSCREEN (e.g. SDL2, SFML or OGRE based apps) to go fullscreen some help from the compositor is necessary. This is currently implemented in mutter. If you are a developer of another compositor and have questions about this, please drop me an email.

I failed to get this all upstream in time for Fedora 31 final. But now it is all upstream, so 've backported the changes and created an update with the changes. This update is currently in updates-testing, to install this update run the following command:

sudo dnf upgrade --enablerepo=updates-testing --advisory=FEDORA-2019-103a594d07

05 Nov 2019 8:41am GMT