21 Aug 2017

feedPlanet GNOME

Zeeshan Ali: Ich bin ein Berliner

Well, no, not really but maybe I'll be able to claim that at some point because I'm moving to Berlin to join Kinvolk. I'm told that I'm changing countries and companies too often but that's not true. I was at Red Hat for 5 years and at Nokia before that for 5 years as well. The decision to move out of Finland was not exactly mine.

Regarding Pelagicore


I'm not as much leaving Pelagicore as I'm leaving the automotive industry, more specifically the software side of it. While the automotive industry is changing and mostly for the good, I realized that it still is not a place for me. Things typically move very slowly in this industry and I realized that I don't have the required patience for it. Also, C++/Qt are big here and while an year ago I thought it's just another language and Open Source UI framework, I no longer think so. Since you can find a lot of rants from very experienced C++ developers on why C++ is a horrible language, I won't rant about that in here.

My experience with Qt hasn't been that great either. Most of the documentation I found would simply assume you use Qt Creator and despite my years of experience with D-Bus, it took me weeks to figure out how to make a few D-Bus calls from Qt (I have to admit though that the calls involved complex types). While Nokia made Qt a normal Open Source project by relicensing it under LGPLv2, the Qt company recently realized that it's loosing a lot of money by people using Qt in products without paying them anything so they relicensed it under GPLv3 and commercial (i-e dual-license). I maintained Genivi Development Platform for 6 months and because of this relicensing, we were unable to upgrade Qt5 to Qt6 and overtime it had been becoming a major pain point in that project. To make things even less open-sourcy, they require all contributors to sign a CLA. I believe (and I think many would agree) that CLAs are bad for Open Source. So all those things put together, I don't think of Qt as a typical Open Source project. Feel free to disagree but that's how I feel and hence I'm not keen on working with Qt in future.

Having said all that, Pelagicore is a very awesome company and probably the best place to be if you're fine with C++/Qt and want to be part of the next-gen automotive. It might sound like I just contradicted myself but not everyone thinks and sees the world like me. To each, his own and all. Also Pelagicore is hiring!


Why leave Gothenburg?


Gothenburg is a very lovely city and I'm going to miss it a lot for sure, even though I've been only here for an year. I still love the Swedish language, which I have been learning slowly over the year. However, I've not been happy with the cost of living in here, especially Veterinary costs. I have an old cat with multiple conditions so I need to visit the Vet every few months. The Vet charge 700 SEK just to see him and most often they don't even care to read through his records beforehand.

Gothenburg is also currently the best place to find an accommodation in. To get a first-hand contract, you register yourself in on Boplats website and keep on applying to new listings but typical wait-time is in years, not months or weeks. In practice, it's usually not a problem. Most people just get a second-hand contract or a room in a shared flat to start with and then look for a more permanent solution. However, add a cat into the picture and things get very difficult again.

Kinvolk comes along


Because of the reasons stated above, I've been looking for some exciting opportunities outside automotive world in some nice location. I had been focused on finding jobs that either involve Rust language or at least there were good chances of Rust being involved. Long story short, I ultimately got in touch with Kinvolk folks. I already knew the company's founders: Chris, Alban and Iago. They are very good at what they do and fun folks to hang out with.

While Rust is not a big part of work at Kinvolk currently, they (especially Chris) seem very interested in it. From what I know, main languages at Kinvolk are C and Go. I don't mind coding in C anyway and I've been missing the times when I did any kernel programming in it. I've no experience of Go but from what I hear, it's a pretty decent language.

So after interviews etc, when Kinvolk offered me a job, I couldn't resist accepting it. Berlin is an awesome city and it's hard to say "no" to moving there.

If you're looking for a great place to work at on Linux-related tech with some very competent Open Source developers, do consider applying at Kinvolk.

Cellink


Talking of great companies to work at, I recently got in contact with some folks from Cellink. It's a biotech company based in Gothenburg, whose aim is to end animal testing. They plan to achieve this very noble goal through 3D printers that print human tissues. They already have 2 products that they sell to pharmaceutical companies in 30 countries across the globe. While they already have the bio and hardware side of things covered, they are looking to expand their software side of things now and to do that, they need good software engineers, especially ones with Open Source experience.

Here is a video of a very nice intro to their awesome work from their CEO, Erik Gatenholm.

So if you're an Open Source developer and either live in or willing to relocate to (especially) Gothenburg, Cambridge (USA) or Bay Area, please contact me and I can connect you to the right people. Alternatively, feel free to contact them directly. I only want to help these folks achieve their very noble cause.

21 Aug 2017 7:56pm GMT

Armin Krezović: The GSoC wrap-up

And so, another Summer of Code has ended. During the past three months, I was working on Mutter, a GNOME's Compositor. My main goal was to make Mutter, which was written as an X11 Window Manager more than a decade ago and recently ported to also be a Wayland compositor, start Wayland session without requiring X server (Xwayland) to be present. That goal was accomplished, and Mutter is now able to start without Xwayland being started. Other goals included making Wayland clients start without X11 being present, which wasn't accomplished (or even started), because after main goal was finished, there was only a week left before this evaluation, and nothing could be accomplished in that time, given the complexity of the task.

So, lets sum what needed to be done during these past three months:

- First task that I worked on was splitting X11 specific part from main display object, MetaDisplay. That code was moved into a new object, MetaX11Display, which is owned by MetaDisplay and is either created or not created at startup, depending on how Mutter was started. A more detailed explanation can be found at [1].

- Second task involved getting rid of MetaScreen object, an object which previously had more than one instance, that nowadays has only one. MetaScreen contained both X11 and non-X11 specific parts, so it was split between MetaDisplay and MetaX11Display, accordingly. A more detailed explanation can be found at [1] and [2].

- Third set of tasks (yes, set of tasks) involved getting X11 specific functionality out of objects managed by MetaDisplay. These include workspace management, startup notification, stack tracker and bell (visual/sound notification). A more detailed explanation can be found at [3] and [4].

- Final task involved getting rid of GDK usage early in the code, and moving control of GDK X11 display to MetaX11Display itself. This also involved (sadly) getting rid of some GtkSettings usage which was used to change some properties when it gets changed in GTK. A more detailed explanation can be found at [4].

- After the final task was done, a --no-x11 switch was added to Mutter command line, which can be used to start Mutter without starting Xwayland.

- Bonus tasks, which did not really need to be done in order for Mutter to be able to work without X11 included renaming and improving X11 error management code (x11 trap code), moving workspace management code from MetaDisplay (previously MetaScreen) into new object - MetaWorkspaceManagement, and getting rid of screen size tracking in MetaDisplay (again, previously MetaScreen), and utilizing MetaMonitorManager directly. A more detailed explanation can be found at [3] and [4].

During this effort, a lot of functions were removed or replaced by functions with different names/parameters. Any compositor using libmutter library (gnome-shell is its most known user) cannot be compiled against my Mutter branch. However, I did create an API changelog which can be found at [5], for those that want to make use of my work, which can now be found at [6], and is rebased against current git master, as of moment of writing. Instructions to try out the compositor code can be found at [4], as well.

This summarizes my work done over these past three months. I'd like to thank both of my mentors, Jonas and Carlos, for helping me when I got stuck, providing feedback whenever it was necessary, or answering calmly and kindly at any of my questions, however simple or stupid they might sound. Also, I'd like to thank Google and GNOME Foundation for accepting me as student developer and allowing me to participate in this year's Summer of Code.

[1] https://armin-gnome.blogspot.ba/2017/06/progress-report-for-period-june-12th.html
[2] https://armin-gnome.blogspot.ba/2017/07/progress-report-june-26th-july-9th.html
[3] https://armin-gnome.blogspot.ba/2017/07/progress-report-july-10th-july-23rd.html
[4] https://armin-gnome.blogspot.ba/2017/08/progress-report-july-31st-august-13th.html
[5] https://armin-gnome.blogspot.ba/2017/08/documented-mutter-api-changes-after.html
[6] https://github.com/krezovic/mutter/commits/gsoc-final

21 Aug 2017 5:44pm GMT

20 Aug 2017

feedPlanet GNOME

Lucie Charvat: GSoC/GUADEC: Wrapping Things Up

The Google Summer of Code is slowly but surely coming to an end and it's time to start wrapping thing up for the final evaluation. The documentation cards have been officially pushed to the master of the GNOME Builder and last couple of days were spent just tweaking the feature and going through the code reviews.

I would also like to take a quick look back at the amazing GUADEC that was held in Manchester this summer and share some of my photos. I was so glad I could attend and connect the faces with the people I have only met online.

1DSC_5859

GUADEC kicked of at MMU's Birley Campus with series of talks varying from technical updates of tools to strengthening the community and the principles which it stands for.

4DSC_5869

The social calendar was packed as well, making it very easy to meet and get to know a bit all the amazing people who have been part of the for years. Including, for me the highpoint of the events, the GNOME's 20th birthday celebration held in Museum of Science and Industry. If you haven't yet, don't forget to checkout the GNOME's birthday page.

The last days in Manchester were spent in nearby Shed providing space to discuss and work on ideas surrounding GNOME.

Hope to see you all next year!

asponsored-badge-simple.png


20 Aug 2017 6:26pm GMT

19 Aug 2017

feedPlanet GNOME

Jussi Pakkanen: Apple laptops have become garbage

When OSX launched it quite quickly attracted a lot of Linux users and developers. There were three main reason for this:

  1. Everything worked out of the box
  2. The hardware was great, even sexy
  3. It was a full Unix laptop

It is interesting, then, that none of these things really hold true any more.

Everything works out of the box

I have an Android phone. One of the things one would like to do with it is to take pictures and then transfer them to a computer. On Linux and Windows this is straightforward: you plug in the USB cable, select "share pictures" on the phone and the operating system pops up a file dialog. Very simple.

In OSX this does not work. Because Android is a competitor to the iPhone (which makes Apple most of its money nowadays) it is in Apple's business interest to not work together with competing products. They have actively and purposefully chosen to make things worse for you, the paying customer, for their own gain. Google provides a file transfer helper application but since it is not hooked inside the OS its UX is not very good.

But let's say you personally don't care about that. Maybe you are a fully satisfied iPhone user. Very well, let's look at something completely different: external monitors. In this year's Europython conference introductory presentation the speaker took the time to explicitly say that if anyone presenting had a latest model Macbook Pro, it would not work with the venue's projectors. Things have really turned on their heads because up to a few years ago Macs were pretty much the only laptops that always worked.

This problem is not limited to projectors. At home I have an HP monitor that has been connected to many a different video source and it has worked flawlessly. The only exception is the new work laptop. Connecting it to this monitor makes the system go completely wonky. On every connection it does an impressive impersonation of the dance floor of a german gay bar with colors flickering, things switching on and off and changing size for about ten seconds or so. Then it works. Until the screen saver kicks in and the whole cycle repeats.

If this was not enough every now and then the terminal application crashes. It just goes completely blank and does not respond to anything. This is a fairly impressive feat for an application that reached feature stability in 1993 or thereabouts.

Great hardware

One of the things I do in my day job is mobile app development (specifically Android). This means connecting external display, mouse and keyboard to the work laptop. Since macs have only two USB ports they are already fully taken and there is nowhere to plug the development phone. The choices here are to either unplug the mouse whenever you need to deploy or debug on the device or use a USB hub.

Using dongles for connectivity is annoying but at least with a hub one can get things working. Except no. I have a nice USB hub that I have used for many years on many devices that works like a charm. Except on this work computer. Connecting anything through that hub causes something to break so the keyboard stops working every two minutes. The only solution is to unplug the hub and then replug it again. Or, more specifically, not to use the hub but instead live without an external mouse. This is even more ridiculous when you consider that Apple was the main pioneer for driving USB adoption back in the day.

Newer laptop models are even worse. They have only USB-C connectors and each consecutive model seems to have fewer and fewer of them. Maybe their eventual goal is to have a laptop with no external connection slots, not even a battery charger port. The machine would ship from the factory pre-charged and once the juice runs out (with up to 10 hours of battery life™) you have to throw it away and buy a new one. It would make for good business.

After the introduction of the Retina display (which is awesome) the only notable hardware innovation has been the emojibar. It took the concept of function buttons and made it worse.

Full Unix support

When OSX launched it was a great Unix platform. It still is pretty much the same it was then, but by modern standards it is ridiculously outdated. There is no Python 3 out of the box, and Python 2 is several versions behind the latest upstream release. Other tools are even worse. Perl is 5.18 from 2014 or so, Bash is 3.2 with the copyright year of 2007, Emacs from 2014 and Vim from 2013. This is annoying even for people who don't use macs, but just maintain software that supports OSX. Having to maintain compatibility with these sorts of stone age tools is not fun.

What is causing this dip in quality?

There are many things one could say about the current state of affairs. However there is already someone who has put it into words much more eloquently than any of us ever could. Take it away, Steve:

Post scriptum

Yes, this blog post was written on a Macbook, but it is one of the older models which were still good. I personally need to maintain a piece of software that has native support for OSX so I'm probably going to keep on using it for the foreseeable future. That being said if someone starts selling a laptop with a Risc-V processor, a retina-level display and a matte screen, I'm probably going to be first in line to get one.

19 Aug 2017 11:26pm GMT

18 Aug 2017

feedPlanet GNOME

Yash Singh: My first (and definitely not the last) GUADEC!

Hey folks!

I recently attended GNOME Users and Developers European Conference (GUADEC) 2017 held in Manchester, UK. It was my first time in the UK and my first time at a conference and needless to say, I had a wonderful time.

Core Conference Days

The core conference days were held at the Brooks Building in Manchester Metropolitan University. I attended a lot of great talks. Some of the talks I found helpful were-

I gave a lighting intern talk presenting my GSoC project - Adding recurring events support to GNOME Calendar. My talk was accompanied by a live demonstration too!

IMG_20170729_142019.jpg

Unconference Days

The Unconference days were held at the Shed (MMU). There I took part in various workshops and Birds of Feather (BoF) sessions. I had a sit-down with my mentor, Georges Stavacras, and we discovered and fixed (not all) quite a lot of bugs in GNOME Calendar.

Social Events

20170729_204433.jpg

Lots of social events and fun activities were organised. The GNOME 20th Anniversary party was one of the best parties I've been to yet. The decorations and ambiance were one of a kind. The newcomers dinner was also very engaging. We also played indoor football at the MMU sports hall and lots of other fun games. And let's not forget all the local pubs we visited throughout the conference. We also had a walking tour of Manchester which was very informative.

IMG_20170802_161648.jpg

IMG_20170802_171646.jpg

The Peak District Hike was an altogether different experience. We went on a 9km Trek on Loose Hill. The trek was so scenic and beautiful! I sincerely thank Allan Day for organizing and leading that awesome trek.

IMG_20170801_205906.jpg

I was also a part of the volunteering team and made lots of friends on the team. The 'hosts' from Codethink were a very friendly bunch of people with whom I had a great time. Kudos to Sam Thursfield who took me rock climbing to the Manchester Climbing Centre.

It was really exciting to finally meet the GNOME contributors that I had previously chatted with online. Meeting my mentor Georges was really fun. I thank him for the work he's put in mentoring my GSoC project.

I would like to thank the GNOME Foundation for sponsoring me to attend GUADEC. It would not have been possible for me to attend the event without the financial aid provided by the GUADEC travel committee. Thank you all the organisers, volunteers and attendees for making this year's GUADEC successful. See you next year at Almeria!

sponsored-badge-simple.png


18 Aug 2017 10:25pm GMT

Matthias Clasen: Post-GUADEC distractions

Like everybody else, I had a great time at GUADEC this year.

One of the things that made me happy is that I could convince Behdad to come, and we had a chance to finally wrap up a story that has been going on for much too long: Support for color Emoji in the GTK+ stack and in GNOME.

Behdad has been involved in the standardization process around the various formats for color glyphs in fonts since the very beginning. In 2013, he posted some prototype work for color glyph support in cairo.

This was clearly not meant for inclusion, he was looking for assistance turning this into a mergable patch. Unfortunately, nobody picked this up until I gave it a try in 2016. But my patch was not quite right, and things stalled again.

We finally picked it up this year. I produced a better cairo patch, which we reviewed, fixed and merged during the unconference days at GUADEC. Behdad also wrote and merged the necessary changes for fontconfig, so we can have an "emoji" font family, and made pango automatically choose that font when it finds Emoji.

After guadec, I worked on the input side in GTK+. As a first result, it is now possible to use Control-Shift-e to select Emoji by name or code.

https://blogs.gnome.org/mclasen/files/2017/08/c-s-e.webm

This is a bit of an easter egg though, and only covers a few Emoji like ❤. The full list of supported names is here.

A more prominent way to enter Emoji is clearly needed, so i set out to implement the design we have for an Emoji chooser. The result looks like this:

As you can see, it supports variation selectors for skin tones, and lets you search by name. The clickable icon has to be enabled with a show-emoji-icon property on GtkEntry, but there is a context menu item that brings up the Emoji chooser, regardless.

I am reasonably happy with it, and it will be available both in GTK+ 3.92 and in GTK+ 3.22.19. We are bending the api stability rules a little bit here, to allow the new property for enabling the icon.

Working on this dialog gave me plenty of opportunity to play with Emoji in GTK+ entries, and it became apparent that some things were not quite right. Some Emoji just did not appear, sometimes. This took me quite a while to debug, since I was hunting for some rendering issue, when in the end, it turned out to be insufficient support for variation selectors in pango.

Another issue that turned up was that pango did place the text caret in the middle of Emoji's sometimes, and Backspace deleted them piece-meal, one character at a time, instead of all at once. This required fixes in pango's implementation of the Unicode segmentation rules (TR29). Thankfully, Peng Wu had already done much of the work for this, I just fixed the remaining corner cases to handle all Emoji correctly, including skin tone variations and flags.

So, what's still missing ? I'm thinking of adding optional support for completion of Emoji names like :grin: directly in the entry, like this:

https://blogs.gnome.org/mclasen/files/2017/08/emoji-completion.webm

But this code still needs some refinement before it is ready to land. It also overlaps a bit with traditional input method functionality, and I am still pondering the best way to resolve that.

To try out color Emoji, you can either wait for GNOME 3.26, which will be released in September, or you can get:

It was fun to work on this, I hope you enjoy using it! ❤

18 Aug 2017 9:38pm GMT

Richard Hughes: Shipping PKCS7 signed metadata and firmware

Over the last few days I've merged in the PKCS7 support into fwupd as an optional feature. I've done this for a few reasons:

Did you know GPGME is a library based around screen scraping the output of the gpg2 binary? When you perform an action using the libgpgme APIs you're literally injecting a string into a pipe and waiting for it to return. You can't even use libgcrypt (the thing that gpg2 uses) directly as it's way too low level and doesn't have any sane abstractions or helpers to read or write packaged data. I don't want to learn LISP S-Expressions (yes, really) and manually deal with packing data just to do vanilla X509 crypto.

Although the LVFS instance only signs files and metadata with GPG at the moment I've added the missing bits into python-gnutls so it could become possible in the future. If this is accepted then I think it would be fine to support both GPG and PKCS7 on the server.

One of the temptations for X509 signing would be to get a certificate from an existing CA and then sign the firmware with that. From my point of view that would be bad, as any firmware signed by any certificate in my system trust store to be marked as valid, when really all I want to do is check for a specific (or a few) certificates that I know are going to be providing certified working firmware. Although I could achieve this to some degree with certificate pinning, it's not so easy if there is a hierarchical trust relationship or anything more complicated than a simple 1:1 relationship.

So this is possible I've created a LVFS CA certificate, and also a server certificate for the specific instance I'm running on OpenShift. I've signed the instance certificate with the CA certificate and am creating detached signatures with an embedded (signed-by-the-CA) server certificate. This seems to work well, and means we can issue other certificates (or CRLs) if the server ever moves or the trust is compromised in some way.

So, tl;dr: (should have been at the top of this page…) if you see a /etc/pki/fwupd/LVFS-CA.pem appear on your system in the next release you can relax. Comments, especially from crypto experts welcome. Thanks!

18 Aug 2017 4:36pm GMT

17 Aug 2017

feedPlanet GNOME

Alexandre Franke: 20 years strong

20 years ago, Miguel and Federico created GNOME.

We had an early party during GUADEC at the Manchester Museum of Science and Industry. I organized a local one in Strasbourg yesterday with the help of Marie-France, a student who was already behind the logistics of the Sympa hackfest. It was part of a string of similar events around the world.

Marie-France baked us an awesome cake. We had a love wall, a longtime tradition at GNOME events that the locals weren't used to, but they sure appreciated it!

Stickers that I brought back from GUADEC got dispatched and brought smiles to the faces of attendees.

17 Aug 2017 5:44pm GMT

Petr Kovar: GUADEC 2017 Notes

With GUADEC 2017 and the unconference days over, I wanted to share a few conference and post-conference notes with a broader audience.

First of all, as others have reported, at this year's GUADEC, it was great to see an actual increase in numbers of attendees compared to previous years. This shows us that 20 years later, the community as a whole is still healthy and doing well.

At the conference venue.

While the Manchester weather was quite challenging, the conference was well-organized and I believe we all had a lot of fun both at the conference venue and at social events, especially at the awesome GNOME 20th Birthday Party. Kudos to all who made this happen!

At the GNOME 20th Birthday Party.

As I reported at the GNOME Foundation AGM, the docs team has been slightly more quiet recently than in the past and we would like to reverse this trend going forward.

At the GNOME 20th Birthday Party.

At the closing session.

Last but not least, I'd like to say thank you to the GNOME Foundation and the Travel Committee for their continuous support, for sponsoring me again this year.

17 Aug 2017 5:08pm GMT

Mandy Wang: Went to COSCUP 2017

I joined COSCUP2017 which is held in Taipei from August 5 to 6. 'COSCUP' means Conference for Open Source Coders, Users and Promoters, is an annual conference held by Taiwanese Open source community participants since 2006. It's a major force of Free software movement advocacy in Taiwan.

People of different careers, different ages, different areas and different organizations went there to talk about their experience about FOSS. Especially some organizations such as WoFOSS and PyLadies in Taiwan, they talked about how to make community diversified and how to respect women in a community.

I also gave a speech at this conference about how college women participate in FOSS easily, I shared my experience during Outreachy and GSoC in my talk and encouraged other women to join them. My speech attracted many college women and I hope my story can give them some useful information and more confidence.

IMG_0182


17 Aug 2017 3:41pm GMT

Alexandru Ionut Pandelea: GUADEC 2017

This year is my second one attending GUADEC, this time around, in Manchester. It was a great experience this time too, because I got to meet again with the friends I made last year, but also I got to make new friends as well.

During the core days I attended a lot of great talks and I got to learn cool new things. Among these, I could mention learning about technologies that I didn't know they existed, like Emeus, improve my view about how a good design should look like or discover more about the history of GNOME. Since this year I am a GSoC student again, I also had a lightning talk and I'm happy to say that this year I was slightly less nervous about talking in front of a lot of people.

During the unconference days I had a bit of time to work on my project and talked about it with my mentors, Carlos Soriano and Carlos Garnacho. Also, I got some feedback on the work I did this summer from Allan Day.

Besides that, we also had a lot of fun. For instance, the 20th anniversary party was great and the venue was very cool as well: the Museum of Science and Industry in Manchester. There was a trip to the Peak District, which is quite close to Manchester, and even if at the top it was a bit foggy, we still got some great views.

GUADECFinally, I would like to thank the GNOME Foundation for making it possible for me to attend GUADEC this year and I'm looking forward to the next one, which will be in Almería.

sponsored-badge-shadow


17 Aug 2017 9:55am GMT

Federico Mena-Quintero: Correctness in Rust: building strings

Rust tries to follow the "make illegal states unrepresentable" mantra in several ways. In this post I'll show several things related to the process of building strings, from bytes in memory, or from a file, or from char * things passed from C.

Strings in Rust

The easiest way to build a string is to do it directly at compile time:

let my_string = "Hello, world!";

In Rust, strings are UTF-8. Here, the compiler checks our string literal is valid UTF-8. If we try to be sneaky and insert an invalid character...

let my_string = "Hello \xf0";

We get a compiler error:

error: this form of character escape may only be used with characters in the range [\x00-\x7f]
 --> foo.rs:2:30
  |
2 |     let my_string = "Hello \xf0";
  |                              ^^

Rust strings know their length, unlike C strings. They can contain a nul character in the middle, because they don't need a nul terminator at the end.

let my_string = "Hello \x00 zero";
println!("{}", my_string);

The output is what you expect:

$ ./foo | hexdump -C
00000000  48 65 6c 6c 6f 20 00 20  7a 65 72 6f 0a           |Hello . zero.|
0000000d                    ^ note the nul char here
$

So, to summarize, in Rust:

This is a bit different from C:

Okay, just kidding. In C:

Building a string from bytes

Let's say you have an array of bytes and want to make a string from them. Rust won't let you just cast the array, like C would. First you need to do UTF-8 validation. For example:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
fn convert_and_print(bytes: Vec<u8>) {
    let result = String::from_utf8(bytes);
    match result {
        Ok(string) => println!("{}", string),
        Err(e) => println!("{:?}", e)
    }
}

fn main() {
    convert_and_print(vec![0x48, 0x65, 0x6c, 0x6c, 0x6f]);
    convert_and_print(vec![0x48, 0x65, 0xf0, 0x6c, 0x6c, 0x6f]);
}

In lines 10 and 11, we call convert_and_print() with different arrays of bytes; the first one is valid UTF-8, and the second one isn't.

Line 2 calls String::from_utf8(), which returns a Result, i.e. something with a success value or an error. In lines 3-5 we unpack this Result. If it's Ok, we print the converted string, which has been validated for UTF-8. Otherwise, we print the debug representation of the error.

The program prints the following:

$ ~/foo
Hello
FromUtf8Error { bytes: [72, 101, 240, 108, 108, 111], error: Utf8Error { valid_up_to: 2, error_len: Some(1) } }

Here, in the error case, the Utf8Error tells us that the bytes are UTF-8 and are valid_up_to index 2; that is the first problematic index. We also get some extra information which lets the program know if the problematic sequence was incomplete and truncated at the end of the byte array, or if it's complete and in the middle.

And for a "just make this printable, pls" API? We can use String::from_utf8_lossy(), which replaces invalid UTF-8 sequences with U+FFFD REPLACEMENT CHARACTER:

fn convert_and_print(bytes: Vec<u8>) {
    let string = String::from_utf8_lossy(&bytes);
    println!("{}", string);
}

fn main() {
    convert_and_print(vec![0x48, 0x65, 0x6c, 0x6c, 0x6f]);
    convert_and_print(vec![0x48, 0x65, 0xf0, 0x6c, 0x6c, 0x6f]);
}

This prints the following:

$ ~/foo
Hello
He�llo

Reading from files into strings

Now, let's assume you want to read chunks of a file and put them into strings. Let's go from the low-level parts up to the high level "read a line" API.

Single bytes and single UTF-8 characters

When you open a File, you get an object that implements the Read trait. In addition to the usual "read me some bytes" method, it can also give you back an iterator over bytes, or an iterator over UTF-8 characters.

The Read.bytes() method gives you back a Bytes iterator, whose next() method returns Result<u8, io::Error>. When you ask the iterator for its next item, that Result means you'll get a byte out of it successfully, or an I/O error.

In contrast, the Read.chars() method gives you back a Chars iterator, and its next() method returns Result<char, CharsError>, not io::Error. This extended CharsError has a NotUtf8 case, which you get back when next() tries to read the next UTF-8 sequence from the file and the file has invalid data. CharsError also has a case for normal I/O errors.

Reading lines

While you could build a UTF-8 string one character at a time, there are more efficient ways to do it.

You can create a BufReader, a buffered reader, out of anything that implements the Read trait. BufReader has a convenient read_line() method, to which you pass a mutable String and it returns a Result<usize, io::Error> with either the number of bytes read, or an error.

That method is declared in the BufRead trait, which BufReader implements. Why the separation? Because other concrete structs also implement BufRead, such as Cursor - a nice wrapper that lets you use a vector of bytes like an I/O Read or Write implementation, similar to GMemoryInputStream.

If you prefer an iterator rather than the read_line() function, BufRead also gives you a lines() method, which gives you back a Lines iterator.

In both cases - the read_line() method or the Lines iterator, the error that you can get back can be of ErrorKind::InvalidData, which indicates that there was an invalid UTF-8 sequence in the line to be read. It can also be a normal I/O error, of course.

Summary so far

There is no way to build a String, or a &str slice, from invalid UTF-8 data. All the methods that let you turn bytes into string-like things perform validation, and return a Result to let you know if your bytes validated correctly.

The exceptions are in the unsafe methods, like String::from_utf8_unchecked(). You should really only use them if you are absolutely sure that your bytes were validated as UTF-8 beforehand.

There is no way to bring in data from a file (or anything file-like, that implements the Read trait) and turn it into a String without going through functions that do UTF-8 validation. There is not an unsafe "read a line" API without validation - you would have to build one yourself, but the I/O hit is probably going to be slower than validating data in memory, anyway, so you may as well validate.

C strings and Rust

For unfortunate historical reasons, C flings around char * to mean different things. In the context of Glib, it can mean

What a particular char * means depends on which API you got it from.

Bringing a string from C to Rust

From Rust's viewpoint, getting a raw char * from C (a "*const c_char" in Rust parlance) means that it gets a pointer to a buffer of unknown length.

Now, that may not be entirely accurate:

The Rust standard library provides a CStr object, which means, "I have a pointer to an array of bytes, and I know its length, and I know the last byte is a nul".

CStr provides an unsafe from_ptr() constructor which takes a raw pointer, and walks the memory to which it points until it finds a nul byte. You must give it a valid pointer, and you had better guarantee that there is a nul terminator, or CStr will walk until the end of your process' address space looking for one.

Alternatively, if you know the length of your byte array, and you know that it has a nul byte at the end, you can call CStr::from_bytes_with_nul(). You pass it a &[u8] slice; the function will check that a) the last byte in that slice is indeed a nul, and b) there are no nul bytes in the middle.

The unsafe version of this last function is unsafe CStr::from_bytes_with_nul_unchecked(): it also takes an &[u8] slice, but you must guarantee that the last byte is a nul and that there are no nul bytes in the middle.

I really like that the Rust documentation tells you when functions are not "instantaneous" and must instead walks arrays, like to do validation or to look for the nul terminator above.

Turning a CStr into a string-like

Now, the above indicates that a CStr is a nul-terminated array of bytes. We have no idea what the bytes inside look like; we just know that they don't contain any other nul bytes.

There is a CStr::to_str() method, which returns a Result<&str, Utf8Error>. It performs UTF-8 validation on the array of bytes. If the array is valid, the function just returns a slice of the validated bytes minus the nul terminator (i.e. just what you expect for a Rust string slice). Otherwise, it returns an Utf8Error with the details like we discussed before.

There is also CStr::to_string_lossy() which does the replacement of invalid UTF-8 sequences like we discussed before.

Conclusion

Strings in Rust are UTF-8 encoded, they know their length, and they can have nul bytes in the middle.

To build a string from raw bytes, you must go through functions that do UTF-8 validation and tell you if it failed. There are unsafe functions that let you skip validation, but then of course you are on your own.

The low-level functions which read data from files operate on bytes. On top of those, there are convenience functions to read validated UTF-8 characters, lines, etc. All of these tell you when there was invalid UTF-8 or an I/O error.

Rust lets you wrap a raw char * that you got from C into something that can later be validated and turned into a string. Anything that manipulates a raw pointer is unsafe; this includes the "wrap me this pointer into a C string abstraction" API, and the "build me an array of bytes from this raw pointer" API. Later, you can validate those as UTF-8 and build real Rust strings - or know if the validation failed.

Rust builds these little "corridors" through the API so that illegal states are unrepresentable.

17 Aug 2017 1:26am GMT

Luke Jones: Shooting Your Foot in Rust

I've had a bit of difficulty getting this post done in a decent timeframe as I have 4 papers on the go this semester, one of which I was enrolled for 2.5 weeks late and had to scramble to catch up on - there were some other things I wanted to discuss here but time constraints are pushing those to the next post. Never-the-less, onwards.

Before I am able to use Rust in GJS effectively, I've needed to create FFI bindings for Rust to use to call in to C libraries, such as GLib, GIRepository, and libffi; doing this required the use of both bindgen and gtk-rs/gir.

In both cases, these tools produce unsafe Rust, this is code that does one of;

  1. Dereferencing a raw pointer
  2. Calling an unsafe function or method
  3. Accessing or modifying a mutable static variable
  4. Implementing an unsafe trait </a> For the bindings I am using, the unsafety generally comes from the use of points 1, 2, and 3. And example of this is from the [libffi] bindings (generated using bindgen):
extern "C" {
    pub fn ffi_prep_cif(cif: *mut ffi_cif, abi: ffi_abi,
                        nargs: ::std::os::raw::c_uint, rtype: *mut ffi_type,
                        atypes: *mut *mut ffi_type) -> ffi_status;
}

extern "C" is a marker that tells Rust that the function is called externally - as such it is regarded as an unsafe function - and the function prototype follows. The function takes a variety of arguments; cif: *mut ffi_cif is a raw pointer to a ffi_cif which has the layout of:

#[repr(C)]
#[derive(Debug, Copy)]
pub struct ffi_cif {
    pub abi: ffi_abi,
    pub nargs: ::std::os::raw::c_uint,
    pub arg_types: *mut *mut ffi_type,
    pub rtype: *mut ffi_type,
    pub bytes: ::std::os::raw::c_uint,
    pub flags: ::std::os::raw::c_uint,
}

#[repr(C)] marks this struct definition as having the order, size, and alignment of the same definition in C. This is important for anything being passed over the FFI boundary between Rust & C. There are some small restrictions here, Rust tuples and tagged unions (enum) don't exist in C, and should not ever be passed over the FFI, and drop flags need to be added (drop flags are what Rust uses to free memory).

Raw pointers in Rust are safe, you can copy, move, create, and borrow raw pointers. But when you dereference one, it is classed as unsafe. Why? Rust can't guarantee that the data pointed to is actually valid data - this is a task for the implementor.

You will see ::std::os::raw::c_uint and other variants of c_ types popping up, these are basic data types which are guaranteed to have the same size as its C counterpart.

The other oddity here is pub arg_types: *mut *mut ffi_type. arg_types is a Vector, and the *mut *mut is a mutable raw pointer to the first element in the Vector, which is also a mutable pointer (to an ffi_type). That is, a pointer to an vector of pointers.

If you're curious, ffi_type is defined as;

#[repr(C)]
#[derive(Debug, Copy)]
pub struct _ffi_type {
    pub size: usize,
    pub alignment: ::std::os::raw::c_ushort,
    pub type_: ::std::os::raw::c_ushort,
    pub elements: *mut *mut _ffi_type,
}
pub type ffi_type = _ffi_type;

Making unsafe Safe

Rust is supposed to be a safe langugae, right? It still is, even when you use unsafe code. Using unsafe code doesn't disable all the safety checks, it only enables the use of some extra features which are unsafe (see points 1-4). The caveat is that the unsafe code must be contained within an unsafe { } block to enable these features, and it is up to the programmer to validate these blocks and make sure they actually are safe. If you do end up with problems, eg, leaked memory then you can be sure that the problem lies within an unsafe block.

An example is from my gi-girffi wrapper (this is a translation of the functions in girepository/girffi.c);

pub fn g_callable_info_get_ffi_return_type(callable_info: &mut GICallableInfo) -> Option<ffi_type> {
    let mut return_type;
    unsafe {
        return_type = g_callable_info_get_return_type(callable_info)
            .as_mut() // make the raw pointer a mutable reference
            .unwrap_or(return None);
    }
    Some(g_type_info_get_ffi_type(return_type))
}

You will see that g_callable_info_get_return_type() is the only function within an unsafe block; it is a GLib function which has a signature definition of;

pub fn g_callable_info_get_return_type(info: *mut GICallableInfo) -> *mut GITypeInfo;

it takes a raw mutable pointer to a GICallableInfo (which is a type alias for GIBaseInfo), and returns a raw mutable pointer to a newly allocated GITypeInfo. Since this is a new allocation of memory, and the only reference to it is this pointer, I convert it to a Rust mutable reference using as_mut(). This conversion also checks if the pointer is null, and returns None if so - it doesn't check that the data is valid, however…

return_type is passed on to g_type_info_get_ffi_type(), which I've written in similar fashion as a safe function and takes ownership of it (moved value), and drops it once done with it, we then return the result of that call wrapped in an <Option>.

Regarding g_type_info_get_ffi_type: within this function I've added a manual call to drop return_type via g_free. At some point int he near future I may wrap some things like this with a manually defined Drop trait so that the data is correctly freed when dropped.

But that's a lot of unsafety

Yes, it is. But there are several aspects to all this:

Having said all that, can you shoot your foot in Rust? Absolutely! That is why unsafe code is boxed in with unsafe keywords. If something is hinky, then you know where to start looking. It is up to the implementor to honour the contracts with Rust when producing a safe function that contains unsafe code.

Pain Points

Unions

Note: I'm referencing stable rustc here

The biggest headache I've had so far is purely with deciding how to represent a C union in Rust. It looks like bindgen produces rust code for unions by using a Rust struct and ::std::marker::PhantomData<T>. PhantomData is is used as a marker of sorts in many instances, and in this case it is used to indicate ownership of this data. I don't really understand it very well at this point, more info is available here and here.

Untagged Union support is coming to stable rustc soon, and is available in nightly rust (RFC). I hope it lands in rustc 1.19 so that I can incorporate it in to the gir-ffi bindings, and the gtk-rs project can add support to the gir->rust binding producer. I worked on enabling proper Union bindings in gir and the PR was accepted - ~it is only usable in nightly Rust as of yet, and is behind a feature-gate~.

Update: I am actively working to move the gir union support to default since untagged unions are now stable in Rust as of 1.19

Bit-fields

Bit-fields feature in a few structs within the GNOME libs, and likely in a fair few other libraries. In particular there are a few structs that have mixed data, and these are the ones which don't really have an ideal solution yet. An example of this is;

struct _GHookList
{
  gulong       seq_id;
  guint                hook_size : 16;
  guint                is_setup : 1;
  GHook               *hooks;
  gpointer     dummy3;
  GHookFinalizeFunc finalize_hook;
  gpointer     dummy[2];
};

where hook_size and is_setup are the bit-fields and as such they change the size and alignment of the struct. For now the gir to Rust binding gen is replacing the first instance of a bit-field with ` _truncated_record_marker: c_void` and commenting out the following fields. The programmer is expected to ackowledge which structs are truncated and write their code accordingly. A bit of extra work.

We now need a Rust RFC to be finished off to get C type bit-fields in to Rust (I will be taking this on as soon as I get the time).

Next post?

My next post will be about what I've learned from this project, and may span a few posts as I try and clarify things for myself enough to write about them. In particular I want to highlight the pros/cons of this project, and I want to try and translate what I've learned in Rust back to the C++ codebase - this means analysing use of pointers, switching to unique_ptr and the ownership model it presents, references instead of pointers, and a few other things.

So far this has been an incredibly rewarding project for myself, and I really hope I can share this knowledge in a way that is adequate for others to follow.

17 Aug 2017 1:00am GMT

16 Aug 2017

feedPlanet GNOME

Fabián Orccón: GUADEC 2017

One of the things I like the most about GNOME are the annual conferences called GUADEC. You can see in person to the folks you were chatting with on IRC. As I have mentioned in my previous posts, I am an accepted GSoC 2017 student this year, and thus like other students, I was invited to give a lightning talk about my project with Pitivi, that consist on adding a plugin system as I mentioned in other posts. I live in Peru, so the duration of the flight is really long. Actually, I found a great deal by going first to Madrid and then taking other flight to Manchester. From Lima to Amsterdam (the stop of my flight to Madrid) it was about 12 hours, from Amsterdam to Madrid it was about 3 hours and from Madrid to Manchester about 2 hours. Yup! Almost 17 hours flying. But it was worth it.

I arrived to Manchester on July 27 at 12:20 p.m. and the weather surprised me with a strong rain. It may be more surprising when you live in a city were it does not rain. Then I had to go to the Manchester Metropolitan University where many of the GNOME contributors would be hosted. When the bus stopped on the Manchester Metropolitan University I went to the first building of it I see and asked how to get to the Birley Fields, our accomodations. A man told me the directions and gave me a map. After some minutes walking I got the office that assigns the rooms to the new residents. I met there Mario, a guy from GNOME who is involved in Flatpak. It was interesting to talk with him in English when at the end we realized that we both could speak in Spanish. After leaving my stuff on my room, I left the room to walk outside and I found David King. It was incredible because it was almost three years we didn't see to each other. In that day, I also met hadess (Bastien Nocera). He helped me to get a traveler adapter. This was also the day of pre-registration in a bar called KROBAR. I got joined to the GStreamer folks who I met before in GUADEC 2014. Some of the guys came up with the idea that the GNOME logo needs a new design. I talked about it before on the #gnome-design channel. I also met ystreet00 (Matthew Waters) who helped once to create a GStreamer plugin with OpenGL.

alt text

GNOME stickers

The next day the talks started. In the venue, one of the first things I did was to buy a GNOME t-shirt. One of the talks I was very interested in was the one of Alexander Larsson about Flatpak and the one of Christian Hergert about GNOME Builder. I was very interested in the conference of hergertme because I has taken some of the ideas of GNOME Builder to apply them in Pitivi. I don't always use this application because I am not totally adapted to it, but now I am considering to use it more instead of just coding on Gedit. That day I met Lucie Charvát, a student of the Google Summer of Code who is working in a nice feature of GNOME Builder that I always was thinking that was missing. Finally, I met to suhas2go (Suhas), other of the GSoC guys working on Pitivi like me. It was really awesome to meet him :) That day I also found Julita, who put strong effort to spread the word of GNOME in Peru. She introduced me to Rafał Lużyński who is from Krakow. It was a great coincidence because I was going to visit visit Warsaw and Krakow after GUADEC.

Davind King explaining me about GTK dialogs

David King explaining me about GTK dialogs

The second day of talks I woke up very early, about 4:00 a.m. and it was so the rest of the days. I took this as an advantage to continue with my GSoC project. One of the conferences I was most interested in was the one about Wayland, which is a project I have some interest on getting involved because it seems challenging to me and because of the EVoC. Other talk I found pretty interesting and I think I will investigate more about it was the one titled "Fantastic Layouts And Where To Find Them". I promise to post about it as soon I try it, because you can create different layouts in GTK+ with a very simple syntax, that seems really easy to remember and understand. That day we had the party for the 20th anniversary of GNOME. I met there Federico Mena, who was telling me about how they started GNOME. It was awesome to listen to him, it was like traveling to the past and I am very grateful with the work of this man. After the event finish, I met Christian Hergert in person. I was talking with him about libpeas and GSettings. After talking with him I was convinced that Pitivi should use GSettings instead of ConfigParser.

With Federico Mena on the screens

Selfie with Federico Mena on the 20th anniversary party

The last day of the conference three Pitivi contributors (Jeff, Suhas and I) were together. I showed to Jeff a project I was working on during my vacations of the university that was a plugin for GStreamer I called gstblendersrc which I hope to continue and finish after GSoC finishes. During talks there was an open talk I was interested in that was Microsoft loves Linux. I has never supported Microsoft, but they are good in business, my interest was basically because before arriving to Manchester, during my flight from Madrid to Manchester I was reading a book named Conscious Capitalism by John Mackey (CEO of Whole Foods Market) who states that capitalism gives efficiency to non-profit organizations. By the way, I recommend the GNOME Board to read this book. Then the groupal photo took place and the lighning talks. I hope to be in Almeria the next year that is where next GUADEC will take place. Then it was the city tour, but, unfortunately, I lost the group. Anyway, it was a great opportunity to hack on Pitivi.

GUADEC 2017 t-shirt

GUADEC 2017 t-shirt

The next days were the workshops. Suhas and I were working on Pitivi. Suhas learns very fast, I think. When I did my first GSoC I had some problems. We sometimes helped between us, but that was not happening with frequency. The last day of the workshops I was looking for David King because I was thinking about working in Cheese for my thesis, but I couldn't find him. I was with Julita and Felipe Borges. I told him about a project I have on my mind to implement in Cheese that is adding stickers over the detected faces on people. He started to give me more ideas, like to have a library of stickers fed by the community. Also he told me that it could be possible to add watermarks in Cheese so in presentations events of GNOME, people can take pictures with Cheese with the stickers and the final picture would have a watermark of even the Cheese logo or the GNOME logo. Now I need to talk about it to some professors. That was my next-to-last day in Manchester. Felipe Borges showed me some pictures of the tour he had in the Manchester United Stadium. So I went there the last day of the workshops.

I took a photo to Suhas without telling him

I took a photo to Suhas without telling him anything

alt-text

16 Aug 2017 5:41pm GMT

15 Aug 2017

feedPlanet GNOME

Jim Hall: Happy birthday, GNOME!

The GNOME desktop turns 20 today, and I'm so excited! Twenty years is a major milestone for any open source software project, especially a graphical desktop environment like GNOME that has to appeal to many different users. The 20th anniversary is definitely something to celebrate!

I wrote an article on OpenSource.com about "GNOME at 20: Four reasons it's still my favorite GUI." I encourage you to read it!

In summary: GNOME was a big deal to me because when GNOME first appeared, we really didn't have a free software "desktop" system. The most common desktop environments at the time included FVWM, FVWM95, and their variants like WindowMaker or XFCE, but GNOME was the first complete, integrated "desktop" environment for Linux.

And over time, GNOME has evolved as technology has matured and Linux users demand more from their desktop than simply a system to manage files. GNOME 3 is modern yet familiar, striking that difficult balance between features and utility.

So, why do I still enjoy GNOME today?

  1. It's easy to get to work
  2. Open windows are easy to find
  3. No wasted screen space
  4. The desktop of the future


The article expands on these ideas, and provides a brief history of GNOME throughout the major milestones of GNOME 1, 2, and 3.

15 Aug 2017 4:44pm GMT

Julita Inca: My first Keynote at CONECIT 2017 in Tingo Maria

Yesterday, I open the KeyNote session at CONEISC 2017 with a talk in an hour and a half. I have presented some experiences I had with HPC (High Performance Computing) in universities and during the ISC 2016 to show what is going on in the world regarding HPC, not only in architecture matters, also in programming. Comming soon the video 🙂It was a large audicence that congregates more than 1000 students and professionals in Computer Science and all the Engineering School in Peru.People participated with question I did and also it seems that are so interested in the topic.I want to thanks all people who helped me backstage, this is not only my effort, this is a community effort! Thanks Leyla Marcelo and Toto Cabezas, part of the GNOME Lima! ❤Thanks so much CONECIT 2017 - Tingo Maria 😀


Filed under: FEDORA, GNOME, τεχνολογια :: Technology Tagged: CONECIT, CONECIT 2017, CONECIT TGI, CONECIT Tingo Maria, fedora, GNO, GNOME, High Performance, HPC, HPC in the jungle, Julita Inca, Julita Inca Chiroque, KeyNote, Selva Peru

15 Aug 2017 4:38pm GMT