fidzu : ubuntu

Getting started - clang-tidy AST Matchers

Over the last few weeks I published some blogs on the Visual C++ blog about Clang AST Matchers. The series can be found here:

• Extending clang-tidy
• Examining the clang AST with clang-query
• Rewriting code with clang-tidy

I am not aware of any similar series existing which covers creation of clang-tidy checks, and use of clang-query to inspect the Clang AST and assist in the construction of AST Matcher expressions. I hope the series is useful to anyone attempting to write clang-tidy checks. Several people have reported to me that they have previously tried and failed to create clang-tidy extensions, due to various issues, including lack of information tying it all together.

Other issues with clang-tidy include the fact that it relies on the "mental model" a compiler has of C++ source code, which might differ from the "mental model" of regular C++ developers. The compiler needs to have a very exact representation of the code, and needs to have a consistent design for the class hierarchy representing each standard-required feature. This leads to many classes and class hierarchies, and a difficulty in discovering what is relevant to a particular problem to be solved.

I noted several problems in those blog posts, namely:

• clang-query does not show AST dumps and diagnostics at the same time<
• Code completion does not work with clang-query on Windows
• AST Matchers which are appropriate to use in contexts are difficult to discover
• There is no tooling available to assist in discovery of source locations of AST nodes

Last week at code::dive in Wroclaw, I demonstrated tooling solutions to all of these problems. I look forward to video of that talk (and videos from the rest of the conference!) becoming available.

Meanwhile, I'll publish some blog posts here showing the same new features in clang-query and clang-tidy.

clang-query in Compiler Explorer

Recent work by the Compiler Explorer maintainers adds the possibility to use source code tooling with the website. The compiler explorer contains new entries in a menu to enable a clang-tidy pane.

I demonstrated use of compiler explorer to use the clang-query tool at the code::dive conference, building upon the recent work by the compiler explorer developers. This feature will get upstream in time, but can be used with my own AWS instance for now. This is suitable for exploration of the effect that changing source code has on match results, and orthogonally, the effect that changing the AST Matcher has on the match results. It is also accessible via cqe.steveire.com.

It is important to remember that Compiler Explorer is running clang-query in script mode, so it can process multiple let and match calls for example. The new command set print-matcher true helps distinguish the output from the matcher which causes the output. The help command is also available with listing of the new features.

The issue of clang-query not printing both diagnostic information and AST information at the same time means that users of the tool need to alternate between writing

set output diag

and

set output dump

to access the different content. Recently, I committed a change to make it possible to enable both output and diag output from clang-query at the same time. New commands follow the same structure as the set output command:

enable output dump
disable output dump

The set output <feature> command remains as an "exclusive" setting to enable only one output feature and disable all others.

Dumping possible AST Matchers

This command design also enables the possibility of extending the features which clang-query can output. Up to now, developers of clang-tidy extensions had to inspect the AST corresponding to their source code using clang-query and then use that understanding of the AST to create an AST Matcher expression.

That mapping to and from the AST "mental model" is not necessary. New features I am in the process of upstreaming to clang-query enable the output of AST Matchers which may be used with existing bound AST nodes. The command

enable output matcher

causes clang-query to print out all matcher expressions which can be combined with the bound node. This cuts out the requirement to dump the AST in such cases.

Inspecting the AST is still useful as a technique to discover possible AST Matchers and how they correspond to source code. For example if the functionDecl() matcher is already known and understood, it can be dumped to see that function calls are represented by the CallExpr in the Clang AST. Using the callExpr() AST Matcher and dumping possible matchers to use with it leads to the discovery that callee(functionDecl()) can be used to determine particulars of the function being called. Such discoveries are not possible by only reading AST output of clang-query.

Dumping possible Source Locations

The other important discovery space in creation of clang-tidy extensions is that of Source Locations and Source Ranges. Developers creating extensions must currently rely on the documentation of the Clang AST to discover available source locations which might be relevant. Usually though, developers have the opposite problem. They have source code, and they want to know how to access a source location from the AST node which corresponds semantically to that line and column in the source.

It is important to make use a semantically relevant source location in order to make reliable tools which refactor at scale and without human intervention. For example, a cursory inspection of the locations available from a FunctionDecl AST node might lead to the belief that the return type is available at the getBeginLoc() of the node.

However, this is immediately challenged by the C++11 trailing return type feature, where the actual return type is located at the end. For a semanticallly correct location, you must currently use

getTypeSourceInfo()->getTypeLoc().getAs().getReturnLoc().getBeginLoc()

It should be possible to use getReturnTypeSourceRange(), but a bug in clang prevents that as it does not appreciate the trailing return types feature.

Once again, my new output feature of clang-query presents a solution to this discovery problem. The command

enable output srcloc

causes clang-query to output the source locations by accessor and caret corresponding to the source code for each of the bound nodes. By inspecting that output, developers of clang-tidy extensions can discover the correct expression (usually via the clang::TypeLoc heirarchy) corresponding to the source code location they are interested in refactoring.

Next Steps

I have made many more modifications to clang-query which I am in the process of upstreaming. My Compiler explorer instance is listed as the 'clang-query-future' tool, while the clang-query-trunk tool runs the current trunk version of clang-query. Both can be enabled for side-by-side comparison of the future clang-query with the exising one.

11 Nov 2018 10:46pm GMT

A conversa andou pelo assunto do momento: o chapéu azul resultante da aquisição da Red Hat pela IBM, (os novos) equipamentos comercializados com GNU/Linux incluindo os Thelios da System 76 e o smartphone e tablet da Pine64, novidades do UBports, os eventos que vão acontecer muito em breve, com especial destaque para o Secure Open Source day do qual somos Media Partners e vamos tentar fazer a melhor cobertura possível para todos os que não puderem estar presentes entre outras coisas!

Já sabes: Ouve, subscreve e partilha!

• https://enventelibre.org/t-shirts/138-t-shirt-ubuntu-cosmic-cuttlefish.html
• https://shop.canonical.com/
• https://www.olimex.com/
• https://www.redhat.com/en/blog/red-hat-ibm-creating-leading-hybrid-cloud-provider
• https://www.zdnet.com/article/linus-torvalds-is-back-in-charge-of-linux/
• https://itsfoss.com/pinebook-kde-smartphone/
• https://www.youtube.com/watch?v=N_l3erCuZb4
• https://certification.ubuntu.com/desktop/
• https://slimbook.es/
• https://system76.com/
• https://puri.sm/products/librem-5/
• https://www.pine64.org/
• https://www.youtube.com/watch?v=N_l3erCuZb4
• https://thel.io/
• https://snapcraft.io/store
• https://flathub.org/
• http://loco.ubuntu.com/events/ubuntu-pt/3806-encontro-ubuntu-pt-sintra/
• https://secosday.eu/
• https://secosday.eu/speaker/diogo-constantino
• https://secosday.eu/sessions
• https://ubuntu-paris.org/

Patrocínios

Este episódio foi produzido e editado por Alexandre Carrapiço (Thunderclaws Studios - captação, produção, edição, mistura e masterização de som) contacto: thunderclawstudiosPT-arroba-gmail.com.

Atribuição e licenças

A imagem de capa: q.phia em Visualhunt e está licenciada como CC BY.

A música do genérico é: "Won't see it comin' (Feat Aequality & N'sorte d'autruche)", por Alpha Hydrae e está licenciada nos termos da CC0 1.0 Universal License.

Este episódio está licenciado nos termos da licença: Attribution-NonCommercial-NoDerivatives 4.0 International (CC BY-NC-ND 4.0), cujo texto integral pode ser lido aqui. Estamos abertos a licenciar para permitir outros tipos de utilização, contactem-nos para validação e autorização.

09 Nov 2018 12:49am GMT

This week we've been using windows Subsystem for Linux and playing with a ThinkPad P1. IBM buys RedHat, System76 announces their Thelio desktop computers, SSD encryption is busted, Fedora turns 15, IRC turns 30 and we round up the community news.

It's Season 11 Episode 35 of the Ubuntu Podcast! Alan Pope, Mark Johnson and Martin Wimpress are connected and speaking to your brain.

In this week's show:

• We discuss what we've been up to recently:
  • Mark has been testing Windows Subsystem for Linux.
  • Martin been playing with a Thinkpad P1.
• We discuss the news:
  • IBM Buys Red Hat
  • System76 announce Thelio desktop computers with open-source designs
  • SSD Encryption busted
  • Fedora turns 15
  • Internet Relay Chat turns 30
• We discuss the community news:
  • Disco Dingo
  • Xubuntu Development Update November 2018
  • Mir News: 2nd of November 2018
  • Congratulations to Erich Eickmeyer, from Ubuntu Studio, on becoming an Ubuntu Member.
• Image credit: NordWood Themes

That's all for this week! You can listen to the Ubuntu Podcast back catalogue on YouTube. If there's a topic you'd like us to discuss, or you have any feedback on previous shows, please send your comments and suggestions to show@ubuntupodcast.org or Tweet us or Comment on our Facebook page or comment on our Google+ page or comment on our sub-Reddit.

• Join us in the Ubuntu Podcast Telegram group.

08 Nov 2018 3:00pm GMT

I know I'm very late, but I really wanted to talk about this year Google Summer of Code.

For the third year in a row, I was a mentor. And this year I have a huge deception to share. I'm really sad. This week, I'v received the GSoC 2011 t-shirt. They sent me the wrong size. XXXL. I can use it as a sleeping bag with my girlfriend. I'm really disappointed.

Hopefully, GSoC is not only about receiving a t-shirt. It is also about mentoring a student.

Nearly two years ago, I started working on a complete refactoring of GTG. The code was a mess, with a lot of duplicate everywhere, with two bugs appearing while you were trying to solve one, etc.

I abstracted the structure we were using in several places and started to write a library to handle those "Acyclical Directed Graphs". As usual, it appeared that development was taking longer than expected. Weeks turned into months. Then, when it started to look good, I discovered that I forgot one critical point: thread-awareness. I felt hopeless.

Because I didn't had the motivation to do that heavy work, I proposed it as a Summer of Code project to a very motivated student: Izidor Matušov^[1].

Words doesn't do any justice to the excellent work that Izidor did this summer. He's simply awesome. Some students are goods because they have previous experience. Izidor kicks asses. He learns so quickly, he's so assertive. The work was even harder than what we anticipated. But he managed to achieve everything, including feeding me with cookies at the Desktop Summit, where we met and had an awesome hacking week.

As much as I'm deceipted about the t-shirt, I'm delighted about the work achieved this summer. Izidor now knows GTG nearly as much as I do. He's taking initiatives, like organizing an online GTG hackfest on November 26th^[2] and he's a bug-answering machine.

Dear Google, GNOME foundation and Lanedo^[3], I would like to thank you. Thanks to your support:

1. I received a worthless piece of clothes that travelled half of the world in order to clean my cat's dirtiness.
2. GTG 0.2.9 should be released before the end of the year
3. GTG gained a new co-maintainer
4. I gained a new friend. And it probably worth everything else.

Congratulations, co-maintainer Izidor. And welcome to the community!

06 Nov 2011 6:23pm GMT

Part 1 : Introduction
Part 2 : Home is Desktop
Part 3 : There's no tray icon in GNOME !
Part 4 : Kill The Double Click

When I started installing the best desktop possible for Marie and Jean, we were still in the GNOME 2.X era. GNOME 3 solved my previous concerns. No in the way I envisioned it, but solved them anyway. No more desktop icons, no more tray icons.

But now that I'm introducing Marie and Jean to GNOME 3, I still have some concerns. And one of that main concern is the infamous double-click!

Do you remember? Jean is a very brilliant mind, even though he never used a computer during his whole life. As a reasoning scientist, he was trying to find the logic behind my teaching.

During one of our first lesson, "Using the mouse", the conversation went like this:
- How do I know if I have to click or double click?
- Well, you double-click on icon and simple click on links and buttons.
- How do I know what is a button or an icon?
- …

Since that time, I've tried many times to find a logic behind single or double clicking. There is not. You have to learn it by experience. And it is totally, utterly pointless.

I also realized that a single click was something really hard for Jean. Achieving to click on a given point without moving the mouse is really hard for older people. Then, ask them to click twice, with a completely arbitrary speed, without moving the mouse, not to quickly, not to slowly. Impossible.

Marie, on her side, was double-clicking everywhere. And, surprisingly, it works most of the time.

So, why do we have double-click in some places? Because we want to be able to select an item without "activating" it. How often does it happen? Never for Jean. Very rarely for Marie.

To summarize, we are making the most frequent action very hard to nearly impossible in order to allow a very rare action?

I tried to disable completely the double-clicking in Nautilus.

Do you know what?

It works. Even for me. I had chronic pain in my hand and disabling double-click was a relief. I explained to Marie to never double-click anymore. She's still double-clicking from time to time but everything works even better than before. Jean was eventually able to launch a file from within Nautilus.

Selection of one or multiple file with single mouse click

What about selection of files? I explained to Marie to draw a square with the mouse. And, yes, she found that absolutely intuitive. The only drawback I found so far was the inconsistency with lists, where double-clicking is still required. Marie called me one day because she tried to play a specific song in Rhythmbox. It wasn't working. I realize that she had to double-click on the song. "But you told me to never double click anymore!". Sorry Marie.

I'm myself incredibly frustrated by any system that requires double-click. Why do we still have double-click by default in GNOME3?


Part 1 : Introduction
Part 2 : Home is Desktop
Part 3 : There's no tray icon in GNOME !
Part 4 : Kill The Double Click


Picture by Dave Dugdale

05 Nov 2011 12:17pm GMT

Si vous deviez me décrire en deux mots, nul doute que fourbe et profiteur vous viendraient spontanément à la bouche. Paresseux, parasite et inutile suivraient de près. Et j'en suis fier. J'en ai même fait mon mode de vie.

Ma technique est simple mais éprouvée. Je croise un inconnu dans la rue à l'air affable. Tenez, prenez ce jeune homme à l'allure dynamique. Il s'appelle Jean, c'est ma prochaine victime. Il ne se doute encore de rien mais j'irai dormir dans le lit de sa femme tout en vidant son frigo.

Au premier abord, je fais le numéro du sympa-sociable, les circonstances m'ont conduit dans la rue, où j'ère sans but précis, mais je ne me plains pas, je ne quémande rien, au contraire, je refuse tout geste de pitié trop ostentatoire. J'ai ma fierté.

Lorsque Jean se propose de m'emmener manger à la maison, juste pour la soirée, je fais d'abord mine de ne pas être intéressé. Mais mes yeux acquiescent et Jean, en rigolant, insiste, me forçant presqu'à le suivre. Inutile de vous dire que c'est ce que j'attendais mais la victime doit croire qu'elle a l'initiative, c'est primordial.

Martine, la femme de Jean, n'est que moyennement contente de cet imprévu. Qu'à cela ne tienne, je fais mon charmeur, je séduis tout en ayant l'air de ne pas vouloir déranger. Je fais également un peu le pitre pour la dérider.

Et ça marche. Avant la fin de la soirée, elle discutera avec moi plus qu'avec Jean lui-même, ce dernier étant parfaitement inconscient du destin de proie que je lui réserve. De manière indirecte, je fais comprendre que je n'ai nul part où aller. Jean et Martine n'ont pas le cœur de me renvoyer seul dans le froid de la nuit. Ils se proposent donc de m'héberger, juste pour une nuit. Tandis que je m'installe confortablement sur le sofa, j'entends Martine descendre l'escalier. Elle est en déshabillé, prête à aller au lit.

- « Bonne nuit ! » me lance-t-elle avec un sourire innocent avant de remonter dare-dare dans sa chambre.

Je ricane. Je n'ai même pas eu besoin de répondre. Une seule soirée me suffit. Homme ou femme, nul ne me résiste. Je suis comme ça moi.

Bien entendu, le « seulement pour une nuit » se prolongera. Je commencerai doucement à faire comprendre mes goûts précis, envoyant Jean au supermarché afin de m'acheter ce que je souhaite. Lorsqu'elle rentre du travail, Martine a à peine un regard pour Jean. Elle se rue à l'intérieur pour voir comment je vais. Pendant ce temps-là, je me prélasse sur le canapé, je me balade un peu. Avec mon air faussement négligent, j'ai pris soin de casser quelques bibelots auxquels ils tenaient beaucoup, par pure cruauté.

Lorsque Jean partit quelques jours dans sa famille à l'étranger, je n'hésitai pas: je me glissai une nuit dans le lit de Martine, sans même lui demander, sans même m'annoncer. Elle prit un air faussement surpris mais je sais qu'elle n'attendait que cela. Elles sont toutes les mêmes. Jean nous a surpris en rentrant plus tôt. Cela ne lui a pas plu. Il m'a dit qu'il m'avait sorti de la rue, qu'il n'acceptait pas cela.

Par méchanceté, j'ai répondu en déféquant sur la moquette du salon. Il a pu tout nettoyer. Il n'était vraiment pas content mais Martine a fini par le convaincre de me garder et d'exercer le moindre de mes désirs.

Il faut dire qu'ils sont vraiment bien mes deux esclaves. Je dors dans leur lit, ils me nourrissent, nettoient sans que je n'aie besoin de faire attention à rien. Quoi que je fasse, ils me regardent avec un air attendri et me trouvent adorable. Même au milieu de la nuit, il suffit que je me mette à miauler pour qu'ils s'enquièrent immédiatement de mes besoins.

Des esclaves aussi dociles, c'est rare. Je vais les garder encore quelques temps.

28 Oct 2011 4:59pm GMT