25 Apr 2017

feedPlanet KDE

Cutelyst 1.6.0 released, to infinity and beyond!

Once 1.5.0 was release I thought the next release would be a small one, it started with a bunch of bug fixes, Simon Wilper made a contribution to Utils::Sql, basically when things get out to production you find bugs, so there were tons of fixes to WSGI module.

Then TechEmpower benchmarks first preview for round 14 came out, Cutelyst performance was great, so I was planning to release 1.6.0 as it was but second preview fixed a bug that Cutelyst results were scaled up, so our performance was worse than on round 13, and that didn't make sense since now it had jemalloc and a few other improvements.

Actually the results on the 40+HT core server were close to the one I did locally with a single thread.

Looking at the machine state it was clear that only a few (9) workers were running at the same time, I then decided to create an experimental connection balancer for threads. Basically the main thread accepts incoming connections and evenly pass them to each thread, this of course puts a new bottleneck on the main thread. Once the code was ready which end up improving other parts of WSGI I became aware of SO_REUSEPORT.

The socket option reuse port is available on Linux >3.9, and different from BSD it implements a simple load balancer. This obsoleted my thread balancer but it still useful on !Linux. This option is also nicer since it works for process as well.

With 80 cores there's still the chance that the OS scheduler put most of your threads on the same cores, and maybe even move them when under load. So an option for setting a CPU affinity was also added, this allows for each work be pinned to one or more cores evenly. It uses the same logic as uwsgi.

Now that WSGI module supported all these features preview 3 of benchmarks came out and the results where still terrible… further investigation revealed that a variable supposed to be set with CPU core count was set to 8 instead of 80. I'm sure all this work did improve performance for servers with a lots of cores so in the end the wrong interpretation was good after all

Preview 4 came out and we are back to the top, I'll do another post once it's final.

Code name "to infinity and beyond" came to head due scalability options it got

Last but not least I did my best to get rid of doxygen missing documentation warnings.

Have fun https://github.com/cutelyst/cutelyst/archive/v1.6.0.tar.gz


25 Apr 2017 8:09pm GMT

24 Apr 2017

feedPlanet KDE

ReleaseMe CI States, Help, Git

Since we recently wired up the KDE release tool releaseme with KDE neon's git-build technology I had the opportunity to spend some quality time on the code and managed to sneak in a load of both quality and speed improvements as well as some new features.

CI States

For quite a while I've been considering to pull CI data into releaseme. It makes all sorts of sense. You want to release high-quality software, so whenever possible you'd want to make sure CI is green. But to do that you'd have to open a browser, navigate to build.kde.org, and find the right jobs. Yet, there is absolutely no reason why a human would have to do that. So now releaseme does it for you before getting started on assembling the sources.

Help

Unfortunately, the release process itself is often a bit convoluted and it is not clear what to do once you have a tarball. To help people with publishing their new tarball releaseme will now print a handy help suggesting common steps to take once you have a tarball. It also points to the relevant location for publishing tarballs on KDE infrastructure and even gives you a handy link with a sysadmin ticket template to quickly file a publishing request.

Git Builds

Aleix Pol worked on adding a cool new feature to the extra-cmake-modules frameworks, enabling everyone who builds from git to fetch translations as well. This is currently slated to release with KDE Frameworks 5.34. Once landed you'll be able to run make fetch-translations with just about any git clone of KDE software and grab the latest translations into your build directory.

24 Apr 2017 10:44am GMT

Efficient barcode scanning with QZXing

QZXing is a very useful library: It provides an easy to use Qt integration API around the barcode scanning library ZXing (zebras crossing).

Because it is so easy to setup QZXing in a Qt application, we and most of our customers end up using it when they need to scan images for barcodes. There is, or rather was, a big problem though: When you want to analyze a live camera video stream, e.g. from a phone camera, the latency can …

The post Efficient barcode scanning with QZXing appeared first on KDAB.

24 Apr 2017 9:00am GMT

23 Apr 2017

feedPlanet KDE

KDE neon Translations

One of the best things about making software collaboratively is the translations. Sure I could make a UML diagramming tool or whatever all by my own but it's better if I let lots of other people help out and one of the best crowd-sourcing features of open community development is you get translated into many popular and obscure languages which it would cost a fortune to pay some company to do.

When KDE was monolithic is shipping translation files in separate kde-l10n tars so users would only have to install the tar for their languages and not waste disk space on all the other languages. This didn't work great because it's faffy for people to work out they need to install it and it doesn't help with all the other software on their system. In Ubuntu we did something similar where we extracted all the translations and put them into translation packages, doing it at the distro level makes more sense than at the collection-of-things-that-KDE-ships level but still has problems when you install updated software. So KDE has been moving to just shipping the translations along with the individual application or library which makes sense and it's not like the disk space from the unused languages is excessive.

So when KDE neon came along we had translations for KDE frameworks and KDE Plasma straight away because those are included in the tars. But KDE Applications still made kde-l10n tars which are separate and we quietly ignored them in the hope something better would come along, which pleasingly it now has. KDE Applications 17.04 now ships translations in the tars for stuff which uses Frameworks 5 (i.e. the stuff we care about in neon). So KDE neon User Editions now include translations for KDE Applications too. Not only that but Harald has done his genius and turned the releaseme tool into a library so KDE neon's builder can use it to extract the same translation files into the developer edition packages so translators can easily try out the Git master versions of apps to see what translations look missing or broken. There's even an x-test language which makes xxTextxx strings so app developers can use it to check if any strings are untranslated in their applications.

The old kde-l10n packages in the Ubuntu archive would have some file clashes with the in-tar translations which would often break installs in non-English languages (I got complaints about this but not too many which makes me wonder if KDE neon attracts the sort of person who just uses their computer in English). So I've built dummy empty kde-l10n packages so you can now install these without clashing files.

Still plenty to do. docs aren't in the Developer Edition builds. And System Settings needs some code to make a UI for installing locales and languages of the base system, currently that needs done by hand if it's not done at install time (apt install language-pack-es). But at last another important part of KDE's software is now handled directly by KDE rather than hoping a third party will do the right thing and trying them out is pleasingly trivial.

Facebooktwittergoogle_pluslinkedinby feather

23 Apr 2017 1:00pm GMT

SUSE Hack Week 15

Back in February the fifteenth SUSE Hack Week took place. As always this was a week of free hacking, to learn, to innovate, to collaborate, and to have a lot of fun. I didn't have the full time, so I worked on a couple of small things and a few projects I maintain. I did want to summarize that, so here you go.

The first project remained unfinished. I wanted to fill out Tim Urban's Life Calendar (you might have seen that on his excellent blog "Wait But Why"), but realized that it's not trivial to map dates to weeks in your life. So I wrote a small tool to calculate that, stopped after I had a failing test and had a rough feeling for how to put the dots on the life calendar.

The second project was something I always wanted to do, implement Conway's Game of Life. I had once started an implementation in 68000 assembler on the Amiga many years ago, but never finished it. Technology has advanced, so I decided to do at as ASCII pixel art. Who needs high resolution? The result turned out to be a bit more generic, as a command line tool to manipulate pixel matrices stored in text files, the Pixelist. While I was at it, I also implemented Langton's Ant and a simulation of diffusion limited aggregation.

GivenFilesystem is a Ruby gem I maintain for writing unit tests dealing with files on disk. It's quite convenient, if you test code, which writes files on disk and you want to have a well-defined environment without side effects for testing this code. There were some open pull requests. I reviewed and merged them and released given_filesystem 0.2.0.

I already wrote about Inqlude, where I used Hack Week to finally publish the new Inqlude web site, which is based on the work Nanduni did during last year's Google Summer of Code. It's a great improvement. I also did some cleanup work, including reviewing the open issues. So we have a nice roadmap now. There is some interesting work to do. People who want to help with that are always welcome.

Trollolo is another side project I work on from time to time. We use it for supporting our Scrum work flows at SUSE in Trello, such as generating burndown charts. I didn't get around to write code, but I consolidated some of the ideas floating around and put them into issues. This also is a nice opportunity, if you want to have some fun with writing a bit of Ruby code for a small project. Issues are up for takers.

Finally I worked a bit on the next iteration of my attempt to make git support multiple authors. This would make life with git in a pair programming situation much nicer. Based on the feedback I got on my first iteration and at the Git Merge conference, I started to work on a version which puts the data into the trailers of the commit messages. This is less intrusive and with a bit of tooling it achieves similar results as the version which worked directly on the commit headers. I have something working, but it needs a rework of the trailer handling code. I'll continue to work on that when I find some more time to do that.

23 Apr 2017 12:52pm GMT

21 Apr 2017

feedPlanet KDE

One week to LaKademy 2017 \o/

We were just a bunch of nice-looking guys and girls :) hanging wifi routers over the windows and trying not being intimidated by an unceasing rain when we had the Brazilian KDE Summit (Akademy-BR) in Praia do Forte (BA) back in 2010. This was somehow the birth of LaKademy (Latin-American KDE Summit), started in 2012 and now having its fifth edition taking place in Belo Horizonte (Minas Gerais) from 29th April to 1st May.

LaKademy is an effort to promote the culture of FLOSS, hacking sessions, and collaborative work inside the Latin-American KDE community. It is also a way to let people know about the Brazilian/Latin-American KDE community, the sort of work we usually do, and how newcomers can start their journeys into the free software world. And, of course, is the place to strengthen relationships, make new friends, and let people fall in love about such an amazing FLOSS community <3.

Sixteen contributors will attend LaKademy 2017, interested in contributions to translation, promo, artwork, Atelier, Umbrello, KDE-Edu, Dolphin, KDE Games, KDevelop, and Kdenlive. The traditional LaKademy promo meeting will happen on 30th morning, where we make a retrospect of 2016 actions and start thinking about what we want to do for 2017/2018.

See you in Belo Horizonte and stay tuned for LaKademy 2017 news and photos :)

21 Apr 2017 8:34pm GMT

Qt Creator 4.2.2 released

butterfly

The spring has arrived, so we took our brooms and swept the bugs from under the carpets and out of the door.

We meticulously noted down all the classes, orders, families, species, and other details of what we found and removed, which you can look up in our spring cleaning log.

Get Qt Creator 4.2.2

The opensource version is available on the Qt download page, and you find commercially licensed packages on the Qt Account Portal. Qt Creator 4.2.2 is also available through an update in the online installer. Please post issues in our bug tracker. You can also find us on IRC on #qt-creator on chat.freenode.net, and on the Qt Creator mailing list.

The post Qt Creator 4.2.2 released appeared first on Qt Blog.

21 Apr 2017 10:57am GMT

Five days and counting

It is five days left until foss-north 2017, so it is high time to get your ticket! Please notice that tickets can be bought all the way until the night of the 25th (Tuesday), but catering is only included is you get your ticket on the 24th (Monday), so help a poor organizer and get your tickets as soon as possible!

And just to reiterate what's on the menu. This is a full day conference with two tracks and many interesting speakers from many projects, including Debian, Wikimedia, KDE, systemd, PulseAudio, Nextcloud, PostgreSQL, OpenRISC, flatpak, AsteroidOS and more.

Last year, over 30% of the tickets was sold in the last 48h. This year, we cannot provide catering for the ones buying tickets the last 24h, so hopefully the peak will come earlier. For now, we have 93 persons on the visiting list, but there are chairs for 100 more!

21 Apr 2017 10:12am GMT

Kdenlive 17.04 released

We release 17.04 with a redesigned profile selection dialog to make it easier to set screen size, framerate, and other parameters of your film. Now you can also play your video directly from the notification when rendering is finished. Some crashes that happened when moving clips around on the timeline have been corrected, and the DVD Wizard has been improved.

Please note that while this major release may seem to have few features development is at full throttle in the refactoring branch. You can monitor the progress here.

21 Apr 2017 3:31am GMT

KDE PIM update for Zesty available for testers

Since we missed by a whisker getting updated PIM (kontact, kmail, akregator, kgpg etc..) into Zesty for release day, and we believe it is important that our users have access to this significant update, packages are now available for testers in the Kubuntu backports landing ppa.

While we believe these packages should be relatively issue-free, please bear in mind that they have not been tested as comprehensively as those in the main ubuntu archive.

Testers should be prepared to troubleshoot and hopefully report issues that may occur. Please provide feedback on our mailing list [1], IRC [2], or optionally via social media.

After a period of testing and verification, we hope to move this update to the main backports ppa.

You should have some command line knowledge before testing.
Reading about how to use ppa purge is also advisable.

How to test KDE PIM 16.12.3 for Zesty:

Testing packages are currently in the Kubuntu Backports Landing PPA.

sudo add-apt-repository ppa:kubuntu-ppa/backports-landing
sudo apt-get update
sudo apt-get dist-upgrade

1. Kubuntu-devel mailing list: https://lists.ubuntu.com/mailman/listinfo/kubuntu-devel
2. Kubuntu IRC channels: #kubuntu & #kubuntu-devel on irc.freenode.net

21 Apr 2017 1:31am GMT

20 Apr 2017

feedPlanet KDE

Okular 1.1 released!

Today KDE Applications 17.04 was released.

It includes Okular 1.1, it contains a nice set of features:
* Add annotation resize functionality
* Add support for auto-calculation of form contents via JavaScript
* Allow to rotate the page view using two-finger pinches on a touchscreen
* Change pages in presentation mode by swiping on touch screen
* Added support for Links that change the Optional Content visibility status
* Allow to disable automatic search while typing
* Allow to create bookmarks from the Table Of Contents

This release was brought to you by Albert Astals Cid, Oliver Sander, Luigi Toscano, Martin T. H. Sandsmark, Tobias Deiminger, Antonio Rojas, Burkhard Lück, Christoph Feck, Elvis Angelaccio, Gilbert Assaf, Heiko Becker, Hrvoje Senjan, Marco Scarpetta, Miklós Máté, Pino Toscano, Yuri Chornoivan.

20 Apr 2017 9:08pm GMT

18 Apr 2017

feedPlanet KDE

Updates on the KActionRunner thinkering.

First I would like to answer a few questions that I had on my latest entry:

1 - KActionRunner is *not* a unified-menu, and it does not tries to be. It shouldn't be used to display the menu of the application in a unified way.

2 - Somebody told me that this looked like the Eclipse "Quick Access" feature, and that's exactly what this is.

Changes in the code currently on my machine, *not merged yet*:

Obligatory Screenshoot running on KDevelop:

I certainly need to iron out a few things before trying to merge this, so I'm not doing a review again this soon (my older review was shut down pretty fast

I'm using it on all my KDE applications locally and this is starting to behave the way I want.

18 Apr 2017 10:51am GMT

Multithreaded Programming with Future & Promise

This is a guest post by 2016 Qt Champion Ben Lau.

Ben has a long history with Qt, and many interesting projects on GitHub.

Here's an idea from him on making multithreading simpler in some cases.

The Basics

Multithreading programming may not be difficult at the first glance. You have to pay attention to your shared data to avoid race conditions/deadlocks. So you learn mutex and semaphore and do it carefully. The result works perfectly on your machine.

But one day, your program hangs. You spend an hour to trace out the problem and find out the order of code execution is not same as your expectation. So you add a few mode condition checking and fix the problem.

After a few week of development, the program is getting more complicated. And it begins to crash randomly. This time even after a day you still can't figure out what is wrong and admit that it is totally out of control.

Does that sound like a familiar story? It is not rare to find complaints about random crashes/hangs due to misuse of a thread. Is it really difficult to write multithreaded programs?

The answer is yes and no. It depends on your software requirement and architecture.

In this article, it is going to introduce a lock-free multithreaded programming method by using QtConcurrent and AsyncFuture. These make multithreaded programming easier.

Let's take an example. The code below shows an asynchronous ImageReader class. The readImageWorker function will be executed on another thread that won't block the UI. QFuture represents the result of computation and reports the status change.

class ImageReader : public QObject {
public:
    QFuture<QImage> read(const QString& fileName);
};

QFuture<QImage> ImageReader::read(const QString &fileName)
{
    auto readImageWorker = [](const QString &fileName) {
        QImage image;
        image.load(fileName);
        return image;
    };
    return QtConcurrent::run(readImageWorker, fileName);
}

Example of use

ImageReader reader;

QFuture future = reader.read(INPUT);

QFutureWatcher *watcher = new QFutureWatcher();

connect(watcher, &QFutureWatcher::finished,
 [=]() {
 setImage(future.result());
 });

watcher->setFuture(future);

Multithreaded programming with QtConcurrent is pretty easy. It just takes an input, then produce an output later. QtConcurrent handles all of the low-level threading primitives.

But it is limited to the condition that the concurrent function does not access shared data with other threads. If that happens, it may still need a lock in order to maintain a critical session. That will fall back to the old traditional way.

Make it support Image caching

The above example is quite an ideal case. And of course, a real world problem is usually not that simple. Let's change the requirement - Make it support image caching.

QFuture ImageReader::read(const QString &fileName)
 {
 auto readImageWorker = [](const QString &fileName) {
 QImage image;
 image.load(fileName);
 return image;
 };

QFuture future = QtConcurrent::run(readImageWorker, fileName);

QFutureWatcher *watcher = new QFutureWatcher(this);

auto updateCache = [=]() {
 m_cache[fileName] = future.result();
 watcher->deleteLater();
 };

connect(watcher, &QFutureWatcher::finished, updateCache);
 watcher->setFuture(future);
 return future;
 }

The class declaration:

class ImageReader : public QObject {
 public:
 bool isCached(const QString& fileName) const;
 QImage readCache(const QString& fileName) const;
 QFuture read(const QString& fileName);
 private:
 QMap<QString,QImage> m_cache;
 };
 bool ImageReader::isCached(const QString &fileName) const
 {
 return m_cache.contains(fileName);
 }

QImage ImageReader::readCache(const QString &fileName) const
 {
 return m_cache[fileName];
 }

Before getting an image, you have to query is the cache available:

if (reader.isCached(INPUT)) {
 setImage(reader.readCache(INPUT));
 return;
 }
 QFuture future = reader.read(INPUT);

This solution works, but the API is not ideal. Because it would violate the "Tell, don't ask" principle. The best way is to combine readCache() and read() into a single function that always returns a QFuture object. But there is a problem, QFuture/QtConcurrent can only obtain a result from a thread. It is quite odd to start a thread but the data is already available. To get rid of this problem, we need a 3rd party library.

AsyncFuture

AsyncFuture is a C++ library that could converts a signal into a QFuture type and uses it like a Promise object in Javascript. It provides a unified interface for asynchronous and concurrent tasks. The library only contains a single header file, so that it is very easy to bundle in your source tree. Or you may install it by qpm.

Project Site:
https://github.com/benlau/asyncfuture

Let's rewrite the above function with AsyncFuture:

QFuture ImageReader::read(const QString &fileName)
 {
 if (m_cache.contains(fileName)) {
 // Cache hit. Return an already finished QFuture object with the image
 auto defer = AsyncFuture::deferred();
 defer.complete(m_cache[fileName]);
 return defer.future();
 }

if (m_futures.contains(fileName)) {
 // It is loading. Return the running QFuture
 return m_futures[fileName];
 }

auto readImageWorker = [](const QString &fileName) {
 QImage image;
 image.load(fileName);
 return image;
 };

auto updateCache = [=](QImage result) {
 m_cache[fileName] = result;
 m_futures.remove(fileName);
 return result;
 };

QFuture future = AsyncFuture::observe(QtConcurrent::run(readImageWorker, fileName)).context(this, updateCache).future();
 m_futures[fileName] = future;
 return future;
 }

This time it is almost perfect. The deferred object provides an interface to complete/cancel a QFuture manually. That could replace readCache() by returning an already finished future object.

Moreover, it has added a new feature to avoid duplicated image reading. In case you have made requests to load the same image twice before it is cached, the original design would start two threads which are totally wasting CPU power. This version solves it by keeping all the running future in a future pool and return that future for duplicated read.

Make the example more complicated

Currently the example is very simple. Let's try to make it more complicated.

Requirements:

  1. Add a readScaled(fileName, size) function that returns an image which is scaled to specific size
  2. Code reuse is a must
  3. The scaling must be done in another thread to emulate a high CPU usage function
  4. Load cached image if available
  5. But scaled image do not need to keep in cache

The most optimistic solution is to make use of the result of read() directly. That mean you have to create a thread that depends on the result of another thread. That is a bit hard to get it works with only QtConcurrent and probably it needs to use a lock. But it can be easy to be done with AsyncFuture's future chaining feature.

QFuture ImageReader::readScaled(const QString &fileName, const QSize &size)
 {
 auto scaleImageWorker = [=](QImage input) {
 return input.scaled(size);
 };
 auto callback = [=](QImage result) {
 return QtConcurrent::run(scaleImageWorker, result);
 };
 QFuture input = read(fileName);
 QFuture output = AsyncFuture::observe(input).context(this, callback).future();
 return output;
 }

First of all, it calls read() function to obtain an image from QFuture. It doesn't care about the caching mechanism as it is already handled by the read() function.

Then it creates a new future object to represent the whole work flow of the chain:

QFuture output = AsyncFuture::observe(input).context(this, callback).future();
                                                                     ^^^^^^^^^

A chain begins with a observe() function, then followed by an observer function to bind the callback to the observed future, and that will create a new future object to represent the result of the callback.

auto callback = [=](QImage result) {
 return QtConcurrent::run(scaleImageWorker, result);
 };

You may wonder if it is alright to run another worker function within the callback function. In fact, this is a feature of AsyncFuture. It provides a chainable API that works like a Promise object in JavaScript. If the callback returns a QFuture object, it will be added to the chain. Then the final output future will depend on the returned future. Therefore, the output future is in fact represents the result of read() , callback() and scaleImageWorker(). The flow could be visualised by this diagram:

sequence-diagram_promises

Diagram: The workflow of readScaled() - it shows how it uses a single QFuture to represent the result of a multiple steps task.

Conclusion

Using QtConcurrent without sharing data between threads could make multithreaed programming easier because it doesn't need to manage an access lock. But real world problems are usually more complicated. A task may not be able to complete without interaction from other threads. In this case, it may still need an access lock to protect critical session. But once you have used it, it will fall back to the old traditional way, and probably it may get the same problem mentioned at the beginning.

In this article an alternative solution is presented: Use Concurrent API together with asynchronous callback then chain them into a sequence by a promise like API. It works by breaking down a task into multiple steps. Whanever a concurrent function seeks for extra information from another thread, it should just terminate itself and pass the control back to the main thread. So that it doesn't need an access lock that may raise issues like deadlocks and race conditions.

The whole workflow could be represented by a QFuture object, a unified interface for all kind of asynchronous workflow.

However, this doesn't mean we get rid of locks completely. They are still necessary for some scenarios. So choose your solution case by case.

The post Multithreaded Programming with Future & Promise appeared first on Qt Blog.

18 Apr 2017 8:19am GMT

Uncovering 32 Qt best practices at compile time with clazy

In a previous blog post we introduced clazy, a clang plugin which makes the compiler understand Qt semantics, allowing you to get compile-time warnings about Qt best practices ranging from unneeded memory allocations to misuse of API, including fix-its for automatic refactoring.

Today we'll do a round-up and present the checks related to connect statements, Qt containers (including QString) and misc Qt facilities.

Connects

1. old-style-connect

Finds connect() statements still using the old SIGNAL()/SLOT() syntax. The Qt …

The post Uncovering 32 Qt best practices at compile time with clazy appeared first on KDAB.

18 Apr 2017 8:00am GMT

17 Apr 2017

feedPlanet KDE

Calligra 3.0.1 update released

We have released a bugfix update of Calligra. The 3.0.1 version contains the following fixes:

General

Sheets

Plan

Chart

17 Apr 2017 7:14am GMT

16 Apr 2017

feedPlanet KDE

Kubuntu 17.04 Banner

Hi to everyone!!
I'm really proud to have designed the banner for Kubuntu 17.04.
I used the beautiful Ken Vermette wallpaper as a base. I hope you like it!

rect4147a

You can find it on Kubuntu.org


16 Apr 2017 3:48pm GMT