This is good and sane advice to survive the attention economy and take care of your mental health. It's not too hard to put in place if you're not already doing it.
This piece is maybe a bit too much on the panic side… Which part of the Web are we talking about? The commercial stuff? Overall, the logic makes sense though. We see many examples of power plays in guise of "innovation" which lead to killing openly sharing (and so killing real innovation). It's urgent to fight back and ensure things stay open.
The Internet culture definitely changed at the turn of the 21st century. Before this it was a more civilized and hopeful place. I'd like to see the netizen etiquette make a strong come back.
This could be a piece full of nostalgia. There is a bit of nostalgia of course, but it's also a path to use what we got in a more valuable and humane way.
This is a good point, unlike what some claims the Internet isn't dying. The commercial land on top is thinning out and is getting filled with crap indeed. Still, what made the Internet and the Web are still here for people to use.
This part of the industry is struggling more and more (or more likely silently taking more risks to hide the struggle). It has no path to sustainability and it starts to show.
It totally makes sense. If you're a FOSS project you have to invest in getting more long term contributors, which requires mentoring. The contributions themselves are not something to maximise. I wish more communities would follow that path.
So many requests based on vanity and hype… I like the question "when you go to other websites do you use it?", we should use it more. Maybe at some point we'll realise that simplicity matters.
Good first half of the post, there's indeed more paths out of GitHub than jumping from a centralised system to another one (even though Codeberg and Forgejo are much saner from a governance standpoint). We'll see what the future brings.
Indeed, a reminder that the two concepts are not necessarily aligned. It kind of misses the point about corporate Open Source with no open contributions which can be easily captured as well. But indeed for the individual side project you might not need the whole burden of issues and contributions, you get to choose.
Podman rootless containers and the Copy Fail exploit
Tags: tech, podman, containers, security
Worthwhile exploration on the impact of CopyFail in the context of Podman. The baseline security posture seems better and you can even improve things using older techniques. Definitely worth switching.
Don't think this piece really needed to talk about AI but oh well… I guess it's the obsession of the moment. That said, it's interesting to see how far the microVM ecosystem matured so far. The pieces are falling in place and that opens the door to interesting architectures.
Straight from the uutils rewrite. This is interesting both for the class of bugs which made it (very system integration oriented, unsurprisingly) and the ones which didn't appear at all (anything to do with memory).
Collective Speed Is Not the Summation of Individual Speed
Tags: tech, team, organisation, productivity
A bit of a stretched metaphor in here, but indeed being individually faster doesn't automatically make the team faster. Sometimes quite the contrary in fact.
Hire based on the conversation about code, not the code itself
Tags: tech, hiring, interviews
Another post which shows that the right technical job interviews are the ones creating a real conversation. It's the only way to have a chance to figure out what the candidate is made of.
After you have registered, you think which interesting talks will we have on the weekend?
And you know who has something to say? *YOU*
Yes, *YOU*. I'm sure you've been working on something interesting, or have a great idea to share.
*YOU* may think that your idea is not that great or the things you work on are not interesting, but that's seldomly the case when someone explains me their "boring" thing they've been working on, i always think "Wow that's great".
Ok, so now that I've convinced you to send a talk proposal, when better than *TODAY* to send it?
Yes I know the Call for Participation is open until the 17 of May, but by sending it today you make sure you don't forget sending it later and also [more important for me] you help those of us in the Program Committee not to worry when the final date starts approaching and we don't have lots of talks yet because you all prefer sending talks on the very last minute.
Controller objects have been the main way to glue your QML UI to your application's actual implementation of the I/O and business logic. However, over the years, the way to actually expose that controller object has changed. And now, we contributed a change in QQmlEngine that allows you to change it once again, and we believe: for the better.
What are "controllers" anyway?
Conceptually, controllers are a thin glue layer between your business logic and your QML, exposing the data that the GUI needs in a format it can easily use. They are implemented as QObject-derived instances, usually with properties exposing values that may or may not be writable, as well as potentially some Q_INVOKABLE methods that can be triggered by the QML and maybe some signals.
Usually, these controllers are specific to a single logical group of values and functions within the wider application. An application may have a hand-full to dozens of them for a big system. Models exposing collections of data are usually made available as read-only properties returning a QAbstractItemModel-derived data model on these controllers.
Often, these controllers need to be instantiated with some initialization, as they need references to the business-logic objects they expose to the GUI, listen for signals to get notifications of changes, etc. And that's where the trouble starts...
Pre-Qt 6
Context properties
In the early days of QML, one would often use controller instances exposed to QML as context properties. Doing that allowed one to instantiate the controllers under control of C++, giving it all the references the objects needed at that time. We would often expose them to QML using a naming pattern like starting the name with a double underscore __someController so that it was easy to recognize in the QML code. Using context properties however is no longer recommended. Their lookup is slow, and the QML compilers cannot reason about them, so code using them cannot be optimized. Nor is tooling available to help the QML programmer, as code completion and the likes are not possible.
Singleton Instances
Then came the qmlRegisterSingletonInstance method. This method allowed one to register a QML singleton, but it would return the instance that you passed it as an argument and that you could instantiate however you needed. That was a good solution, but it didn't have a long useful life as it didn't mesh well with the declarative registration and it had issues with the one instance being the instance for every QML engine in your application (if you had more than one).
Post-Qt 6
Since Qt 6, the recommended way to write QML is to create QML modules using declarative registration for C++-based objects. That has many benefits in terms of tooling and optimization, so it's good practice to do this. But it also meant that since Qt 6, one could no longer mix-and match imperative registration with declarative: you either used the one, or the other; which rendered the qmlRegisterSingletonInstance method above useless.
There are many possible approaches that I have seen being applied to still control the creation of controller objects, usually by registering a singleton that has a static create factory function and returning some C++ singletons there or something along those lines. That works, but isn't very elegant. An alternative approach is using initial properties on the root object, but that either requires accessessing the root id from other QML files or propagating the controllers all the way down the stack of items. Neither is a great solution for different reasons. My colleague Javier Cordero Pérez is making a couple of videos about ways to do this, so I won't go into detail here. These videos will be added here once they have been released.
New approach
That building this connection between C++ and QML was so inelegant - despite being so important - inspired me to finally take matters into my own hands and write a patch.
The result is available starting with Qt 6.12 onward and it combines the good things of qmlRegisterSingletonInstance and the declarative registration: you still register your controller type as a QML singleton so that the type is fully known by the tooling and access to it can be optimized. But we gain back the ability to provide a ready-made instance to the QML engine.
setExternalSingletonInstance
The API on QQmlEngine gained a single new method: QQmlEngine::setExternalSingletonInstance. It allows you to provide an instance of a type declared as a singleton as the instance to use in any QML running in that engine, just like you could with qmlRegisterSingletonInstance. In contrast to that old registration function, however, you call this method on your specific QQmlEngine instance. Note that the type has to be (declaratively) registered as a singleton type for this call to work. If you are using more than one engine, it is up to you to decide if you want to provide the same instance to these different engines, or have separate instances.
This simple method gives you back an elegant, supported way to fully control the instantiation of the QML singleton, and thus easily connect it to your business logic or whatever else you need to with it. However, it is up to you make sure that you do this call before any QML code actually tries to access the singleton. Otherwise, the engine will (try to) create it's own instance as it used to. You cannot replace an already existing singleton instance, so once there is one, it is the one.
It's up to you to make sure that the provided singleton instance outlives the QML that depends on it. You can do that in any way that works in your context, but you could consider parenting the instance to the QQmlEngine instance, ordering the variables containing them on the stack correctly, or using QQmlEngine::setObjectOwnership to hand ownership of the singleton to the QML engine.
QML_UNCREATABLE for singletons
If you are providing your QML singleton instance yourself anyway, you logically also don't need it to be creatable by the engine either - although, it still can be, of course. If your controller type has a non-default constructor - perhaps to take in some references to your business logic instances - you can now mark your singleton with QML_UNCREATABLE, just like you can with other QML types. If you do that, you no longer need to supply a factory function (and even if you do, it won't be used).
Of course, if you mark a singleton as uncreatable, it is up to you to make sure you actually supply an instance via QQmlEngine::setExternalSingletonInstance before the singleton is needed from QML.
If you have been on invent.kde.org lately you might have seen some merge requests about "Install Qt metatypes" and wondered what that's all about.
When defining QML types in C++ the buildsytem tries to capture as much information about the type as possible, so that tools like qmllint, qmlls, and the QML compiler know about what API the type provides. If that information cannot be gathered the code will still work fine at runtime, but the development experience will be degraded.
Normally, when all the types involved are from the local project (or Qt), and you are using the qt_add_qml_module CMake API as well as declarative type registration, things will work mostly out of the box. However, in some cases that's not enough, and we need some extra steps.
Imagine we have a library MyLib, that exposes a class MyModel. That model isn't registered to QML at all. Now we have a program MyProgram, that creates a subclass of MyModel, and registers that to QML:
#include <MyLib/MyModel>
class MySubModel : public MyModel {
Q_OBJECT
QML_ELEMENT
...
}
This will work fine at runtime, but produces a suspicious build warning:
Warning: mysubmodel.h:3: MyModel is used as base type but cannot be found.
Opening any QML file using MySubModel in an qmlls-capable editor will show that type information for MySubModel is limited or nonexistant. So how do we fix this? Enter: metatypes files.
During the build process moc processes your classes and extracts information about properties, signals, invokables, etc. That information is then processed by the QML tooling. For Qt's own types that's done out of the box, and for custom QML module's types too, but if your custom module is using types from another library some extra steps are needed.
First, the library needs to extract its metatypes into a consumable file. This is done using the qt_extract_metatypes CMake API:
add_library(MyLib)
qt_extract_metatypes(MyLib)
This will produce a JSON file that contains information about the types in MyLib. If the QML module needing this is in the same buildsystem that's enough to make things work. However, quite often it will be used by something in another project, so we need to install the file alongside the library:
add_library(MyLib)
# the path of the generated file will be stored in ${METATYPES_FILE}
qt_extract_metatypes(MyLib OUTPUT_FILES METATYPES_FILE)
install(TARGETS MyLib)
install(FILES ${METATYPES_FILE} DESTINATION ${KDE_INSTALL_QTMETATYPESDIR})
This will install the file, but that's not enough for the consuming project to pick it up, we need to associate the metatypes file with the library. To make that happen we add the (public) sources for that library:
add_library(MyLib)
# the path of the generated file will be stored in ${METATYPES_FILE}
qt_extract_metatypes(MyLib OUTPUT_FILES METATYPES_FILE)
# extract the filename from the path
get_filename_component(METATYPES_FILE_NAME ${METATYPES_FILE} NAME)
# add metatypes file to the interface sources set
target_sources(MyLib INTERFACE $<INSTALL_INTERFACE:${KDE_INSTALL_QTMETATYPESDIR}/${METATYPES_FILE_NAME}>)
install(TARGETS MyLib)
install(FILES ${METATYPES_FILE} DESTINATION ${KDE_INSTALL_QTMETATYPESDIR})
This scary looking line of CMake basically boils down to "Everything that links against MyLib will get the installed metatypes file added to its sources". This makes the QML machinery in the application pick it up.
With that, no changes to the application are necessary. The build warning disappears, and type information in the editor starts working.
Since it's hard to know in advance whether a library's types are going to be used that way it's probably a good idea to do this for any library, especially since there's effectively no cost to this, other than some CMake code. It would be great if Qt would take care of most of that code though, see https://qt-project.atlassian.net/browse/QTBUG-123052 and related patches.
A word on the install location: When installing Qt-related files there's some subtleties involved when determining where to install things. Fortunatley ECM takes care of that for us, so it gained a new variable KDE_INSTALL_QTMETATYPESDIR for this.
This has been applied to a few KDE libraires already, but there's likely more where it would be benefitial, to allow for better QML tooling and ultimately a better developer experience.
While most new applications use the GPU for rendering to achieve better performance and battery life, there are some new applications and a lot of older applications that still use CPU rendering. More specifically relevant for KDE, while QtQuick is GPU accelerated, QtWidgets uses CPU rendering.
With CPU rendering, instead of sharing GPU buffers with the compositor, wl_shm is used to present images. "shm" stands for "shared memory", and is literally just some system memory allocated by the app and shared with the compositor.
Why is it slow?
The rendering speed of an application using CPU rendering depends a lot on what the application is doing exactly, but a very large factor is simply the sheer number of pixels and thus bytes it manipulates. With high resolution screens, especially single threaded CPU rendering can get pretty slow.
Optimizing the application side isn't my area of expertise though, and not what I'm primarily interested in as a compositor developer. My main goal is to let the application render at whatever speed it can, and to efficiently transfer the results onto the screen.
On the compositor side we can't normally use shm buffers directly. For the GPU to be able to access the data, we first need to copy it to a different buffer that meets the requirements of the GPU. This copy is often done in two steps:
copy the data to a GPU-accessible buffer on the CPU
copy that GPU-accessible buffer to another buffer in GPU memory
With both OpenGL and Vulkan, that first copy is blocking the main thread until it's complete. You can offload the copy to a different thread with some additional code, but that would just move the CPU usage, rather than reduce it.
The second copy is more acceptable, since the GPU does it asynchronously and more efficiently, but on integrated GPUs, this would still end up copying data from system memory to a different region of system memory, for no good reason.
The result of these copies is that on high resolution screens with applications using shm buffers, performance noticeably suffers and CPU usage is much higher than it has any right to be.
On my laptop with a still relatively new and high end Ryzen 7840U, I could see the cursor sometimes skip frames when quickly moving it over project files in KDevelop, since KWin's main thread was being blocked by these texture uploads. Normally that's not really noticeable, but with the power profile set to "power save", it felt really sluggish.
Vulkan will fix it… right?
When you hold a hammer, every problem starts to look like a nail.
Since we recently started using Vulkan in KWin to fix some other problems caused by OpenGL's inadequacies1, I obviously looked for a Vulkan solution first. And lo and behold, VK_EXT_external_memory_host does exist, and it's perfect for this! Or at least it looked like it would be…
The extension allows wrapping a "host pointer" (aka a normal pointer to CPU memory) in a VkBuffer or even VkImage2. With a pretty low amount of new Vulkan code, the GPU could asynchronously copy the VkBuffer to a GPU-local buffer.
Unfortunately, the implementation at least on AMD comes with some limitations. Because of potential security issues, pointers to anything associated with a file descriptor (which shm buffers always are) can't be imported this way by amdgpu.
There is also the more recent VK_EXT_host_image_copy for optimizing image uploads, but it would only allow removing the second copy rather than the first, so it's not exactly what I needed.
udmabuf to the rescue
udmabuf is a Linux driver that can wrap memfd-allocated memory in a dmabuf. A dmabuf is a handle to GPU memory, and memfd is how shm buffers are usually allocated by Wayland clients… so it's a perfect fit for what I wanted to do.
There's one caveat to this: In order to be able to create a udmabuf from it, the allocated memory must be a range of memory pages3, so location and size have to be a multiple of the page size. Applications didn't allocate their buffers with that in mind so far, since there was no benefit to it. Fixing that isn't difficult though! Assuming one memfd per shm buffer (which at least Qt does), fulfilling the page size requirement should even be free4.
With the udmabuf successfully created, we can wrap it into a VkBuffer and do an asynchronous copy to a GPU-local buffer with Vulkan. However, we can do even better: If the stride5 of the buffer matches the requirements of the driver, we can directly use the udmabuf with the GPU.
This stride requirement is a bit more of a tradeoff than the page size one, since some additional memory may need to be allocated as padding at the end of each row in the image. Since most GPUs seem to be fine with a multiple of 256, the amount of "wasted" memory is still pretty low however - for example with a 3841x2160 image, it would be 0.55MB or 1.6% more memory used per buffer.
So I added code to KWin to attempt to create a udmabuf for each shm buffer, and then import that into the GPU driver. If it fails, we just fall back to the old upload code, but if it succeeds, we don't need to do any copies at all.
The compositor side didn't take a lot of code, but the application side was much simpler still. Including a comment explaining the reasoning, it merely took a grand total of 18 changed lines of code.
The result
With the same example of KDevelop I mentioned before, the cursor is now always completely smooth. In terms of concrete numbers, KWin's CPU usage while scrolling in KDevelop went from 80-90% on one core down to 20%!
These improvements will be in Plasma 6.7 and Qt 6.11.2. I would recommend other toolkits and applications that use shm buffers to make the same changes as I did in Qt, it can make a really noticeable difference.
I'll write a blog post about it once there's more to talk about ↩
With a Vulkan renderer, the VkImage would mean the second copy could be skipped as well ↩
A page is the smallest chunk of contiguous memory managed by the OS ↩
The kernel allocates in pages, so the amount of memory used should be the same either way ↩
Stride is how many bytes are used by each row of pixels in an image. There can be unused padding after each row, which is included in the stride. ↩
KeepSecret is our new password management application, based on SecretService, which works both with our old KWallet infrastructure as well as more modern services such as oo7, KeepassXC and many others.
This release Has been focused mainly on small usability papercuts and improved messaging to the user.
An important aspect of this release is that is the first one that's available on flathub, so it's very easy to install and test now, just hop on Discover to find it.
Instead of a painstaking row-by-row or slow flame graph reviews, the QML profiler skill for agentic development allows developers to delegate code performance profiling to AI agents.
The skill guides the developer through the workflow, triggers the QML profiler, crunches through the resulting raw data, presents the performance bottlenecks in a concise report, and suggests improvements.
The skill targets 2D Qt Quick applications and supports four profiling modes - rendering, logic, memory, and full. It can also analyze an existing trace file directly, without re-running the application, for example, if the performance trace has been run on the target hardware.
Video: QML profiling of shopping list app in Claude Code CLI (Some sequences are shortened or accelerated)
It has been a long time since I wrote anything about Union, the new style engine being developed for KDE. However, that does not mean nothing has happened in that time. Quite the opposite, in fact (spoiler alert); we plan to do a first release of Union with Plasma 6.7! So let us go over some of the things that happened with Union.
CSS as Input Format
Image
CSS as used by Union for styling buttons and several other controls
One of the biggest changes that happened last year is that we switched the default input format from SVG (as used by Plasma) to CSS. Somewhere during spring last year I realised that, while using Plasma's SVG served us well to get the initial data model sorted out, it was now holding things back. Additionally, and maybe even more importantly, it was not something that I felt comfortable with shipping and recommend people build styles with.
Plasma's SVG styling, at first glance, looks pretty close to what we would want as an input format: something that would allow designers to easily create and modify styles. The feedback from designers who worked with Plasma's SVG styling was the opposite; getting everything right in the SVG for things to work correctly is a lot harder than it seems, with a number of quirks specific to Plasma that need to be considered, as well as limitations with regards to what features are supported. Additionally, there are several technical issues that made me uncomfortable relying too much on SVGs as an input format.
So we wanted to switch to CSS, which has always been the input format I wanted to use. It is well known, has a fairly reasonable syntax and is already being used for the task we want to use it for.
However, there is not really a good library available that parses CSS. The best I could find is the Rust cssparser crate from the Servo project. Unfortunately, this is Rust and Union is a C++ project. Additionally, it is slightly too low level for Union to use directly, it is more of a parser-building toolkit than a real parser. So I created the cxx-rust-cssparser library that makes use of the cssparser crate and provides a C++ interface to parse CSS files. Then I hooked that up as a new input format for Union to use and we could start styling things with CSS.
Breeze in CSS
So we set out to build yet-another implementation of Breeze, this time using Union's CSS. We needed an initial default style for Union, and while there's quite some work happening on a design system and a new style, we wanted a known baseline to work against, similar to what we did with the Plasma SVG input in the first place. This does not mean Breeze will be the only style supported by Union, but rather it limits the scope of the initial work to what is needed for Breeze to work.
Image
A comparison of four different implementations of Breeze-styled buttons.
As it turns out, because we have multiple implementations of the same thing, there are slight differences between those implementations. Some of them have been intentional, such as qqc2-breeze-style using a different spin box style from the Breeze widgets style; others less so. To make it easy to compare what Union's implementation looked like compared to the other styles, I created a simple application that displays pages with controls side-by-side, each page using a different implementation of Breeze.
Union's implementation of Breeze is not a one-to-one copy of the Qt Widgets implementation as exposed by qqc2-desktop-style, though it comes quite close. However, in certain cases, we intentionally deviate, either because the different implementations disagree on how to do something, or because Union allows things that we could not do before.
Image
Comparing different implementations of spin boxes; Union is on the far left.
For example, consider the spin boxes above. We have long wanted to use a style where the "increase" and "decrease" buttons are bigger and to the left and right of the value. Both qqc2-breeze-style and Plasma already use this style. However, the QtWidgets style uses a version with small up and down arrows stacked on the right, because a lot of widgets applications expect a spinbox to be fairly small. For Union, we actually implement both: If the spinbox has enough space, we use the preferred style with buttons left and right, but if the spinbox is constrained somehow we switch to the style with smaller buttons stacked to the right. This gives us the best of both worlds, where we can use the preferred style but have a decent fallback for cases where that does not fit.
Union's First Release?!
Image
System Settings' Keyboard page as styled by Union
The work on Union's Breeze implementation has progressed to the point where it is very hard to distinguish whether or not you are running the Union version. We have also tested with a bunch of applications and made sure that any differences were fixed. So we are at a stage where we need to get Union into the hands of more people, both to get extra people testing whether there are any major issues, but also to have interested people creating new styles.
This means that with the upcoming Plasma 6.7 release, we plan to include Union. Discussion is currently ongoing whether we will enable it by default, but even if not there will be a way to try it out.
Looking to the Future
I have so far mostly talked about the Qt Quick side of things, but what about Qt Widgets? We so far have focused on the Qt Quick output to have something that we could ship in a good working state. With that work nearing completetion, we can look forward to what to do next.
We already have a prototype implementation of a Qt Widgets output for Union. One of the next goals will be to flesh this out further into a complete QStyle implementation that is as usable as the Qt Quick output.
Image
KDE Plasma using Union for the configuration window. Panel and applet styling is one thing we will add in the future.
Another item we will be looking at is to use Union's CSS input for styling Plasma, making it possible for Union styles to provide styling for things like panels and widget popups. Note that we will not drop support for SVG styling in Plasma, but we do hope we can make Union's CSS-based styling the primary way to style Plasma in the future.
We also want to expand the things Union can do, so that designers are not limited in what kind of styles they can create using Union. This includes things like expanding the support for specific CSS features, adding support for more rendering options and reconsidering how we handle colours.
In Closing
As you can see, there is still a lot of work to be done for Union to unionize all of KDE's styling. For that, we would love to hear your experience with it! If you test out Union, please report every problem you find with any existing Qt Quick-based application. See the README for instructions on how to test. You can report issues here.
If you are a style designer interested in creating new styles, we are working on documentation that explains how to do so, in the meantime, feel free to experinent with the CSS files used by the Breeze style.
If you are interested in working on Union itself, or otherwise would like to discuss something relating to Union, feel free to drop by on Matrix. If you want to get a closer look at what is currently in development, we make heavy use of Gitlab's issues for development tracking,
Reviewing, auditing, or sanity-checking code usually means running separate linters, reading through checklists, and manually verifying Qt-specific patterns across dozens of files. The Qt code review skills help developers to automate part of this code review phase. Developers avoid a laborious manual walkthrough of every file, with the AI agent running a deterministic linter followed by six parallel deep-analysis agents and surfacing real issues with mitigations in a few minutes.
Thank you to the KDE community and Jean-Baptiste for selecting my proposal. Congratulations to all other accepted contributors!
I'll be working on improving the effect widget system in Kdenlive this summer. As someone who uses Kdenlive daily for my own videos, these are problems I've personally hit, which makes this project feel very personal.
Here are my three main goals:
Curves Widget
Replace the channel dropdown with a tab-based interface so each color channel (RGB, Red, Green, Blue, Alpha, Luma) has its own independent curve. Currently you need to apply the effect three times to get per-channel control, this fixes that with a single effect instance.
Gradient Editor
Build a standalone gradient widget with support for arbitrary draggable color stops, replacing the current hardcoded two-stop system in the effects panel.
Speed Ramp
Add bezier curve handles to the time remapping panel so speed transitions can ease in and out smoothly, with presets like Ease In, Ease Out, Ease In/Out, and Linear.
Coding begins May 25. I'll be posting weekly updates here throughout the summer. Looking forward to a productive GSoC!
For a long time, developers of Qt-based C++ applications have only had one option for embedding web content: Qt WebEngine. And while it offers a large API with many useful features, the module has its downsides, since it can consume a lot of system resources and increase binary size. For QML users, we've had an alternative in the Qt WebView module, but that API had never been exposed to C++ until now.
I'm very excited to announce that I've been selected for Google Summer of Code 2026 with KStars to work on the AI-Powered Guiding Assistant for Ekos. Here's what that means and what I'll be building. What is KStars? KStars is a free, open-source desktop planetarium developed by KDE. But it's much more than just a star map. It includes Ekos, a full-featured astrophotography suite that controls telescopes, cameras, filter wheels, and focusers. Astrophotographers around the world use Ekos to automate their imaging sessions, including a critical feature called autoguiding. The Problem: Why Stars Get Streaked When you take a long-exposure photo of the night sky, your telescope mount needs to track the stars precisely. The mount has motors that rotate to compensate for Earth's rotation, but no mount is perfect. Gears have tiny imperfections, wind pushes the telescope tube, and gravity causes flexure as the telescope moves across the sky. Autoguiding compensates for this by using a second camera to watch a single "guide star" every few seconds. If the star drifts, the software sends a correction pulse to nudge the mount back. Think of it as cruise control for your telescope, constantly making micro-adjustments to keep everything steady. The current system uses a reactive proportional controller it measures the star's drift and applies a correction pulse scaled by an aggressiveness parameter. Star drifts right → push left, proportionally. Simple, but it only corrects after the error has already happened. A wind gust causes a spike, the controller overcorrects, and the system oscillates. Beginners spend hours manually tuning parameters like aggressiveness and hysteresis, and even experienced users struggle when conditions change mid-session. There's also Gaussian Process Guiding (GPG), which uses a composite kernel (periodic + squared-exponential) to model the mount's periodic gear error and predict corrections ahead of time. It's elegant and works well for smooth, repeating worm-gear patterns but it struggles with discontinuous behaviors like gear backlash dead zones, sudden impulse disturbances from wind gusts, or complex non-stationary vibrations that don't fit its kernel assumptions. My Project: An AI That Learns Your Mount Rather than relying entirely on reactive corrections or assuming errors follow smooth mathematical models, my project introduces a machine learning layer that learns the specific mechanical quirks of each user's telescope mount. The key insight: every mount has a unique "mechanical fingerprint" which is characterized by tracking errors from the drive system, backlash dead zones when motors reverse direction, and flexure patterns that shift with sky position. These behaviors vary widely: a traditional worm-gear mount has different quirks than a modern harmonic drive, and even two identical mounts will differ due to manufacturing tolerances. A neural network (GRU - Gated Recurrent Unit) can learn these mount-specific patterns from real guiding data and predict corrections before the error fully manifests without assuming any particular drive type. How It Works The system operates in two distinct modes: Data Collection Mode - During a normal guiding session, KStars silently logs all the telemetry in the background: star drift (dx/dy), time between frames, signal-to-noise ratio, star mass, mount altitude, and the correction pulses the guide algorithm sent. This builds a dataset of the mount's real-world behavior which is its unique mechanical fingerprint. Training the Model One challenge here is that we're logging the guide algorithm's imperfect corrections, not "correct" outputs. The mount's response is non-linear (backlash, stiction, load-dependent friction), so we can't simply assume that adding more pulse always fixes the error proportionally. I plan to explore two approaches: Approach 1 - Retrospective labeling: The training script processes the logs offline, so it can look ahead. For each frame t, it checks the residual drift at frame t+1 and uses Ekos's calibration scale (ms-per-pixel) to calculate what the ideal pulse should have been. For example, if the guide algorithm sent 50ms but 0.2px of residual error remained, and calibration says 100ms/px, the ideal pulse was 50 + 20 = 70ms. The GRU learns to predict that retrospectively computed ideal pulse. This is a simple first approximation, the linearity assumption won't hold perfectly, but over many samples the network should learn the underlying patterns. Approach 2 - Predictive drift model: Instead of predicting the ideal pulse, the GRU predicts the residual drift at the next timestep given recent drift history and pulses sent. The correction delta is derived from that prediction, similar in spirit to how GPG predicts periodic error from observed residuals. This avoids the linearity assumption entirely since we're predicting drift, not pulses. I'll experiment with both and see which handles real mount behavior better. Inference A Python script (PyTorch) trains the GRU model on the logged data offline on the user's laptop. The trained weights are exported to a simple JSON file, which KStars loads and executes natively in C++ using Eigen3 matrix math. No heavy dependencies, no GPU required it runs on a Raspberry Pi. The AI never replaces the existing guide algorithm. It augments it by adding a learned correction delta on top of the standard pulse, clamped to safe limits. If the AI outputs garbage, the proportional controller keeps working. Safety first. Concrete Goals Here's exactly what I'll deliver by the end of GSoC: | Deliverable | Description | |:| | Telemetry Pipeline | A data logger inside Ekos that records guide star drift, SNR, pulses, and mount state to CSV | | Python Training Script | A PyTorch script that trains a GRU model on the logged data and exports weights | | C++ Inference Engine | Hand-written GRU feed-forward math using Eigen3 - lightweight enough for Raspberry Pi | | Pulse Augmentation Layer | Logic to blend AI predictions with the existing guide corrections, with safety clamping | | UI Controls | Training/Inference mode toggles, aggressiveness slider, and RMS comparison plots in the Ekos Guide panel | | Documentation & Tests | Developer docs, KStars Handbook updates, and unit tests for the inference pipeline | Why This Excites Me I'm a fourth-year Theoretical Computer Science student who is very much interested in the field of Machine Learning and Deep Learning. I'm taking a course on Deep Learning this semester which gave me the opportunity to learn more about the RNN architecture and its extensions such as LSTM and GRU. I also participated in Season Of KDE 2026, which introduced me to a lot of new open source software and made me interested in contributing to open source. So this project is a perfect combination of my passion for Deep Learning and my interest in contributing to open source software. Even though I'm very new to astrophotography, I'm very interested in learning more about this domain. Follow Along I'll be posting updates throughout the summer right here on this blog. You can also follow my progress on: - KDE Invent: pavansg - Matrix: @pavankumarsg:matrix.org If you're an astrophotographer, a KDE contributor, or just someone curious about applying ML to real world hardware problems. I'd love to hear from you. Mentor: Jasem Mutlaq (@mutlaqja).
