28 May 2025

feedPlanet Python

Kay Hayen: Nuitka Release 1.2

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release contains a large amount of new compatibility features and a few new optimization, while again consolidating what we have. Scalability should be better in many cases.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is again mainly a consolidation of previous release, as well as finishing off existing features. Optimization added in previous releases should have all regressions fixed now, again with a strong series of hotfixes.

New optimization was focused around findings with static optimization not being done, but still resulting in general improvements. Letting static optimization drive the effort is still paying off.

Scalability has seen improvements through some of the optimization, this time a lot less anti-bloat work has been done, and some things are only started. The focus will clearly now shift to making this a community effort. Expect postings that document the Yaml format we use.

For macOS specifically, with the sections work, onefile should launch faster, should be more compatible with signing, and can now be used in notarization, so for that platform, things are more round.

For Windows, a few issues with version information and onefile have been addressed. We should be able to use memory mapped view on this platform too, for faster unpacking of the payload, since it doesn't have to go through the file anymore.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 1.1

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release contains a large amount of new compatibility features, while consolidating what we have. Scalability should be better in some cases.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is mainly a consolidation of previous release. Optimization added in previous release did in fact introduce regressions, that needed to be addressed and were cause for relatively many hotfixes.

The Yaml Nuitka Package Configuration feature is getting ever more powerful, but is not one bit more documented, such that the community as a whole is not yet capable of adding missing dependencies, data files, DLLs, and even anti-bloat patches.

New optimization was focused around compatibility with very few exceptions, where the non-automatic standard library work is standing out, and allows for smaller binaries in many cases.

Scalability has seen improvements through a few optimization, but mainly again with anti-bloat work being done. This is owed to the fact that consolidation was the name of the game.

For Anaconda specifically, a lot more software is covered, and generally, cv2 and torch related tools are now working better, but it seems DLL handling will remain problematic in many instances.

The compilation report contains much more information and is getting there is terms of completeness. At some point, we should ask for it in bug reports.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 1.0

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release contains a large amount of new features, while consolidating what we have with many bug fixes. Scalability should be dramatically better, as well as new optimization that will accelerate some code quite a bit. See the summary, how this release is paving the way forward.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release improves on many things at once. A lot of work has been put into polishing the Yaml configuration that now only lacks documentation and examples, such that the community as a whole should become capable of adding missing dependencies, data files, DLLs, and even anti-bloat patches.

Then a lot of new optimization has been done, to close the missing gaps with dict and str methods, but before completing list which is already a work in progress pull request, and bytes, we want to start and generate the node classes that form the link or basis of dedicated nodes. This will be an area to work on more.

The many improvements to existing code helpers, and them being able to pick target types for the arguments of comparisons and binary operations, is a pre-cursor to universal optimization of this kind. What is currently only done for constant values, will in the future be interesting for picking specific C types for use. That will then be a huge difference from what we are doing now, where most things still have to use PyObject * based types.

Scalability has again seen very real improvements, memory usage of Nuitka itself, as well as compile time inside Nuitka are down by a lot for some cases, very noticeably. There is never enough of this, but it appears, in many cases now, large compilations run much faster.

For macOS specifically, the new DLL dependency analysis, is much more capable or resolving conflicts all by itself. Many of the more complex packages with some variants of Python, specifically Anaconda will now be working a lot better.

And then, of course there is the big improvement for Onefile, that allows to use cached paths. This will make it more usable in the general case, e.g. where the firewall of Windows hate binaries that change their path each time they run.

Future directions will aim to make the compilation report more concise, and given reasons and dependencies as they are known on the inside more clearly, such that is can be a major tool for testing, bug reporting and analysis of the compilation result.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.9

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release has a many optimization improvements, and scalability improvements, while also adding new features, with also some important bug fixes.

Bug Fixes

New Features

Optimization

Cleanups

Organizational

Tests

Summary

This release marks a point, that will allow us to open up the compatibility work for implicit dependencies and anti-bloat stuff even further. The Yaml format will need documentation and potentially more refinement, but will open up a future, where latest packages can be supported with just updating this configuration.

The scalability improvements really make a difference for many libraries and are a welcome improvement on both memory usage and compile time. They are achieved by an accord of static optimization of

One optimization aimed at optimizing tuple unpacking, was not finished in time for this release, but will be subject of a future release. It has driven many other improvements though.

Generally, also from the UI, this is a huge step forward. With links to the website for complex topics being started, and the progress bar flicker being removed, the tool has yet again become more user friendly.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.8

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release has a massive amount of bug fixes, builds on existing features, and adds new ones.

Bug Fixes

New Features

Optimization

Cleanups

Organizational

Tests

Summary

This release has seen a lot of consolidation. The plugins layer for data files is now all powerful, allowing much nicer handling of them by the plugins, they are better reported in normal output, and they are also part of the report file that Nuitka can create. You may now also inhibit their inclusion from the command line, if you decide otherwise.

The pyproject.toml now supporting Nuitka arguments is closing an important gap.

Generally many features got more polished, e.g. non-automatic inclusion of stdlib modules has most problems fixed up.

An important area of improvement, are the hard imports. These will be used to replace the source based resolution of package requirements with ones that are proper nodes in the tree. Getting these hard imports to still retain full compatibility with existing imports, that are more or less __import__ uses only, was revealing quite a bit of technical debt, that has been addressed with this release.

For onefile, the cached mode is being prepared with the variables added, but will only be in a later release.

Also a bunch of new or upgraded packages are working now, and the push for anti-bloat work has increased, making many compilations even more lean, but scalability is still an issue.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.7

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release is massively improving macOS support, esp. for M1 and the latest OS releases, but it also has massive improvements for usability and bug fixes in all areas.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is tremendous progress for macOS. Finally biting the bullet and paying obscene amounts of money to rent an M1 machine, it was possible to enhance the support for this platform. Currently typical packages for macOS are being made compatible as well, so it can now be expected to perform equally well.

On the quality side, the spell checker has had some positive effects, finding typos and generally misspelled code, that codespell does not, due to it being very conservative.

The trend to enhance plugins has continued. The copying of DLLs is very nearly finalized. Making more plugins enabled by default is seeing a lot of progress, with 2 important ones addressed.

Work on the size of distributions has seen a lot of positive results, in that now standalone distributions are often very minimal, with many extension modules from standard library no longer being present.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.9

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This releases contains important bug fixes for regressions of the 0.6.8 series which had relatively many problems. Not all of these could be addressed as hotfixes, and other issues were even very involved, causing many changes to be necessary.

There are also many general improvements and performance work for tracing and loops, but the full potential of this will not be unlocked with this release yet.

Bug Fixes

New Features

Optimization

Cleanups

Organizational

Summary

The main focus of this release has been bug fixes with only a little performance work due to the large amount of regressions and other findings from the last release.

The new constants loading for removes a major scalability problem. The checked and now consistently possible use of ccache and clcache allows for much quicker recompilation. Nuitka itself can still be slow in some cases, but should have seen some improvements too. Scalability will have to remain a focus for the next releases too.

The other focus, was to make the binaries contain no original path location, which is interesting for standalone mode. Nuitka should be very good in this area now.

For optimization, the new loop code is again better. But it was also very time consuming, to redo it, yet again. This has prevented other optimization to be added.

And then for correctness, the locals scope work, while very invasive, was necessary, to handle the usage of locals inside of contractions, but also will be instrumental for function inlining to become generally available.

So, ultimately, this release is a necessary intermediate step. Upcoming releases will be able to focus more clearly on run time performance again as well as on scalability for generated C code.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.8

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This releases contains important general improvements and performance improvements and enhanced optimization as well as many bug fixes that enhance the Python 3.8 compatibility.

Bug Fixes

New Features

Optimization

Organizational

Tests

Cleanups

Summary

This release is huge in terms of bugs fixed, but also extremely important, because the new loop SSA and type tracing, allows for many more specialized code usages. We now can trace the type for some loops to be specifically an integer or long value only, and will become able to generate code that avoids using Python objects, in these cases.

Once that happens, the performance will make a big jump. Future releases will have to consolidate the current state, but it is expected that at least an experimental addition of C type float or C long can be added, add to that iterator type shape and value analsis, and an actual jump in performance can be expected.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.3

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This has a focus on organizational improvements. With more and more people joining Nuitka, normal developers as well as many GSoC 2019 students, the main focus was to open up the development tools and processes, and to improve documentation.

That said, an impressive amount of bug fixes was contributed, but optimization was on hold.

Bug Fixes

Optimization

New Features

Organizational

Cleanups

Tests

Summary

With the many organizational changes in place, my normal work is expected to resume for after and yield quicker improvements now.

It is also important that people are now enabled to contribute to the Nuitka web site and the Nuitka speedcenter. Hope is to see more improvements on this otherwise neglected areas.

And generally, it's great to see that a community of people is now looking at this release in excitement and pride. Thanks to everybody who contributed!

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.2

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release has a huge focus on organizational things. Nuitka is growing in terms of contributors and supported platforms.

Bug Fixes

New Features

Optimization

Organizational

Summary

This release is a sign of increasing adoption of Nuitka. The GSoC 2019 is showing early effects, as is more developers joining the effort. These are great times for Nuitka.

This release has not much on the optimization side that is user visible, but the work that has begun is capable of producing glorious benchmarks once it will be finished.

The focus on this and coming releases is definitely to open up the Nuitka development now that people are coming in as permanent or temporary contributors in (relatively) high numbers.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.19

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release adds support for 3.10 while also adding very many new optimization, and doing a lot of bug fixes.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is another big step forward.

The amount of optimization added is again very large, some of which yet again unlocks more static optimization of module imports, that previously would have to be considered implicit. Now analyzing these on the function level as well, we can start searching for cases, where it could be done, but is not done yet.

After starting with dict, method optimization has focused on str which is esp. important for static optimization of imports. The next goal will here be to cover list which are important for run time performance and currently not yet optimized. Future releases will progress there, and also add more types.

The C type specialization for Python3 has finally progressed, such that it is also covering the long and unicode and as such not limited to Python2 as much. The focus now needs to turn back to not working with PyObject * for these types, but e.g. with += 1 to make it directly work with CLONG rather than LONG for which structural changes in code generation will be needed.

For scalability, the anti-bloat work has not yet progressed as much as to be able to enable it by default. It needs to be more possible to disable it where it causes problems, e.g. when somebody really wants to include pytest and test frameworks generally, that's something that needs to be doable. Compiling without anti-bloat plugin is something that is immediately noticeable in exploding module amounts. It is very urgently recommended to enable it for your compilations.

The support for Windows has been further refined, actually fixing a few important issues, esp. for the Qt bindings too.

This release adds support for 3.10 outside of very special match statements, bringing Nuitka back to where it works great with recent Python. Unfortunately orderedset is not available for it yet, which means it will be slower than 3.9 during compilation.

Overall, Nuitka is closing many open lines of action with this. The setuptools support has yet again improved and at this point should be very good.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.18

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release has a focus on new features of all kinds, and then also new kinds of performance improvements, some of which enable static optimization of what normally would be dynamic imports, while also polishing plugins and adding also many new features and a huge amount of organizational changes.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is again a huge step forward. It refines on PGO and LTO for C level to work with all relevant compilers. Internally Python level PGO is prepared, but only a future release will feature it. With that, scalability improvements as well as even more performance improvements will be unlocked.

The amount of optimization added this time is even bigger, some of which unlocks static optimization of module imports, that previously would have to be considered implicit. This work will need one extra step, namely to also trace hard imports on the function level, then this will be an extremely powerful tool to solve these kinds of issues in the future. The next release will have this and go even further in this area.

With the dictionary methods, and some string methods, also a whole new kind of optimization has been started. These will make working with dict containers faster, but obviously a lot of ground is to cover there still, e.g. list values are a natural target not yet started. Future releases will progress here.

Type specialization for Python3 has not progressed though, and will have to be featured in a future releases though.

For scalability, the anti-bloat work has continued, and this should be the last release, where this is not on by default. Compiling without it is something that is immediately noticeable in exploding module amounts. It is very urgently recommended to enable it for your compilations.

The support for macOS has been refined, with version information being possible to add, and adding information to the binary about which OSes are supported, as well as rejecting Apple Python, which is only a trap if you want to deploy to other OS versions. More work will be needed to support pyenv or even Homebrew there too, for now CPython is still the recommended platform to use.

This release achieves major compatibility improvements. And of course, the experimental support for 3.10 is not the least. The next release will strive to complete the support for it fully, but this should be usable at least, for now please stay on 3.9 if you can.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.17

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release has a focus on performance improvements, while also polishing plugins and adding many new features.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release is one of the most important ones in a long time. The PGO and LTO, and static libpython work make a big different for performance of created binaries.

The amount of optimization added is also huge, calls are much faster now, and object creations too. These avoiding to go through actual dictionaries and tuples in most cases when compiled code interacts gives very significant gains. This can be seen in the increase of pystone performance.

The new type specializations allow many operations to be much faster. More work will follow in this area and important types, str and int do not have specialized comparisons for Python3, holding it back somewhat to where our Python2 performance is for these things.

For scalability, the anti-bloat work is extremely valuable, and this plugin should become active by default in the future, for now it must be strongly recommended. It needs more control over what parts you want to deactivate from it, in case of it causing problems, then we can and should do it.

The support for macOS has been enhanced a lot, and will become perfect in the next release (currently develop). The bundle mode is needed for all kinds of GUI programs to not need a console. This platform is becoming as well supported as the others now.

Generally this release marks a huge step forward. We hope to add Python level PGO in the coming releases, for type knowledge retrofitted without any annotations used. Benchmarks will become more fun clearly.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.15

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release polished previous work with bug fixes, but there are also important new things that help make Nuitka more usable, with one important performance improvement.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

For Windows, it's now easier than ever to create an icon for your deployment, because you can use PNG files, and need not produce ICO files anymore, with Nuitka doing that for you.

The onefile for Linux had some more or less severe problems that got addressed, esp. also when it came to QML applications with PySide.

On the side, we are preparing to greatly improve the caching of Nuitka, starting with retaining modules that were demoted to bytecode. There are changes in this release, to support that, but it's not yet complete. We expect that scalability will then be possible to improve even further.

Generally this is mostly a maintenance release, which outside of the threading performance improvement has very little to offer for faster execution, but that actually does a lot. Unfortunately right now it's limited to 3.9, but some of the newer Python's will also be supported in later releases.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.13

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release follows up with yet again massive improvement in many ways with lots of bug fixes and new features.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

Where the big change of the last release were optimization changes to reduce the global passes, this release addresses remaining causes for extra passes, that could cause these to still happen. That makes sure, Nuitka scalability is very much enhanced in this field again.

The new features for forced outputs are at this time Windows only and make a huge difference when it comes to providing a way to debug Windows Services or programs in general without a console, i.e. a GUI program. These will need even more specifiers, e.g. to cover program directory, rather than exe filename only, but it's a very good start.

On the tooling side, not a lot has happened, with the clcache fix, it seems that locking issues on Windows are gone.

The plugin changes from previous releases had left a few of them in a state where they were not working, but this should be restored. Interaction with the plugins is being refined constantly, and this releases improved again on their interfaces. It will be a while until this becomes stable.

Adding support for PySide2 is a headline feature actually, but not as perfect as we are used to in other fields. More work will be needed, also in part with upstream changes, to get this to be fully supported.

For the performance side of things, the in-place work and the binary operations work has taken proof of concept stuff done for Python2 and applied it more universally to Python3. Until we cover all long operations, esp. * seems extremely important and is lacking, this cannot be considered complete, but it gives amazing speedups in some cases now.

Future releases will revisit the type tracing to make sure, we know more about loop variables, to apply specific code helpers more often, so we can achieve the near C speed we are looking for in the field of int performance.

28 May 2025 6:31pm GMT

Kay Hayen: Nuitka Release 0.6.11

This is to inform you about the new stable release of Nuitka. It is the extremely compatible Python compiler, "download now".

This release is a massive improvement in many ways with lots of bug fixes and new features.

Bug Fixes

New Features

Optimization

Organizational

Cleanups

Tests

Summary

This release has seen Python3.9 and Onefile both being completed. The later needs compression added on Windows, but that can be added in a coming release, for now it's fully functional.

The focus clearly has been on massive cleanups, some of which will affect compile time performance. There is relatively little new optimization otherwise.

The adoption of clcache enables a very fast caching, as it's now loaded directly into the Scons process, avoiding a separate process fork.

Generally a lot of polishing has been applied with many cleanups lowering the technical debt. It will be interesting to see where the hard module imports can lead us in terms of more optimization. Static optimization of the Python version comparisons and checks is needed to lower the amount of imports to be processed.

Important fixes are also included, e.g. the constants loading performance was too slow in some cases. The multiprocessing on Windows and numpy plugins were regressed and finally everything ought to be back to working fine.

Future work will have to aim at enhanced scalability. In some cases, Nuitka still takes too much time to compile if projects like Pandas include virtually everything installed as an option for it to use.

28 May 2025 6:31pm GMT