fidzu : kde

Today's the big release for Oracle Solaris 11, after 7 years of development. For me, the Solaris 11 release comes a little more than 11 years after I joined the X11 engineering team at what was then Sun, and finishes off some projects that were started all the way back then.

For instance, when I joined the X team, Sun was finishing off the removal of the old OpenWindows desktop, and we kept getting questions asking about the rest of the stuff being shipped in /usr/openwin, the directory that held both the OpenLook applications and the X Window System software. I wrote up an ARC case at the time to move the X software to /usr/X11, but there were various issues and higher priority work, so we didn't end up starting that move until near the end of the Solaris 10 development cycle several years later. Solaris 10 thus had a mix of the recently added Xorg server and related code delivered in /usr/X11, while most of the existing bits from Sun's proprietary fork of X11R6 were still in /usr/openwin.

During Solaris 11 development, we finished that move, and then jumped again, moving the programs directly into /usr/bin, following the general Solaris 11 strategy of using /usr/bin for most of the programs shipped with the OS, and using other directories, such as /usr/gnu/bin, /usr/xpg4/bin, /usr/sunos/bin, and /usr/ucb for conflicting alternate implementations of the programs shipped in /usr/bin, no longer as a way to segregate out various subsystems to allow the OS to better fit onto the 105Mb hard disks that shipped with Sun workstations back when /usr/openwin was created. However, if for some reason you wanted to build your own set of X binaries, you could put them in /usr/X11R7 (as I do for testing builds of the upstream git master repos), and then put that first in your $PATH, so nothing is really lost here.

The other major project that was started during Solaris 10 development and finished for Solaris 11 was replacing that old proprietary fork of X11R6, including the Xsun server, with the modernized, modularized, open source X11R7.* code base from the new X.Org, including the Xorg server. The final result, included in this Solaris 11 release, is based mostly on the X11R7.6 release, including recent additions such as the XCB API I blogged about last year, though we did include newer versions of modules that had upstream releases since the X11R7.6 katamari, such as Xorg server version 1.10.3.

That said, we do still apply some local patches, configuration options, and other changes, for things from just fitting into the Solaris man page style or adding support for Trusted Extensions labeled desktops. You can see all of those changes in our source repository, which is searchable and browsable via OpenGrok on src.opensolaris.org (or via hgweb on community mirrors such as openindiana.org) and available for anonymous hg cloning as well. That xnv-clone tree is now frozen, a permanent snapshot of the Solaris 11 sources, while we've created a new x-s11-update-clone tree for the Solaris 11 update releases now being developed to follow on from here.

Naturally, when your OS has 7 years between major release cycles, the hardware environment you run on greatly changes in the meantime as well, and as the layer that handles the graphics hardware, there have been changes due to that. Most of the SPARC graphics devices that were supported in Solaris 10 aren't any more, because the platforms they ran in are no longer supported - we still ship a couple SPARC drivers that are supported, the efb driver for the Sun XVR-50, XVR-100, and XVR-300 cards based on the ATI Radeon chipsets, and the astfb driver for the AST2100 remote Keyboard/Video/Mouse/Storage (rKVMS) chipset in the server ILOM devices. On the x86 side, the EOL of 32-bit platforms let us clear out a lot of the older x86 video device drivers for chipsets and cards you wouldn't find in x64 systems - of course, there's still many supported there, due to the wider variety of graphics hardware found in the x64 world, and even some recent updates, such as the addition of Kernel Mode Setting (KMS) support for Intel graphics up through the Sandy Bridge generation.

For those who followed the development as it happened, either via watching our open source code releases or using one of the many development builds and interim releases such as the various Solaris Express trains, much of this is old news to you. For those who didn't, or who want a refresher on the details, you can see last year's summary in my X11 changes in the 2010.11 release blog post. Once again, the detailed change logs for the X11 packages are available, though unfortunately, all the links in them to the bug reports are now broken, so browsing the hg history log is probably more informative.

Since that update, which covered up to the build 151 released as 2010.11, we've continued development and polishing to get this Solaris 11 release finished up. We added a couple more components, including the previously mentioned xcb libraries, the FreeGLUT library, and the Xdmx Distributed Multihead X server. We cleaned up documentation, including the addition of some docs for the Xserver DTrace provider in /usr/share/doc/Xserver/. The packaging was improved, clearing up errors and optimizing the builds to reduce unnecessary updates. A few old and rarely used components were dropped, including the rstart program for starting up X clients remotely (ssh X forwarding replaces this in a more secure fashion) and the xrx plugin for embedding X applications in a web browser page (which hasn't been kept up to date with the rapidly evolving browser environment). Because Solaris 11 only supports 64-bit systems, and most of the upstream X code was already 64-bit clean, the X servers and most of the X applications are now shipped as 64-bit builds, though the libraries of course are delivered in both 32-bit and 64-bit versions for binary compatibility with applications of each flavor. The Solaris auditing system can now record each attempt by a client to connect to the Xorg server and whether or not it succeeded, for sites which need that level of detail.

In total, we recorded 1512 change request id's during Solaris 11 development, from the time we forked the "Nevada" gate from the Solaris 10 release until the final code freeze for todays release - some were one line bug fixes, some were man page updates, some were minor RFE's and some were major projects, but in the end, the result is both very different (and hopefully much better) than what we started with, and yet, still contains the core X11 code base with 24 years of backwards compatibility in the core protocols and APIs.

09 Nov 2011 10:10pm GMT

After introducing the automoc feature in my last blog, here comes the next part of this series. More will follow.

The new --find-package mode of CMake

Typically, in projects which are built using autotools or handwritten Makefiles, the tool pkg-config is used to find whether and where some library, used by the software, is installed on the current system, and prints the respective command line options for the compiler to stdout.

Since CMake 2.8.6, also CMake can be used additionally to or instead of pkg-config in such projects to find installed libraries.

With version 2.8.6 CMake features the new command line flag --find-package. When called in this mode, CMake produces results compatible to pkg-config, and can thus be used in a similar way.

E.g. to get the compiler command line arguments for compiling an object file, it can be called like this:

   $ cmake --find-package -DNAME=LibXml2 -DLANGUAGE=C -DCOMPILER_ID=GNU -DMODE=COMPILE
   -I/usr/include/libxml2
   $

To get the flags needed for linking, do

   $ cmake --find-package -DNAME=LibXml2 -DLANGUAGE=C -DCOMPILER_ID=GNU -DMODE=LINK
   -rdynamic -lxml2
   $

As result, the flags are printed to stdout, as you can see.

The required parameters are

• NAME - this is the name of the package, the same way as it is used in
  find_package() calls when using CMake
• LANGUAGE : which language is used, possible options are C, CXX, Fortran and ASM
• MODE : either COMPILE, LINK or EXIST
• COMPILER_ID: this identifies which compiler you are using, so CMake can print
  the options as necessary for this toolchain. When using the GNU compiler, use "GNU"
  here. Many other compilers are compatible to the command line syntax of gcc. Other
  options are "Intel", "Clang", "MSVC" etc.

So, you can insert calls like the above in your hand-written Makefiles.
For using CMake in autotools-based projects, you can use cmake.m4, which is now also installed by CMake.
This is used similar to the pkg-config m4-macro, just that it uses CMake internally instead of pkg-config. So your configure.in could look something like this:

   ...
   PKG_CHECK_MODULES(XFT, xft >= 2.1.0, have_xft=true, have_xft=false)
    if test $have_xft = "true"; then
        AC_MSG_RESULT(Result: CFLAGS: $XFT_CFLAGS LIBS: $XFT_LIBS)
    fi

   CMAKE_FIND_PACKAGE(LibXml2, C, GNU)
   AC_MSG_RESULT(Result: CFLAGS: $LibXml2_CFLAGS LIBS: $LibXml2_LIBS)
   ...

This will define the variables LibXml2_CFLAGS and LibXml2_LIBS, which can then be used in the Makefile.in/Makefiles.

What does that mean for developers of CMake-based libraries ?

You don't have to install pkg-config pc-files anymore, just install a Config.cmake file for CMake, and both CMake-based and also autotools-based or any other projects can make use of your library without problems.
Documentation how this is done can be found here:

• http://www.cmake.org/Wiki/CMake/Tutorials#CMake_Packages
• https://projects.kde.org/projects/kde/kdeexamples/repository/revisions/master/show/buildsystem

What does that mean for developers working on e.g. autotools-based projects, and using a project built with CMake ?

Take a look at the cmake_find_package() m4-macro installed since CMake 2.8.6 in share/aclocal/cmake.m4, it contains documentation, and will help you using that library.
Thanks go to Matthias Kretz of Phonon fame, now working on HPC stuff, who wrote the cmake.m4 from scratch (which was necessary since it had to be BSD-licensed in order to be included in CMake).

Internals

Internally, CMake basically executes a find_package() with the given name, turns the results into the command line options for the compiler and prints them to stdout.
This means it works basically for all packages for which a FindFoo.cmake file exists or which install a FooConfig.cmake file.
There is one issue though: FindFoo.cmake files, which execute try_compile() or try_run() commands internally, are not supported, since this would required setting up and testing the compiler toolchain completely.
It works best for libraries which install a FooConfig.cmake file, since in these cases nothing has to be detected, all the information is already there.

All this stuff is still very new, and has not yet seen wide real world testing.
So, if you use it and find issues, or have suggestions how to improve it, please let me know.

Alex

09 Nov 2011 9:15pm GMT

There was a lot of bug fixing regarding nepomuk and its indexing. However you might still get a high CPU-usage. Reporting this is a bit useless unless you can at least give some info about what's happening.

So what you can do is query virtuoso's status. On openSUSE it works like this: first find the .ini file currently in usage to get the port virtuoso is using, connect to virtuoso and finally query virtuoso for its status and running statements. The latter are unfortunately truncated so I would appreciate some hint on how to get around that.

ps aux | grep virtuoso

finds /usr/bin/virtuoso-t +foreground +configfile /tmp/virtuoso_T18122.ini +wait

cat /tmp/virtuoso_T18122.ini | grep Port

finds ServerPort=1111

isql-vt -H localhost -P 1111 -U dba -P dba

which connects to virtuoso

status();

which shows you some info and which queries are keeping the process busy. isql-vt is part of the virtuoso-server package but it might be that only recent packages have it compiled and older packages lack the tools.

Further you can do the following:

• open kdebugdialog and search for nepomuk
• enable debug output for nepomuk
• on a konsole do: qdbus org.kde.NepomukServer /nepomukserver quit
• wait until nepomuk and virtuoso are gone
• restart it from the konsole: nepomukserver

If you have any other hints regarding this piece of software feel free to mention them and I will add them to the post.

09 Nov 2011 3:10pm GMT

Some weeks ago Lydia blogged about a German bank giving away 1000 euros for each 1000 associations who can get the most votes. Well, until four days ago we were at postion 320, now we are at 735 and falling. Please, read Lydia's post about how to vote and help KDE e.V., it is just a few clicks.

It surprises me KDE e.V. have had only 3652 votes so far. If each people voted three times, which is allowed by the rules, it would give ~1218 people. Is there only 1218 people using KDE in the world?

Just the poll page is in German, but the poll is not limited to German citizens. Anybody can vote and according to the rules you can vote three times with your e-mail.

09 Nov 2011 12:22pm GMT

After some period of silence, this blog returns in bringing you the latest and greatest news from the Intel Linux Graphics world.

So if you were sad, depressed and crying in despair without having a chance of reading about what was going on with the Intel Graphics for the past days, rejoice!

Those past few days were quite busy on all the projects, and with thousands of emails to keep track of it is hard to select the most relevant news - all of them are! But I'll try to summarize the most interesting stuff that happened for the past few days.

Starting with kernel, as you all already know, we are living in the post-3.1 era now, with the release of Linux 3.2-rc1. It brings lots and lots of fixes and improvements all around, and much more are yet to come.

On Intel Graphics side, the following items caught my attention for the past days:

• Keith Packard sent yet more eDP-related patches, allowing eDP displays connected to the PCH to, well, work . The main issue came from the fact that the driver incorrectly was treating a PCH-connected eDP panel like a CPU-connected one, setting the wrong bits in the wrong places.
• Daniel Vetter has sent out a series of patches for simulating GPU hangs. As you all know, GPU hangs are an (unfortunate) part of a GPU driver life, and there are many factors which could cause them, ranging from incorrect GL instructions to out-of-bounds variables somewhere in the stream of commands and down to hardware failures. Prior to Daniel's patches, the only way to see a GPU hang was by having a hung GPU, so most of them were not easy to reproduce and investigate. Now, it is possible to stop the GPU at will, and see the effects. This is particularly interesting at least for me, because I was working on some tools for doing heuristics analysis of the root-causes of GPU hangs. Now, I can do this task with much more ease.
• Jesse Barnes sent out a new round of planes support, and support for SNB and IVB video sprites. The video sprites support different video formats natively and can do scaling as well, and their support was added to the DRM overlay code with this patchset.
• Ken and Daniel did a bit of cleanup of code specific to some pre-production SNB systems. Now that Sandy Bridge is out there, those bits are not necessary anymore, so they were wiped out.
• Daniel has sent a 13-patch series of pwrite/pread reworking. This patchset fixes some spurious -EFAULTS issues which could lead to kernel issues, improves the performance of pread/pwrite calls on LLC machines and cleanups unused code, replacing them with faster execution paths.
• Also on kernel, I've found an issue which can cause division-by-zero in kernel when accessing power-related registers from userspace, and sent a small patch fixing it. Apparently, the very same issue was already found back in July by Konstantin Belousov. It is a small fix for a potential kernel crashing issue, and let's hope it will be picked up in one of the next kernel pull requests.
• And Jesse Barnes sent out some documentation and cleanup patches for the drm subsystem.
• Still on kernel, but outside of our team's area, one big news for Linux Graphics will certainly make some people out there happy. Yes, I am speaking about Alan Cox moving of basic GMA500 driver out of staging. This driver support accelerated console and non-accelerated KMS on Poulsbo, Oaktrail, Cedarview and Medfield hardware. Note that medfield support in this initial patchset is left out on purpose, as it needs considerable rework to be ready to enter main kernel.

One particular issue worth highlighting is that a long-standing issue on GL-based applications (among which Unigine Tropics and Sanctuary are probably the most notable examples, among many others) was finally fixed, thanks to an amazing work by Eric Anholt, Kenneth Graunke and Keith Packard. This issue can be described as 'small moving ants on top of image edges' or 'flickering pixels'. So if you have had this issue, make sure to check out the patches!

Going to Mesa, out of hundreds of emails and commits, it is hard to choose the most interesting ones. Work on GL 3.0 support proceeds quickly, and new mesa stability release, 7.11.1 is almost out of the door. Our Q&A team did a full testing of this bugfixing release, and haven't found much issues. So prepare yourself, as in few weeks we'll have MESA 7.11.1 out there. Stay tuned for Ian's announcement in nearby future.

But as for mesa master branch, the following patches called my attention the most:

• Eric sent a patch series to add support for GL_EXT_texture_integer on i965 driver.
• More work towards EXT_transform_feedback was done by Dan McCabe and Paul Berry, and Marek from the community side.
• And we had lots of fixes for potential segmentation faults, safety checks, better hardware specification compatibility, piglit-based fixes and other issues from Yuanhan, Eric, Ken, Chad and Ian.

On Wayland side of the force, lots of patches went in those days. Among those, there was an interesting proposal for the screen locking protocol by Pekka Paalanen, and some bugfixing patches from Juan Zhao.

Going to the other components, we had a release of xorg-xserver 1.11.2 RC2, with several crashes and correctness fixes; and new stable pixman 0.24.0 which brings many performance improvements and usage of architecture-specific instructions to improve overall performance over a number of different operations (such as bilinear scaling for example).

And finally, for the intel-gpu-tools, I was working on a new intel_gpu_analyze application, which I was using to tracing and analyzing CPU and GPU performance data during workloads, and also checking on the corresponding power consumption. This is a very experimental code yet, and it lives at my freedesktop.org git for now. But still, I can already do some nice performance analysis like this one.

08 Nov 2011 10:08pm GMT

At LinuxCon Europe/ELCE I had the chance to moderate the kernel hackers panel with Linus Torvalds, Alan Cox, Paul McKenney and Thomas Gleixner on stage. I like to believe it went quite well, but check it out for yourself, as a video recording is now available online:

For me personally I think the most notable topic covered was Control Groups, and the clarification that they are something that is needed even though their implementation right now is in many ways less than perfect. But in the end there is no reasonable way around it, and much like SMP, technology that complicates things substantially but is ultimately unavoidable.

Other videos from ELCE are online now, too.

07 Nov 2011 3:53pm GMT