Bobulate

Thanks to the efforts of the KDE4-OpenSolaris group — among whom I number myself, hnhn, Hajma, and Ben as well as various testers and supporters — the KDE SC 4.4 beta1 (version number 4.3.80) is available as OpenSolaris IPS packages. Note that beta2 has been released already so there is more updating to do. In the course of the beta cycles, the amount of patching that needs dealing with tends to drop — at least as long as there are no large code drops.

Speaking of large code drops, Dario tries to explain polkit — a good thing, but I’m still not sure if this thing actually exists for me or not.

Packages: there is an IPS server hosting packages for 4.3.80. This is still my FreeBSD port of the pkg.depotd, so I do hope it remains compatible with the pkg command — pkg is still under interesting development, and it does suffer from API changes every now and then. In theory, you can just add this IPS server (say, as kdedev-ips .. oh, hang on, that is one of the things that has changed: I don’t think you can free-form name your IPS servers anymore). This should be a straightforward upgrade of your existing KDE packages.

Korona 4.3.80: there is a new release of the OpenSolaris live CD Korona (by Pavel Heimlich). It is available from genunix.org. Genunix is where you can find all of the different OSOL distributions and specializations. You can also get Belenix there, which is the KDE4-with-gcc distribution of OSOL. Note that the most recent distro’s are based on an OSOL with ZFS data deduplication, which may help in reducing disk space requirements if you have virtual machines or multiple source trees checked out.

Next steps in KDE4 on OpenSolaris are bumping everything to beta2 and plenty of runtime testing. I’m pretty happy with the stability of the desktop and KDE PIM right now, even Akonadi works; I think my main annoyance is with hangups in name resolution. Konqueror, for instance, regularly hangs (up to 10 seconds) in a dbus call (yay pstack). I think that’s either name resolution or favicon handling.

Just a heads up that the KDE4-OpenSolaris group has now got KDE SC 4.4.0 beta 1 (version number 4.3.80) building the following components: base, pim, games and graphics. For me that’s what makes a minimal desktop. Getting Akonadi up and running was the biggest challenge, because it needed a fix down in Qt related to bit-twiddling, compiler flags related to aliasing, and mutexes. One line, but it took quite some time to hunt down and rebuild.

You can get the sources for the specfiles here, and build from source. It will take a while. If you’ve built Qt 4.6.0 previously from the repo, do remember to remove it first because of this mutex change (we still don’t have a good means of getting pkgtool to decide when to upgrade). Background information for this effort may be found on KDE Techbase.

I’m writing these lines from inside Konqueror 4.3.80 on OpenSolaris. I wrote previously that Konq is much better with WordPress nowadays, so that means I can blog from a browser I like under an OS that I like (or I could boot into FreeBSD and do the same thing thanks to the efforts of miwi and the gang). It was Sinterklaas this weekend, and the ole Sint brought the kids too many toys and brought me updates to the spec files for KDE software on OSOL.

Ben Taylor has picked up the prodigious (or is it Sysiphean?) task of updating the dependency tree for Solaris 10 again.

Elsewhere in the stack, strigi was updated to 0.7.0 (thanks Jos for having an official release available), shared-desktop-ontologies was added at version 0.2 (a simple build), soprano was massaged once, then virtuoso added for version 3.52.7, then soprano massaged again, then cursed, then discarded. Then all the staightforward stuff involved in updating libs, pimlibs, base-* — that is basically updating packing lists to reflect new or changed files and removing the patches that we had pushed upstream.

Virtuoso was .. a trial, in any case. I don’t even understand why the dependency is referred to as Virtuoso when it’s about libiodbc; at that point I end up wondering why we require one ODBC library and not another (e.g. UnixODBC has a Qt SQL driver and is KDE friendly). Maybe there’s more undocumented dependencies underneath there — but I can’t find anything about it. Albert seems to be compiling something completely different when he mentions Virtuoso.

Getting libiodbc to compile at all (the code isn’t all that bad) did show up an interesting pattern: there’s a wchar_t based API and a char based API available, and the interface header does this:

#ifndef SQL_NOUNICODEMAP
#ifdef UNICODE
#define SQLInstallODBC SQLInstallODBCW

This is fine (sortof .. it’s very C-ish) except when the same interface header is included from the library itself. Since the library contains definitions for both SQLInstallODBC(char) and SQLInstallODBCW(wchar_t), the compile breaks. You end up with two definitions for SQLInstallODBCW() instead of one of each. That meant going through and patching each file in the library to not be stupid that way by adding #define SQL_NOUNICODEMAP (I suppose I could have added it to CFLAGS instead). It’s apparent that noone has ever tested this stuff with UNICODE turned on (or perhaps more restrictively, on Solaris with UNICODE on).

Once the library is no longer stupid then it turns out Soprano itself isn’t all that secure about the difference between wchar_t and char. If UNICODE support is on, then consider a line like this one:

SQLExecDirect( hstmt, ( UCHAR* )utf8Request.data(), utf8Request.length() )

Explicitly munging a UTF-8 request into UCHAR (um .. I’m not sure that the non-unicode interface is supposed to understand UTF-8, really) is going to break if that function is #defined to version that expects a wchar_t parameter there. All in all enough to make me give up on this rapidly and think about other parts of the stack instead.

Since there’s still no KDE 4.4 compilation requirements posted on KDE Techbase as of this writing, (and I don’t feel qualified to add it myself based on my rather grumpy interpretation of what needs documenting) it’s hard to figure out what we should be looking for.

Anyway, in positive news all of KDEBase built for 4.3.80, so that’s why I have konq and konsole; PIM needs some additional new patches (remember: if Akonadi, then add_includes(${Boost_INCLUDE_DIRS})) but on the whole it looks like fairly smooth sailing. Expect updated specs for a build maybe next weekend when Ben and Hajma have finished thrashing the repo.

Just because the source code isn’t all delivered in a tidy bundle to my doorstep with a thank-you note attached, doesn’t mean that I won’t try stuff out. Thanks to Christoph and Holger for explaining some of the WebKit things to me. In the course of today the KDE4-OpenSolaris folks updated the soprano packages, phonon (from an SVN checkout), and futzed around sufficiently that you should be able to build the Essential KDE Desktop (I don’t know where that fits in the new branding: it’s konsole, konqueror, kmail, plus integration with GDM so you can actually log in). As always, the specfiles and patches can be found in the Mercurial repo.

We still have 21 open bugreports against KDE trunk with patches attached (these are the same as patches in that repo, only reported upstream). That’s still a bunch to “encourage” maintainers to pick up. As a general rule, we commit trivia directly (happy constructors, const mismatches), file bugs on larger items, and try to keep the amount of unmergeable cruft to a minimum. I wrote some such cruft today for Dolphin, where apparently ? (some const KFileItemList method) : KFileItemList() << item isn’t acceptable. Sun Studio is particularly picky about types in the ternary operator; come to think of it a const_cast might have done the trick as well: something to experiment with later, because that would be a mergeable-trivia kind of patch.

Sometimes it’s hard to decide what flavor a patch is: today I added find_package(Boost) include_directories(Boost_INCLUDE_DIRS) to a few subdirectories in KDE PIM; that’s because Boost doesn’t live in the default compiler search path, but Akonadi’s headers #include them — so anything using Akonadi must also have the Boost include dirs in its include search path, which wasn’t the case. It’s a minor change, but it doesn’t scream “trivial” to me, so it’s sitting in our repo as “unmergeable cruft” for now.

All that said, that part of KDE SC 4.3.77 that I’ve gotten up and running is really nice; I’ve said it before, but the stuff integrates really well, including the GNOME network manager and notifications done by the pkgbuild tools when I’m building new stuff and the device manager; there’s still plenty to work on as well, but on the whole it pebbles (a small way to rock). Now if only I could start konqueror or dolphin from the menu or krunner — must be some weird crash going on behind the scenes when there’s no tty output. [[ And as a final shot, I should add that Konqueror is almost infinitely better with WordPress now than it was in 4.3.0. ]]

Qt 4.6.0 is out. That’s the first of the trials and tribulations for today, because it suggests that packagers should update their work on Qt. Which is one of those things that can take up to half of a KDE SC minor release cycle to get right. And when you find code in WebKit that has actually shipped that looks like this (real version in FastMalloc.cpp line 1438):

void *foo() {
#if COMPILER(MSVC)
  // Without this, Visual Studio will complain that this method does not return a value.
  return 0;
#endif
}

.. then it’s not so much reading Qt code as it is looking at the DailyWTF. No duh, you need a return statement to return a value. WebKit has other bits of badness in it as well, like missing configure checks. Remember when it was all the rage to check if the platform had char *strnstr() and then working with HAVE_STRNSTR instead of going #ifdef Q_OS_WINDOWS? Anyway, our sixteen variegated patches against Qt 4.5.3 still mostly apply, so I’ll go and experiment with them.

But actually I was trying to start updating the packages for OpenSolaris in the run up to KDE SC 4.4.0, which is still two months away. Fortunately Hajma has wrestled with KDE SVN trunk compilation and the dashboard now, so we have some idea of what’s going on in trunk and can slay bugs (happy constructor pattern seems popular again) as they show up. However, building from SVN checkouts (including all of kdesupport) is a world apart from building packages where you’re looking for released tarballs, clear dependency versioning and good documentation. Unfortunately, the Techbase documentation for KDE SC 4.4 requirements isn’t started yet as of this writing. It strikes me that there’s nothing about the parts coming from kdesupport in the 4.3 requirements either — I must have missed a memo somewhere.

Soprano just released a new version, 2.3.70, meant to work with the 4.4 beta cycle, so we can get started on that. Phonon is a bit trickier, since there’s no version check for phonon done in KDEbase-runtime (in the released 4.3.77 tarball, at least) and it fails sometime during compile with a missing phonon/globalconfig.h. The strigi website hasn’t been updated in over a year, but I know Jos has been doing releases of it — I just need to find out where to get a stable tarball.

Maybe it’s just a month too early to want to build KDE SC 4.4 against anything resembling release packages or on top of a stable software stack. Certainly when the stack that you’re delivering (various support packages, then the KDE development platform, then applications and a desktop on top of that) is very deep it can be tricky to get everything in sync, but it does place a burden on packagers. There’s a lot of stuff to update when a new release comes out, and not much time to do it in (which, incidentally, makes the KDE-FreeBSD team’s 4.3.3 packages all the more impressive to me — now if only 8-R didn’t have that little root hole).

Let’s take a look at what happens when other Free Software operating systems run on my laptop, in terms of power consumption (and nothing else — I’m not going to explain in detail what’s running on each one, and rest assured that the available apps and toolsets in each of these installs is very different, reflecting what I use each OS for):

• OpenSolaris, display off, nvidia drivers: 31W
• OpenSolaris, display on, idle, nvidia drivers: 38W
• OpenSolaris, display on, disk + network activity, nvidia drivers: 52W
• Just the power brick: 1W

So it looks like Solaris is marginally (2W) better with power than Kubuntu at idle, and this laptop draws quite a lot in regular use. Maybe Solaris is not switching some hardware feature on, like Bluetooth, rather than being more efficient — but I haven’t noticed anything missing (Bluetooth is not something I’d miss). FreeBSD 8 fares no different — roughly same usage numbers as the other two OSsen.

Many commenters suggested trying the nouveau driver — so I did, whatever is available on Kubuntu 9.04 with no updates applied, and it makes no difference in power consumption, fails utterly at resume-from-suspend (nv got me no video, but I could ssh in, while now it just hangs), and is just as slow on logout as the nv driver. Perhaps there’s newer versions available — not something I’m going to experiment with this week.

At FSCONS I attended a talk by Johan Thelin — good to see someone whose blog I’ve followed for a long time, but whom I’d never met before — about QtCreator. The talk itself was a bit simple, more a “how to get the toolchain going” than a “deep secrets of Qt creator.” But because I was there I decided to update the OpenSolaris specfile for qtcreator to version 1.2.90 (the latest version for which a source download is available), and it took remarkably little effort to get it going.

Several of the patches from previous versions of the spec had been upstreamed — thanks Ossi — and only the bogus cruft patches remain like dragging in extra libraries and dealing with PTRACE.

The resulting qtcreator.bin works reasonably well — there’s an issue in detecting where Qt is installed, so I had to set that manually and the RPATH isn’t set to include lib/qtcreator/ — which means the method-autocompletion works and it does compile and run an application. I haven’t tried debugging or anything complicated, though.

This bit of Qt coding was in between my talk — on license selection and governance models in Free Software projects, where I was glad to have received useful feedback from the GNU hackers meeting yesterday — and a talk by Christina Haralanova. Just enough time to compile once, and it works.

For those tracking KDE4 OpenSolaris, I’ve updated the pkg repository for KDE4 packages. There’s a couple of things to note:

• This build requires nv126 or later. Build 124 introduced some new incompatibilities in OpenSolaris itself, which trips up kdebase-apps builds — this has been hacked around, but we need to find a good fix that we can upstream.
• There will be no KDE 4.3.3 from the KDE4-OpenSolaris folks. We’re going to focus on 4.4.0, since there’s plenty to chase after there. Hajma has been doing a great job in tracking trunk these days, and I’ve been tagging along with 4.3.2.
• Upstreaming patches is tough. We intend to actually set up nightly builds that report to the KDE dashboard, as Alexander Neudorf has once again extended the hand of friendship to us non-Linux folks for the dashboard.
• As always, the source specfiles can be found on solaris.bionicmutton.org/hg/ and they all point to the original upstream sources.

Some time in march I will need to go back to Kano, Nigeria, hopefully for the second Nigerian Free and Open Source software conference. I’m looking forward to seeing Mustapha (now a proud father) and Ibrahim and Immanu and the rest of the guys again. The news out of Nigeria hasn’t been very good recently, and the Dutch government is advising against travel to the south of the country at all; this doesn’t apply to the north, which I found a wonderful place to visit and talk about Free Software last year. Licensing is always an issue, as are business models.

I was therefore really happy to stumble across the FOSS for Africa wiki with a whole series of educational modules around business models and examples of what works in Africa right now.

Looking at Holger Freyther’s entry on the size of QList::Data object, my first reaction was “no, you can’t make that smaller, not on every architecture.” Holger’s point is that there is a hole of 31 bits in the structure (Thiago points out that that is intentional) where you could re-order elements to remove the hole — the last 24 bits of the hole could be used for something else. Of course, that only works for machine architectures that allow byte-aligned addressing. In any case, this shows that machine-based suggestions on code quality should be taken with a grain of salt.

But Holger’s other comments on memory profiling got me thinking about the tools available on OpenSolaris as well. It comes with something called pmap(1) which spits out a table like this (an edited selection from Firefox):

 Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
07FE4000     400     400     400       - rw---    [ stack ]
08050000       8       8       -       - r-x--  firefox-bin
08061000       8       8       -       - rwx--  firefox-bin
E9600000    4096    4096       -       - rw---    [ anon ]
E9C00000    3072    3072       -       - rw---    [ anon ]
EA000000    1024    1024    1024       - rw---    [ anon ]
F2673000     560     248       -       - r----  DejaVuSans-Bold.ttf
F2EA0000     156      36       -       - r-x--  libaudiofile.so.0.0.2
F2ED6000      12      12      12       - rwx--  libaudiofile.so.0.0.2
F2EE0000      40      36       -       - r-x--  libltdl.so.3.1.4
F2EF9000       4       4       4       - rwx--  libltdl.so.3.1.4
total Kb  249204  155460   54148       -

This seems to be pretty fine-grained, and by totalling the RSS (Lubos help me here) you can find the total real memory usage of the application. Something that might be scripted and collected over time? Heck, there’s probably some DTrace probes to do just that already — it’s just missing a graphing application.

On a vaguely related topic, I should note that KSysGuard on OpenSolaris has received a great deal of attention — not upstreamed yet — both from Belenix and from the KDE4-Solaris project inside Sun, so that it once again produces useful displays on OSOL.