Bobulate

Looking at Holger Freyther’s entry on the size of QList::Data object, my first reaction was “no, you can’t make that smaller, not on every architecture.” Holger’s point is that there is a hole of 31 bits in the structure (Thiago points out that that is intentional) where you could re-order elements to remove the hole — the last 24 bits of the hole could be used for something else. Of course, that only works for machine architectures that allow byte-aligned addressing. In any case, this shows that machine-based suggestions on code quality should be taken with a grain of salt.

But Holger’s other comments on memory profiling got me thinking about the tools available on OpenSolaris as well. It comes with something called pmap(1) which spits out a table like this (an edited selection from Firefox):

 Address  Kbytes     RSS    Anon  Locked Mode   Mapped File
07FE4000     400     400     400       - rw---    [ stack ]
08050000       8       8       -       - r-x--  firefox-bin
08061000       8       8       -       - rwx--  firefox-bin
E9600000    4096    4096       -       - rw---    [ anon ]
E9C00000    3072    3072       -       - rw---    [ anon ]
EA000000    1024    1024    1024       - rw---    [ anon ]
F2673000     560     248       -       - r----  DejaVuSans-Bold.ttf
F2EA0000     156      36       -       - r-x--  libaudiofile.so.0.0.2
F2ED6000      12      12      12       - rwx--  libaudiofile.so.0.0.2
F2EE0000      40      36       -       - r-x--  libltdl.so.3.1.4
F2EF9000       4       4       4       - rwx--  libltdl.so.3.1.4
total Kb  249204  155460   54148       -

This seems to be pretty fine-grained, and by totalling the RSS (Lubos help me here) you can find the total real memory usage of the application. Something that might be scripted and collected over time? Heck, there’s probably some DTrace probes to do just that already — it’s just missing a graphing application.

On a vaguely related topic, I should note that KSysGuard on OpenSolaris has received a great deal of attention — not upstreamed yet — both from Belenix and from the KDE4-Solaris project inside Sun, so that it once again produces useful displays on OSOL.

The Sun Ray thin client is a nifty piece of hardware, especially when it’s painted with a KDE Oxygen logo. It’s a thin client with USB forwarding and sound and session management and smartcard support (but it does not support the FSFE’s Fellowship smartcard). You can put the roughly the same thing together with Free Software, starting with LTSP, but I’ve been a fan of this largely-hardware solution for years. Unfortunately, it has been a poor platform for KDE4 — I’ve blogged about this in the past.

Until now.

There’s new Sun Ray server 5 coming up, and one of the features it has is Xrender (finally!). You need to enable it after installation and there are warnings about performance, but with a small deployment things should be just fine. I deployed it on my desk, with a single DTU with a single 1280×1024 monitor attached, running on an amd64 X2 with 6GB of RAM. Crunchy, but then again I was also building KDEtoys and KDEpim in parallel on it alongside a local GNOME session and a KDE 4.3.1 session on the Sun Ray.

Wait, 4.3.1? Wasn’t it just yesterday that I posted about the availability of KDE 4.3.0 on OpenSolaris? Yes, true. Bumping KDE up one minor revision did not take all that long, although I had to bump Akonadi to 1.2.0 in the process.

We use the gstreamer backend for Phonon on OpenSolaris, because gstreamer is installed anyway — or maybe it’s the Sun native audio backend? I don’t know and I don’t even know how to check. Suffice to say that whatever the default build picked out, sound works. Now, that might not be all that impressive at first thought, but that means that the thin client is receiving whatever audio output KDE is generating. I have not tried Amarok yet (that port needs a little more work still) on OpenSolaris, but that offers new realms of network-transparent desktop use.

The performance work done in Plasma over the past release cycle(s) has been impressive, and it just feels much snappier; Plasmoid handles show up like they should. This is a world of difference with my old laptop and KDE 4.1 which I was not-exactly-showing-off in London last year. Kudos to the Plasma team (and I’m truly looking forward to the results of Tokamak 3). The screenshot here is KDE 4.3.1 running on a Sun Ray, with whatever theme and decoration I ended up with while fiddling around (although Fluffy Bunny is listed in the “Get themes” widget, it doesn’t seem to work). There is a rendering issue with the text in systemsettings, where white-on-white isn’t all that useful, but on the whole it seems to work well enough graphically. A quick test using qgears gets 20fps on the Sun Ray, versus 30fps on the local display of the same machine (Radeon X1200).

There are some functional bugs, though. Many applications do not start up from the K menu, although they do start from the command line (and they do start up if I log in on a local display instead of the Sun Ray). Konqueror takes forever and a day to start or respond to keypresses — this is not production ready, but it is debug-ready, for anyone with a DTrace hammer. There’s a good chance, actually, that the issues are all related to threading problems deep within the C++ libraries. I know there are some newer patches available for it, and I think the library has been fully integrated into Solaris Nevada as of this week (presumably with those patches), which will make our own packaging of those C++ libraries superfluous. Hunting these issues down and integrating some feature work for OpenSolaris will be the KDE4-OpenSolaris team’s focus for the coming release cycle. It’s coming closer.

This week I’ve spent a few late nights working on the OpenSolaris packaging of KDE 4.3.0 — since 4.3.1 is out, it’s just about time, don’t you think? FreeBSD has the good stuff already.

New development repositories: we have been using Mercurial for a while for maintaining the specfile repositories that drive the packaging of KDE4 on OpenSolaris, which — like any DVCS — makes branching and merging and opening up development a breeze. That also means some unrestricted growth in the number of repositories and sometimes feature creep on individual repo’s, so we’ve stopped for a moment to consider what’s what. There are now five repositories, of which one is legacy, two deprecated for being too confused (but kept for a bit just in case someone is using them), and two repositories that are in current use.

The KDE 4.3.0 repository has specfiles for KDE 4.3.0 (this should not be a surprise) and the dependencies that we have built. Those include Qt 4.5.2, BOOST, Apache stdcxx and lots more. This repository is considered stable and should not change any more. The 4.3.1 repo is not stable, and is under construction.

Things that will go into the 4.3.1 repo are an update of poppler to 0.10.7 (current version) and of course updating KDE to 4.3.1; this also means adding the new KDEpim-runtime module to the set of packages. We don’t package it for 4.3.0 and that doesn’t seem to hurt KMail. We hope to improve collaboration with Belenix by stealing all of their updates to the dependency tree, while we’re at it.

[[ And there was a Qt security announcement on September 1st — that one is in both repositories and applies to the packages that are available as well. ]]

New IPS package depot: in the past three months or so we have accumulated a crufty set of packages on various IPS servers; these have been knocked down and a fresh set of IPS packages is available from the publisher http://pkg.bionicmutton.org:10000/ (also supports port 8080, by popular request).

These packages are the ones built from the 4.3.0 repository using Sun Studio Express on OpenSolaris 2009.6 and are supposed to “just work.” As usual, people with the initials MS will find out that they don’t — honestly Michael, what do you do to those poor install images.

Tools usage: the KDE4 packages are still not built using Source Juicer. This is easy enough to explain: because it’s a family of 122 packages which share settings and common files, it’s difficult to merge into Source Juicer effectively. We have tried, and SJ does drive us to do some simplification and a reduction in the number of files used for the build, but we are not there yet.

Instead, packages are built on a stock amd64 (an X2 4850e w/ 6GB RAM under my desk) OpenSolaris 2009.6 install, then tarballed using the tools from the IPS best practices wiki, archivepkgs.py in particular. These are copied to the actual server which then needs to be restarted every time we import new packages — wanting to avoid the restart is what led to our wild growth of IPS servers.

One tool I’m missing (lazyweb wishlist warning) is one that cleans up the on-disk pkg depot; I’d like to be able to strip out all but the most recent version of every package from a depot. I have an inkling of how this would work (determine what’s “latest”, flag all the files it uses; later walk the filesystem tree and remove all non-flagged files), but I don’t feel like going the destructive route myself right now.

There’s a problem in pkg(1) with large file uploads, so publishing the packages directly to the server is infeasible; that’s something that we need to fix with a nice workflow and one appointed person to kick the depot server when needed. This will happen once we have KDE 4.3.1 packages built and somewhat tested.

Stability: There are some known crashes; Dolphin has some threading issues and KMail crashes on exit, for instance, but the overall impression I get from running KDE 4.3.0 as my desktop on OSOL is pretty good. Even with an underpowered X1200 onboard video. With a proper video card (e.g. the GF9600 512MB on my laptop) things are much nicer.

Feedback: Please use the kde-discuss forum/mailing list or file bugs in the regular KDE bug tracking system and set the Operating System to Solaris in your report. We appreciate bug reports (if you can, check that they are Solaris-specific by comparing with, say, FreeBSD behaviour) and appreciate patches even more. Remember, the specfiles are open and everything can be built from source by following the instructions on the KDE TechBase page on OpenSolaris (and that’s a wiki, so if the instructions are not clear, you can help improve them!).

The KDE e.V. board has decided on a location for a new office in Berlin and we will be moving the e.V. office there fairly quickly; we decided this over generous portions of Frankfurter Green Sauce which is quite an interesting substance. Other topics like Akademy 2010 were finished off as well, and we will be doing official announcements on all these later this week. After the meeting I had a few spare hours and it was a gorgeous summer day, so I went over to the Liebieghaus (site doesn’t seem to like Konqueror) and it was just amazing. It was also kids’s day, so the gardens were not restful and quiet but filled with painting and coloring and dance and fröbelen. Inside, the house is a warren of galleries, chapels, turrets and whatnot, and stuffed to the gills with ancient to almost modern art and devotional artifacts. It was the right choice — thanks Claudia for pointing me in that direction. On the train I mucked about with Boost and its compatibility with the upcoming new Apache C++ STL for OpenSolaris.

This week is about conference planning, both attending and organizing, so I have a six month horizon — and it is cluttered. The next one up is OSiMWorld, where I don’t have a booth or anything, but I will be saying “Hi” with my Green and Blue hats to all comers.

I’m in Frankfurt this weekend for a meeting of the board of KDE e.V., where the main topic is the office; we will be moving to Berlin later this year, and selecting a location and planning the move is important for having a pleasant autumn. It’s a typical board meeting, with document sharing, silly drawings on the flip-over chart, a nice lunch organized by Claudia and a bazillion topics to go through besides the ones on the agenda.

Time for a collection of screenshots, as an illustration of Qt applications on OpenSolaris, both on a local display driven by a Radeon X1200 and on a Sun Ray thin client. Not from KDE applications (although we have KDE 4.3.0 packages for OpenSolaris now) but from qtconfig — possibly the first Qt app you will want to run in OpenSolaris to set up some of the fonts correctly. Before running this version of qtconfig, I removed ~/.config — the whole directory tree — so I would get the default settings. There are screenies of the same 300×100 section of the application on four setups: local display or Sun Ray thin client, and system fontconfig or one built from our own packages. I switched my set of package builds to use the system’s fontconfig a while back, but the specfile for fontconfig (useful if you care about Solaris10) is still there. Both are version 2.5.0; for freetype system is 2.3.7 and the specfiles build 2.3.6.

So you would expect very little difference between the two, possibly only a difference in the default fonts. You could use some kind of png diff tool to see the differences, but it’s not a whole heck of a lot.

A bigger change happens when you switch from a local display (even a lousy onboard X1200) to a Sun Ray thin client display. I know I’ve ragged on parts of Qt in the past and the way KDE deals with less-capable displays, so this time I’m not going anywhere near graphics views.

What strikes me immediately is that the style has changed, from one display to another; it now looks a good deal more Morif like. Eww! So what is Qt looking at here to change style like that? (That’s a lazyweb question).

And switching fontconfigs doesn’t do much again. I’m actually curious if either of the fontconfigs does anti-aliasing right now.

One thing you can’t see in the screenie is the window titlebar. Whatever the default GNOME window manager is — metacity? — produces really ugly title bars on a Sun Ray, as it looks like the AA falls over, producing hard-to-read letters with white-ish strips between them. Those are screenies for another day.

Flashing: I spent an intensely annoying morning flashing firmwares and BIOSes. I had purchased an MSI GX620 — a “gamer” laptop, but pretty nice from a developer point of view, too — and noticed that the keyboard would “stick” every now and then, repeating the last key typed ad inifinitum. Power button and trackpad would also cease responding. Either pull the battery and AC — hard power off — or plug in a USB keyboard and mouse and go for a regular shutdown, but then the machine would hang with power and fans still going after the OS says its final goodbyes. It is both reassuring and frightening that “MSI GX620 keyboard problem” turns up plenty of resources about exactly this. And a suggested solution is to flash the firmware and then the BIOS.

Easier said than done, of course, from a dual-boot nonGNU/Solaris and GNU/Linux machine. Heck, even with the Vista that was on the machine when it shipped I’m not sure you can flash. The MSI instructions all assume a DOS environment. I tried the boot disks you can create with Windows XP (which boots to “Windows Millennium”?) but no dice, abnormal program termination. I went looking in my box-of-crap in the attic and found my 3-disk set of MS-DOS 6.2 install floppies, but those no longer seem to work. Heck, it’s a wonder I even have a floppy drive to boot from — I collected a few USB floppy drives a few years back.

This kind of situation makes me long for my usual ASUS motherboards which support flashing from the BIOS itself, and can read CDs.

In any case, I needed to be able to boot an OS I don’t actually have in order to update the machine. Fortunately Daniel documented USB boot disks here, and that finally gave me enough to craft my own simple 10-step procedure: 1. boot ancient windows XP laptop strill used by the kids for Sesame Street games 2. install HP USB disk tool 3. boot modern Linux machine 3a. find USB floppy drive 4. download FreeDOS “ODIN” image 5. try to write the image to a floppy disk 6. download a smaller image, until you get one that fits before the first bad sector on the floppy (for me, fortunately, that was the 720k image) 7. move floppy to windows machine 8. format USB stick with HP tool, using system files from ODIN 9. boot defective laptop from USB stick 10. revel in C:\>

I suppose I could have gone via VirtualBox on Solaris or qemu, but I had neither installed already, and frustration seems to rapidly erode my capacity for lucid thought. In any case, I’ve now gone the day without the keyboard locking up, so the flash updates seem to have helped.

Photos: At Akademy I always try to have some plan on capturing the spirit of the event. In past years I’ve intended to do voice recordings and always forgotten, so this time I did something much more low key: just take pictures of some whiteboards left behind in the conference rooms. This little collage of strings found on boards does seem to capture the working week, though. What are we doing? Even though we’re die-hard computer technologists, we still end up smearing ink on light-coloured surfaces.

[[ You’d be surprised, incidentally, about how many of the photos in my phone are of whiteboards at the end of meetings. Best way to remember things. ]]

Finishing: Congratulations, Gökmen and Görkcem on Pardus 2009. It looks very nice. And I really appreciate a distro that comes with TeX.

KDE 4.3 is also looming terrifyingly close, which means that there’s a fistful of OpenSolaris patches we haven’t gotten around to pushing upstream yet. Some are really peculiar, and I appreciate it when people ask me about things like bool() casts — there was one bug report about operator ?: with a bool and a QBool, which seems very peculiar to me. Interesting how QBool doesn’t have an operator bool(), only an operator void *() — I think SS12 is being extra picky here.

Finally, one of the anonsvn mirrors was down again for a bit; this was due to r.997199 which has a badly-encoded log message. Since we have a wild mix of Subversion 1.4, 1.5 and 1.6 infrastructure — and also users with clients of differing versions as well — we do not get a “clean” history. In this case, subversion 1.6 is more picky about the encoding of log messages than older versions. Since I was running 1.6 on the EBN mirror, it refused to synchronize that particular commit and went down. I have since downgraded to 1.5.2 and the mirror has resumed operation, but it’s an illustration that our source history is “interesting” to say the least and the conversion to any other SCM will once again be rife with manual labour to get it all right.

Back from GCDS; patches for qtcreator handed off; packing for trip to London; putting kids to bed. I realize now that I didn’t tell a single dinosaur story at this conference; that I still owe some people an explanation; that I had far too little time to talk to Till about anything. I intend to spend this week on work-work (which is why I’m going to London) and restoring family ties that 14 days of Free Software travel have raveled.

The closing summary article on GCDS is up on the dot now, and it gives a pretty clear picture of what happened and what the most tangible results are. I’d like to thank the membership of KDE e.V. for giving me another mandate for three years as KDE’s “legal dude”. This will no longer be my primary Free Software affiliation because of my work for the Free Software Foundation Europe, but I expect it to receive as much attention as it needs. It will be evening (GMT+2) work, though. I look forward to working with Frank and Celeste and the “old” board members to maintain our legal and organizational stability.

We have the technology, we can fix it. The nice thing about showing people things that are broken and then sitting down to figure out what’s wrong is that the things get fixed. Holger helped out with understanding some of the rendering issues on the Sun Ray server, we tracked down some more suitable default themes (but there’s so many cool themes out there it’s hard to find a boring flat one), pushed graphicssystem RASTER into startkde, and ended up with a KDE4 that looks like this. Thin client heaven? I think not, but it’s a little better than when it started. Just a wee bit, though, even something totally flat like Heron (wish list: semantic tagging of plasma themes so I can search for “flat and boring”) totally borks the plasma panel.

Aaron was, probably rightly so, kind of annoyed with my cheap shot about ugliness on feature-limited X servers.

Still, it’s important to flag the problems, show them to people. The cheapness was more related to me complaining to the wrong people, but now at GCDS I could show them to the Sun Ray folks — and I’m very happy to have met Bob, Brian, Ken (?) and others from Sun — and collect information. It looks mostly like Qt has two problems: the fallback for non-porter-duff compositing is not very nice, and BGR colormaps are not supported. That’s the guesses from the Sun folk and they sound plausible enough — I’ll be trying them out when I get back home and can spend an evening re-compiling Qt all the time (instead of like here at GCDS where the evenings are for discussion and food).

.. and then we can fix it.

Smallest possible result: the punchline “she got the clap from a gnome” just made me giggle uncontrollably. Curse those webcomics! if there’s anything I’ve caught from GNOMEs this week, it was enthusiasm for cross desktop technologies, and I buried a Mexican. You just can’t get that kind of community building without a big conference, although I do miss a certain sense of “one moment that the entire project is together in a single room” because of the schedule.

Seeing how we (as the Free Software Desktop) are growing a new Pillar of the Free Desktop by moving Strigi analyzers and Tracker and Nepomuk Ontologies and Zeitgeist (really just a visualization of one suitable Nepomuk query) together — this is the Semantic Pillar — is really exciting. Finding this synergy takes a conference. Crossing the t’s and dotting the i’s is part of the work week around Akademy. That’s a good concrete result.

I got some patches for Qt Creator ready, but I’m so totally useless with git that I haven’t managed to push out any merge requests yet. Similarly, I see that the (Open|Nevada) Solaris packages are coming along well too, but haven’t taken the time to sync up these last few days. There are some other bugs I would like to look at in the coming weeks, but there is nothing critical right now.

Finally, I got to send off Akademy and GUADEC with a bang. No additional tools allowed (John Layt, that other photo is also genius). I spent some time going over the major results I could identify, and with Vincent Untz providing the “it was fun” part, we thanked the University, the organizers, and then we moved on to the best part: the Pope on patents.