Bobulate

The Sun Ray thin client is a nifty piece of hardware, especially when it’s painted with a KDE Oxygen logo. It’s a thin client with USB forwarding and sound and session management and smartcard support (but it does not support the FSFE’s Fellowship smartcard). You can put the roughly the same thing together with Free Software, starting with LTSP, but I’ve been a fan of this largely-hardware solution for years. Unfortunately, it has been a poor platform for KDE4 — I’ve blogged about this in the past.

Until now.

There’s new Sun Ray server 5 coming up, and one of the features it has is Xrender (finally!). You need to enable it after installation and there are warnings about performance, but with a small deployment things should be just fine. I deployed it on my desk, with a single DTU with a single 1280×1024 monitor attached, running on an amd64 X2 with 6GB of RAM. Crunchy, but then again I was also building KDEtoys and KDEpim in parallel on it alongside a local GNOME session and a KDE 4.3.1 session on the Sun Ray.

Wait, 4.3.1? Wasn’t it just yesterday that I posted about the availability of KDE 4.3.0 on OpenSolaris? Yes, true. Bumping KDE up one minor revision did not take all that long, although I had to bump Akonadi to 1.2.0 in the process.

We use the gstreamer backend for Phonon on OpenSolaris, because gstreamer is installed anyway — or maybe it’s the Sun native audio backend? I don’t know and I don’t even know how to check. Suffice to say that whatever the default build picked out, sound works. Now, that might not be all that impressive at first thought, but that means that the thin client is receiving whatever audio output KDE is generating. I have not tried Amarok yet (that port needs a little more work still) on OpenSolaris, but that offers new realms of network-transparent desktop use.

The performance work done in Plasma over the past release cycle(s) has been impressive, and it just feels much snappier; Plasmoid handles show up like they should. This is a world of difference with my old laptop and KDE 4.1 which I was not-exactly-showing-off in London last year. Kudos to the Plasma team (and I’m truly looking forward to the results of Tokamak 3). The screenshot here is KDE 4.3.1 running on a Sun Ray, with whatever theme and decoration I ended up with while fiddling around (although Fluffy Bunny is listed in the “Get themes” widget, it doesn’t seem to work). There is a rendering issue with the text in systemsettings, where white-on-white isn’t all that useful, but on the whole it seems to work well enough graphically. A quick test using qgears gets 20fps on the Sun Ray, versus 30fps on the local display of the same machine (Radeon X1200).

There are some functional bugs, though. Many applications do not start up from the K menu, although they do start from the command line (and they do start up if I log in on a local display instead of the Sun Ray). Konqueror takes forever and a day to start or respond to keypresses — this is not production ready, but it is debug-ready, for anyone with a DTrace hammer. There’s a good chance, actually, that the issues are all related to threading problems deep within the C++ libraries. I know there are some newer patches available for it, and I think the library has been fully integrated into Solaris Nevada as of this week (presumably with those patches), which will make our own packaging of those C++ libraries superfluous. Hunting these issues down and integrating some feature work for OpenSolaris will be the KDE4-OpenSolaris team’s focus for the coming release cycle. It’s coming closer.

This week I’ve spent a few late nights working on the OpenSolaris packaging of KDE 4.3.0 — since 4.3.1 is out, it’s just about time, don’t you think? FreeBSD has the good stuff already.

New development repositories: we have been using Mercurial for a while for maintaining the specfile repositories that drive the packaging of KDE4 on OpenSolaris, which — like any DVCS — makes branching and merging and opening up development a breeze. That also means some unrestricted growth in the number of repositories and sometimes feature creep on individual repo’s, so we’ve stopped for a moment to consider what’s what. There are now five repositories, of which one is legacy, two deprecated for being too confused (but kept for a bit just in case someone is using them), and two repositories that are in current use.

The KDE 4.3.0 repository has specfiles for KDE 4.3.0 (this should not be a surprise) and the dependencies that we have built. Those include Qt 4.5.2, BOOST, Apache stdcxx and lots more. This repository is considered stable and should not change any more. The 4.3.1 repo is not stable, and is under construction.

Things that will go into the 4.3.1 repo are an update of poppler to 0.10.7 (current version) and of course updating KDE to 4.3.1; this also means adding the new KDEpim-runtime module to the set of packages. We don’t package it for 4.3.0 and that doesn’t seem to hurt KMail. We hope to improve collaboration with Belenix by stealing all of their updates to the dependency tree, while we’re at it.

[[ And there was a Qt security announcement on September 1st — that one is in both repositories and applies to the packages that are available as well. ]]

New IPS package depot: in the past three months or so we have accumulated a crufty set of packages on various IPS servers; these have been knocked down and a fresh set of IPS packages is available from the publisher http://pkg.bionicmutton.org:10000/ (also supports port 8080, by popular request).

These packages are the ones built from the 4.3.0 repository using Sun Studio Express on OpenSolaris 2009.6 and are supposed to “just work.” As usual, people with the initials MS will find out that they don’t — honestly Michael, what do you do to those poor install images.

Tools usage: the KDE4 packages are still not built using Source Juicer. This is easy enough to explain: because it’s a family of 122 packages which share settings and common files, it’s difficult to merge into Source Juicer effectively. We have tried, and SJ does drive us to do some simplification and a reduction in the number of files used for the build, but we are not there yet.

Instead, packages are built on a stock amd64 (an X2 4850e w/ 6GB RAM under my desk) OpenSolaris 2009.6 install, then tarballed using the tools from the IPS best practices wiki, archivepkgs.py in particular. These are copied to the actual server which then needs to be restarted every time we import new packages — wanting to avoid the restart is what led to our wild growth of IPS servers.

One tool I’m missing (lazyweb wishlist warning) is one that cleans up the on-disk pkg depot; I’d like to be able to strip out all but the most recent version of every package from a depot. I have an inkling of how this would work (determine what’s “latest”, flag all the files it uses; later walk the filesystem tree and remove all non-flagged files), but I don’t feel like going the destructive route myself right now.

There’s a problem in pkg(1) with large file uploads, so publishing the packages directly to the server is infeasible; that’s something that we need to fix with a nice workflow and one appointed person to kick the depot server when needed. This will happen once we have KDE 4.3.1 packages built and somewhat tested.

Stability: There are some known crashes; Dolphin has some threading issues and KMail crashes on exit, for instance, but the overall impression I get from running KDE 4.3.0 as my desktop on OSOL is pretty good. Even with an underpowered X1200 onboard video. With a proper video card (e.g. the GF9600 512MB on my laptop) things are much nicer.

Feedback: Please use the kde-discuss forum/mailing list or file bugs in the regular KDE bug tracking system and set the Operating System to Solaris in your report. We appreciate bug reports (if you can, check that they are Solaris-specific by comparing with, say, FreeBSD behaviour) and appreciate patches even more. Remember, the specfiles are open and everything can be built from source by following the instructions on the KDE TechBase page on OpenSolaris (and that’s a wiki, so if the instructions are not clear, you can help improve them!).

The KDE4 IPS packages that I’ve been working on have been rolled out at bionicmutton so if you are a brave soul, you could install them by adding that as a package publisher (pkg.bionicmutton.org points to the same). Something like:
pfexec pkg set-authority -O http://solaris.bionicmutton.org:10000/ kdeips
pfexec pkg install KDEgdm-integration
This will get you an /opt/foss and a /opt/kde-4.2 (the latter actually contains KDE 4.3-beta2). And you should be able to run bits and pieces from there, or even log in to KDE from the regular OpenSolaris login manager.

Getting the packages out there did have some issues. These are entirely networking related: I build them at home on one of my OSOL machines, then push them out into the IPS package server, which is running on a FreeBSD machine at the university. Since my DSL upload speed is low (about 50% faster than Jon) uploads of large files time out regularly. Pushing Qt or Boost is pretty much a no-go, as it will fall over after, say, 90MB or more. I suppose I could get on my bicycle and carry my laptop to a place where I could use the direct wired network to the server, but that would mean going outside.

Instead, I found — relatively well-hidden, which I find typical of Sun documentation — a wiki page that provides a nice-to-read HOWTO package which in turn points to a download page where you can get tools. The universal toolkit image is sufficient, and it contains pkg-toolkit/pkg/bin/archivepkgs.py, which will extract a package from a local repository and dump it to a tarball. In essence it defines an on-disk format for moving packages around, and you can move the tarball to another IPS repo, unpack it, and it can be served from there.

This means that I can build, publish to a repository on localhost (no network timeouts!), dump to a tarball, scp to the actual server (scp is a lot more robust in that sense than python scripts doing web-services over slow links), unpack, and there it is. It’s still a 40 minute upload of Qt, but knowing that it will succeed is important.

Thanks to Kohsuke Kawaguchi for writing the original tarpkgs.py script and documenting it on his blog.

Right. So here’s a foolish plan: I want to publish some OpenSolaris packages to the world. They are not ready for SourceJuicer (submissions there are slightly complicated, and the forest of KDE packages I’m producing is tedious and time-consuming to import there). Last time I tried that publishing thing, I did it on a package server on an OSOL machine at the end of my own DSL line, which pretty much immediately spanked my bandwidth completely. So I want to do it on a machine with better connections.

It may surprise some that all the real servers I have run FreeBSD. I use OSOL as my main desktop operating system. There must be historical reasons for this. In any case, that means building the OpenSolaris IPS package server pkg.depotd on FreeBSD. It turns out to be relatively straightforward, as most of the actual server is written in Python (yay!) and there’s only a few bits of C in there. So my approach was:

• Fetch the sources for pkg as documented on the pkg project page (thank you Alan Coopersmith),
• Install necessary development tools on my FreeBSD machine:
  portinstall py-openssl py-cherrypy intltool gnome-doc
portinstall py25-mako py25-simplejson
• Run gmake. At this point I realised that the makefile has some bugs leading to No rule to make target `help/C/package-manager.xml' — there is some semi-complicated implicit target stuff going on in there. I built the needed files by hand with
  cd gui
msgfmt -o help/C/C.mo help/C/C.po
xml2po -t help/C/C.mo -o help/C/package-manager.xml help/C/package-manager.xml.in
  and stripped out the locales that I didn’t need.
• Build extract_hostid by hand with
  cd util/misc
cc extract_hostid.c -o extract_hostid -L/usr/local/lib -R/usr/local/lib -lintl
  This is needed because gettext lives in /usr/local on FreeBSD and the Makefile (obviously) assumes the OSOL setup.
• Fix the top-level makefile to use python from /usr/local/bin and run gmake again.
• Patch setup.py to support FreeBSD — there looks to be support for sunos, linux, windows and darwin already (!?). This takes a patch at lines 102 and 194. Run gmake again.
• Run gmake install
• Finally,
  cd gate/proto/root_freebsd_amd64
PYTHONPATH=`pwd`/usr/lib/python2.4/vendor-packages:
sh usr/lib/pkg.depotd -d ~/pkg/ips -p 10000

And there you have it. FreeBSD machines serving up OpenSolaris packages. For the purposes of KDE4 on Solaris, this means that pkg.bionicmutton will soon be returning and you will be able to install IPS packages instead of having to compile everything from source yourself. YMMV, contains rabid weasels, etc.

[[ And, as an addendum, let me congratulate the pkg / IPS team on the enormous improvement in the packaging system between OSOL 2008.11 and 2009.6; it used to be “ugh” and has reached, for me at least, a level of “hey, that’s pretty neat.” ]]