Bobulate

It took some wrestling, but the EBN machine is back up, with some of the VMs up and running FreeBSD 8-STABLE. I’m still repairing the EBN itself, since that has many many more packages installed that need recompiling than the other VMs. There’s also Java to deal with — it may have become superfluous on the machine, but I need to sort that out or build by hand again (due to license restrictions, you still need to fetch Java distribution sources manually).

In any case, with the server up and running again the KDE4-OpenSolaris folks could push their updates to our Mercurial Repository with specfiles (and patches) for the KDE SC 4.4 series, including dependencies. As always, this is intended to build all of the KDE packages on a recent OpenSolaris (I use 130, others use 134, and I would recommend against anything pre-124). Mark and Ben also work on supporting Solaris 10 (the stable, enterprise OS release) so you can probably build the whole thing on there as well. Both SPARC and amd64 are supported, and we do try to support 64-bit builds for all of the dependencies. KDE itself builds only in 32 bit mode, though (simply because I don’t see much point in doubling build times and having an extra copy around).

Let’s take a look at what has happened recently in the repository:

• Updated to KDE SC 4.4.3: after it was released on May 5th, 2010, it took only a day to bump the repo to 4.4.3 as well. We now tag things in the Mercurial repo a little more consistently, so you can pick releases as needed — I do think this is a little more convenient than cloning a separate repo for each minor release as we did for the 4.3 series.
• GPG Updated: updated gpg to 1.3.0, which is the latest version. This also meant updating libassuan to 2.0.0 and some other minor tweaks, but it gives us something modern again.
• icu4c updated: The Unicode consortium’s library for Unicode manipulation — needed by Boost — has been bumped to the latest version. There’s a “competing” version from Sun itself, SUNWicu4c, but that’s built against the old Cstd library so we can’t use it.
• Various administrivia: updated libattica to 0.1.3 and changed the QImageBlitz library version to 0.0.4; this was 4.2.0, matching the KDE release that it originally came with, but the library itself thinks that it is 0.0.4. Looks a bit unmaintained otherwise, and I had to go (thankfully!) to the Debian source mirrors to get a sensible source tarball.
• GNU Telephony: The dependencies for the GNU Telephony project have been imported as well. I don’t know if this is needed on OpenSolaris — it hasn’t been built for any of my KDE needs yet.

So there’s plenty of updates going on, and we’re happy to have a single source where you can type “make” and get a working KDE4 desktop on an OpenSolaris machine. Packages might be available by now, too. I’m still working on transferring the package server to an OpenSolaris VirtualBox on FreeBSD.

By tracking things more closely each minor release (of anything) becomes easier to test and integrate — I suppose I should write something about using ZFS to do that — and we can stick closer to the actual release dates. I don’t think we’ll ever be able to keep up with the Linux distro’s or FreeBSD, but we’re close.

[[ OSOL-only entry here, with interest for KDE only from the perspective of “it’s a package server that will be serving up the KDE packages for OSOL”. ]] There’s some bits and pieces involved in setting up an OSOL appliance to be actually useful for serving up packages, so I thought I would document them here.

Adding a user and configuring ssh access: The adduser command in OSOL is a little arcane compared to the interactive versions available on FreeBSD and GNU/Linux, so may as well: useradd -s `which bash` -m username and remember to set up ssh access for that user. This user is also going to have the pkg repository available.

Making sure that the user can ssh login: On my OSOL appliance, /dev/ptmx is missing (which is a known problem, see this forum thread with the always-helpful Alan Coopersmith on it), so I followed the instructions there.

Setting up static IP: The appliance is configured to use DHCP from the local network. For my later use I need a static IP, so I picked 10.0.0.26 and solaris.bionicmutton.org as hostnames (there’s already a host with that name, elsewhere, which is the current FreeBSD-based package server for KDE’s OSOL packages). I added the hostname and IP in /etc/hosts, checked that /etc/nsswitch.conf was using files and dns, edited /etc/nwam/llp to set the interface to static IP (as described here), and added an /etc/hostname.e1000g0 matching one of the hostnames I had just set up. One reboot later (presumably svcadm restart would do, but it didn’t work for me in my haste), things were working and I could ssh in. While I was at it, I also modified /etc/nodename to use the new name of the machine (see, for instance, this thread on changing the hostname).

Setting up pkg.depotd space: As root, I ran pkg.depotd once to set up the repository space before setting it up to auto-start. /usr/lib/pkg.depotd -d /export/home/pkg -p 10000 --set-property publisher.prefix=solaris.bionicmutton.org starts the server. I checked that it was working at all by visiting 10.0.0.26:10000 from the host machine and then I stopped the server again.

Further configuration of pkg.depotd: Next, I use the Service Management Facility (SMF) to configure the package server further. svccfg -s application/pkg/server starts a command-line for configuration purposes. I set the port and root directory for the pkg.depot server here:
setprop pkg/port = 10000
setprop pkg/inst_root = /export/home/pkg
After that bit of configuration was done, I used svcadm refresh svc:/application/pkg/server ; svcadm restart svc:/application/pkg/server to start up the package server by hand, and checked once again that it was actually running. Then I rebooted, and checked that it was running still, just to be sure it will come back under a reboot of the host.

Minor remaining configuration issues: I switched off atime on the filesystem hosting the package respository, so that those are not updated on every package view (zfs set atime=off rpool/export, which reminds me I could have gone for a finer-grained setup by creating a filesystem for the packages first). I also set the read-only flag so that people can’t just publish into this repository — the KDE4-OpenSolaris people use tarballs + ssh for that. I also modified some settings in /export/home/pkg/cfg_cache to reflect the purpose of the repository better.

The OSOL appliance released by Jignesh Shah is 2009.6 — nearly a year old now. Since then the packaging tools for OpenSolaris have made great strides. IPS remains .. interesting. I’m not entirely convinced that it was worth the effort of not using some of the existing package formats. However, by now I understand the API it brings in as well as the various uses of the tool, (and it crashes no more) so I’m starting to like it.

Anyway, the point of this particular appliance is to host KDE4 packages for OpenSolaris in an IPS repo. To do that means running the IPS pkg.depotd. I have a port of an older version of pkg(5) to FreeBSD, but it all seems a little klunky and hard to maintain. So using an OpenSolaris VM seems the right approach.

Except that the OSOL VM from Jignesh is kind of old, isn’t it. So I ended up looking into upgrading the OSOL VM after installation, and ended up with the following steps:

1. Set the publisher to the dev branch: pkg set-publisher -O http://pkg.opensolaris.org/dev/ opensolaris.org . This means that future packages will come from the dev branch, not the released branch.
2. pkg refresh --full to fetch all the new package references.
3. pkg image-update -f (I have to force it, because something’s wrong with cherrypy and the image will not update otherwise) and wait a while while 138MB of package updates are downloaded.
4. Reboot into the new boot environment (VOSApp-1). This will fall into a root shell for fixit-purposes, because the filesystems are messed up.
5. Fix mountpoints for those messed-up filesystems and reboot again; I recall I needed to re-set a number of zfs mountpoints to get the correct organization for regular system operation. Bear in mind this isn’t needed with normal OSOL upgrades, just for this slightly peculiar application setup. I think I had to set the new rpool/ROOT/VOSAPP-1/opt/ (and var and root) to mount in the right places and move the older rpool/ROOT/VOSApp/ filesystems to somewhere in /mnt.
6. Reboot, possibly fix up remaining mountpoints, then zfs destroy the old ones.

There’s a useful HOWTO entry on updating OSOL to a specific build which I might otherwise have used, but the appliance image does not have an entire package installed which could be used to drive the upgrade. Hence this slightly klunkier approach.

Since Jignesh has left Sun, I’m not sure if there will be any future growth in these appliance builds. They were popular enough in Nigeria, where I distributed verbatim copies of the original (probably still in violation of the license terms though) to people interested in running OSOL at all.

Oracle has confirmed that they’re doing something with OpenSolaris, it’s not being sunk to create a diving reef or anything, so the folks who have been putting effort into KDE4 on OpenSolaris can carry on a bit. There’s KDE 4.4.1 packages available (that link, which includes a specific port number, may be valid only through to mid 2010). A number of crashes has been fixed. Debugging information has been improved. There’s still a little weirdness in q_atomic_decrement every now and then, but on the whole it’s good to use.

It’s also usable on Sun Ray 5 (which has version 4.2, I think). My OSOL machine runs SRSS (following the installation instructions on the Sun Ray wiki for OSOL 2009.6) and I can run a KDE4 session both on the local display and on the Sun Ray DTU, quite acceptably.

I took a look at the “Come out as part of KDE” guidelines, those for distro’s in particular, and find that for the distributions we should (?) be writing something like this:

OpenSolaris 2010.03 comes with either GNOME 2.2.26 or Plasma Desktop 4.4 and many applications like the GIMP 2.10, Amarok 2.3.3, F-Spot, Digikam and many more.

I’m not sure why we’re putting words about the available GNOME components into our distro’s mouths, and the sudden (maybe I haven’t been paying attention) appearance of “Plasma Desktop” is a little strange. My initial response is “that’s nice, Thomas, but I wanted a Bud Lite^W^W^WKDE.” Certainly because GDM’s session manager is going to say “KDE” as session type, not “Plasma Desktop” for the foreseeable future.

The communities working on KDE4 on OpenSolaris are fairly small — there’s one group using Sun Studio and aiming for integration with the usual packaging on OpenSolaris, and there’s the Belenix distribution which uses gcc. The two complement each other. I’m involved with the former group, working on packaging and keeping KDE code tidy (as much as possible — though I’ll admit to introducing hideous #ifdefs). What we don’t know much about is who would want KDE4 on OpenSolaris (or Solaris 10, for that matter — Ben has been really active in bringing the whole thing to S10, which takes even more porting). To that end, Pavel has set up a poll on SurveyBob, writing: If you are already using KDE4 on OpenSolaris or Solaris, or you plan to, please let us know more about your expectations and take a brief survey. There’s no prizes attached to this survey, but we’d be happy to know more about the people for whom we’re doing all this packaging (you don’t necessarily have to show us any love, either).

Although KDE SC 4.4.0 isn’t officially released yet (or there’s no announcement on the dot yet), I’ve bumped the specfiles in the KDE4-OpenSolaris repository to 4.4.0 and kicked off a rebuild. That means that we should have new packages at about the same time that the major GNU/Linux distributions have them. The release candidate was pretty stable on OpenSolaris as long as you didn’t (re)build any parts of it while logged in, but we’ll probably be hearing enough bug reports in the next few months for OSOL. You can report problems to KDE’s bugzilla with the OS set to Solaris, or use the OpenSolaris bug tracker and choose one of the KDE components there.

Of course, the whole future of OSOL is a little .. fuzzy. The Oracle / Sun takeover has caused a fair amount of shake-up in many of Sun’s projects, but OSOL has not, as far as I know, been either committed-to or deprecated. We’ll see.

In the wake (er, .. the bow shock wave, since I’m getting ahead of myself) of the release, Jos asks for something offensive and the KDE website has been re-vamped. Especially the latter is impressive, for breathing new life into the somewhat moribund site. I believe the call for help still applies as the rest of the content is polished and brought up-to-date.

The second beta release of the main collection of software produced by the KDE community, aka KDE SC 4.4 beta 2, was released last week. There are OpenSolaris packages available thanks to the compilation efforts of the Pavels; since we’re still not very good at publishing the packages in a standard repository, nor in managing smooth upgrades, you can find the packages here.

This set of packages includes Qt 4.6.1, which is a marked step up in stability; also Virtuoso support has been added if you build from source, but not in the packages. We’re still wrestling a bit with the C++ stack, as some things still use the older Cstd STL and “our” stack uses the newer Apache stdcxx4. In OpenSolaris, things are supposed to move to the Apache stack, but that takes some time and needs careful coordination, for instance because enchant plugins need to be moved over, but that implies that all of GNOME (the parts of it that use a C++ API anyway) also needs to be recompiled.

The first time I started up 4.4 beta 2 was a real eye-opener, from the new artwork to various bits of polish applied to the desktop (well, on OpenSolaris that includes things like “you can now start an application from the K-Menu again”, which was broken in all the recent versions). Since then I’ve started jotting down comments about aspects of the default installation that bother me — mostly little things, and it’s another indication that the packaging is coming together pretty well on OSOL that little bits that lack in polish are able to attract attention again.

Once I’ve got a more substantial list I suppose I’ll blog about them. That’s a little tricky to do, since many of them might be a “designed that way, but incompatible with [ade]” thing that I just don’t know about and I don’t want to come across as whiny or demanding (I’ll save that for licensing issues). And I don’t think that things like “Akonadi startup screen gets in the way” are worth filing a bug over, not until there’s some consensus that it is a bug. See also the “logout sound” thread on KDE core-devel right now, that feels similar.

Anyway, it’s good to see that the next release will be pretty solid, also on OSOL; if you want to take a peek, bother me at FOSDEM this weekend (just before the 4.4.0 release!).

Another week gone by an no end to my cold in sight. This is getting to be really annoying, as it both fragments my work day and makes me unreliable in responding to just about anything — so various jobs (both real and hobby) have been falling behind.

The hobby bits are easier to write about, so here goes: I wanted to produce a screenie of Krita on OpenSolaris, but it turns out that it hand not been built in my first packaging (neither had KSpread, for that matter). Turns out they require Eigen. Eigen is a C++ template-based linear algebra library. So I installed the Eigen package (also produced by the KDE4-OpenSolaris project: make no mistake, this is a small group with a big software stack to take care of). That’s where the fun starts — I don’t think anything else makes serious use of Eigen. Maybe some stuff in KDEgames or KDeedu, but obviously because I hadn’t had Eigen installed previously, they (if they are there) have been silently ignored.

So, Eigen. Since the KDE4-OpenSolaris project aims to use the Sun Studio compiler (so that the resulting KDE4 could be included in the base system and so that i t can re-use any other C++ infrastructure in the system), we need to make sure that Eigen works with the compiler. In the case of a giant and complicated template library, that can be a big effort. And I’m immediately stymied by the Matrix class, where the compiler complains about multiple definitions of every type in the Eigen generic interface (Base, Scalar, and another half-dozen). I’ve not even managed to strip the examples down to an example that demonstrates the problem, which makes debugging — or asking the compiler gurus for help — difficult.

Suffice to say that Krita screenshots are not to be expected anytime soon.

On the upside, the release of KDE SC 4.4 RC2 means that the other bits are nearing stability, which means we can re-focus on stability and performance on OpenSolaris for a bit. Since Pavel has set up various continuous builds on our crunchy build box (courtesy of Sun Netherlands) so tracking in future should work a little better — I hope we have fewer separate patches to maintain in future as more stuff gets upstreamed.

Inge Wallin recently blogged about the portability of KOffice — spurred on, no doubt, by the success of the port to the Nokia n900 and to Haiku. So he listed GNU/Linux, Mac OSX, Windows, FreeBSD (thanks, Inge, for checking), Haiku. That list is missing (Open)Solaris though, which as a UNIX flavor. ought to be a pretty simple target.

Of course, Solaris has been a primary target for OpenOffice for ages, so KOffice is a little late to the game on this particular platform. But I guess that’s my fault, since I’m one of the packagers for KDE4 on Solaris, and I hadn’t gotten around to it yet. So this weekend I spent a little under two hours hammering together a specfile (RPM-style) for koffice and getting the whole darn thing to build. Screenshot of KWord in action as proof. I tried KPresenter as well, but that crashed on changing the list style, so I didn’t think that was a good demonstration.

Of course, no port is without its patches, so here they are:

• constness — this patch matches the constness of parameters in definitions with those in declarations, so that int foo(const int) is defined with int foo(const int i) and not as int foo(int i). Now, since the last time constness-matching blew up, I’ve learned that this is really a bug in Sun’s compiler, because constness is not supposed to be mangled into the name. However, I’ll claim that code neatness demands that they match, anyway.
• double — some math functions like sqrt() can take both float or double, and Sun’s compiler doesn’t just pick one when you pass it integer parameters, so we need some explicit disambiguation. This is common all across the KDE codebase.
• NaN — the KOffice code uses val != NAN which uses the gcc-specific NAN #define; probably !isnan() is better.
• math — the header file math.h in Solaris uses the identifier “exception”, which becomes ambiguous in the context of the STL, so it needs to be included earlier, rather than later. This patch bungs in math.h as the first include in a number of C++ files — not necessarily something to merge upstream, because it’s mostly working around a bug elsewhere.
• stupid — yes, this is a stupid patch. I can’t convince Sun Studio that 8.5 * 1440 is an secretly an integer constant and that it shouldn’t complain about a bunch of initializers in the MS Word import filters. I’m not really sure what’s going on here, it’s something special about static const initializers of class variables, since doing the same in a non-class context works just fine.

So there you have it. Five tiny patches for a codebase of a little over 600000 lines of code. Nice. Tip of the hat (I have lots of hats, but all of them are baseball caps and I don’t have a decent Stetson) to Inge and the KOffice folks for a nice portable office suite. You may be troubled, but your code is good.

The -440 repository — that is KDE4-OpenSolaris-speak for “the Mercurial repository which holds the specfiles that are being developed for building KDE SC 4.4.0 once it is released” — has been bumped to beta2 now (version 4.3.85). We’re still in the process of rebuilding everything, as there are some new dependencies and the usual fixes and patches to be updated. I count 66 patches still applied; we have started pushing more of them upstream — especially the trivial ones like adding newlines to the last line of a file. Thanks to SadEagle for picking up some of the patches to KHTML again as well — they’re a little odd in that they apply to code that isn’t ever called but that needs to compile anyway.

xz: this is a new dependency, a general purpose compression library. Very straightforward build, no other dependencies — something that we should be trying to foist off on SourceJuicer as quickly as possible so as to make it available to more people.

libattica: Frederik Gladhorn (and whoever else worked on that), I salute you. No patches required, builds without even a single warning. If I’m going to be really picky, I’d point out that the files are licensed GNU LGPL 2.1 (the Lesser GPL) while the included COPYING file is the GNU LGPL 2 (the Library one). And there’s no tests for it, so while I can see it compiles, I have no idea if it works. Good job, anyway.

The status continues to be: it works. It’s kind of slow, and backtraces involving the Flash plugin are non-ASCII text, but for straightforward web browsing and KMail and konsole — my desktop — it’s in OK shape. As usual additional compile and runtime testers are encouraged to step forward.