From Out There
Thoughts from years of using Free Software in the wild
Microsoft is Guilty, but the Threat for the EU is in Software Patents
September 25th, 2007
Telepolis has an excellent article (German, unfortunately) that explains how, even now that Microsoft was proven guilty, their tactics have already changed. Unbundling products like the media player and offering competitors access to MS interfaces (for a fee!) will not stop them from stifling innovation and destroying competition by unfair means.
They have started applying for software patents in European countries even though these are not legal here. They have been lobbying to legalize software patents in the EU. The result? Let’s look at the competition in the media player sector. Windows Media Player is a very average product. VLC or Mplayer are European media players that can play a much wider variety of media files with a lot less fuss and no need to cumbersomely install extra codecs. They also come with far, far greater functionality across the board, offering everything from built-in live encoding and streaming to very advanced picture processing. On top of that, they are free, and in some cases even funded by EU government grants. It is in the best interest of the EU to develop such Free Software, in the EU, through EU citizens.
But, and here’s the problem, there are potentially hundreds of very, very trivial software patents registered in the USA that could theoretically apply to these products. These software patents are registered willfully by big corporations for the express purpose of being abused as weapons against competitors. They do not benefit the "small inventor", which is what patents were intended to accomplish. If these same patents were legal in the EU, and if the patent holders (not in every case Microsoft) were to raise charges against the projects, this could potentially destroy them. Even if a judge only orders a preliminary injunction, it could mean the death of the project as people look for different media players to fit their needs.
This is the threat, and the threat still remains. If you are an EU citizen, you should do everything in your power to say no to software patents — they stifle innovation, they massively strengthen billion-dollar companies at the expense of small and medium size enterprises and they cut into your own freedoms as a computer user, as a software developer, as an employee and as a person.
Our OpenVZ Virtualization Experience
September 25th, 2007
We are currently finalizing the server consolidation in our department. The product we chose for virtualization is OpenVZ, because it sports creepy Russians.
All in all, it was a bit of a roller coaster ride, but once we figured out that most of the problems came from our own incompetence, we quickly stopped pointing fingers and shaking fists and instead read some documentation. Then all was good. We went from 12 servers to 5, killing 7 physical servers and saving roughly 1500W of power consumption. The new virtualization servers we used were actually the old database server and the old main web server, both overpowered. A change in the mentality and the technical competence level required from our students in the last few years has made the extra power for these boxes unneccessary. Now we’re using them much more efficiently because each of them runs several virtual servers.
Technicalities
We wasted a lot of time learned a lot by going the opposite route when it comes to OpenVZ configuration. Most people are advised to start with a BIG configuration for each virtual private server (VPS), we started with a tiny one. This meant that memory parameters were at a bare minimum, normally mimicking the specs of the hardware machine we were virtualizing. In the same go, we grouped services differently so that we could reduce the number of servers, again making better use of the available hardware. For servers that are created from scratch (not based on an existing physical machine), we also started from a minimal config file and went up from there.
This approach not only made me grow at least six new white hairs in my beard, but it also taught me about the importance of KMEMSIZE. KMEMSIZE is your friend. KMEMSIZE loves you. KMEMSIZE is soft and fluffy. Trust KMEMSIZE.
The problem with KMEMSIZE was that while we did assign enough memory in the main UBC memory categories (vmguarpages, oomguarpages, privvmpages), we didn’t have enough KMEMSIZE for our NUMPROCS. Just picture this! The poor NUMPROCS! The net result was that the server couldn’t fork new processes once its amount of KMEMSIZE was eaten up. So after multiplying our expected NUMPROCS with the estimated unswappable memory consumption per process, things worked perfectly.
Now our OpenVZ environment is very, very stable and very, very efficient. We’re very, very happy. This is a very, very success story.
Automated GNU/Linux Installation CDs by the Truckload
June 23rd, 2007
A few days ago, HP unveiled the latest version of their LinuxCOE (Linux Common Operating Environment). In as few words as possible, LinuxCOE allows you to set up a web page where you can make customized GNU/Linux installation CDs that can install a Linux distribution without any human interaction. This is very handy if you often have a lot of PCs to set up with GNU/Linux, and it is especially useful if those PCs all have varying hardware characteristics (different hard drive sizes etc.). If you want to see LinuxCOE in action, check out Instalinux, which is powered by LinuxCOE, and generate your own CD immediately 🙂
What your finished CD will do is start an installer and retrieve, via some sort of network connection, the installation packages for the distribution of your choice. It will set up the system according to the settings you entered when you generated the CD image. That means that each CD can preconfigure a PC’s specific IP address (or DHCP of course), create user accounts with predefined passwords, partition the target hard drive according to predefined settings etc.
There are of course network setup systems and configuration management systems that have done similar things before and that fall into the same software category, usually even cutting out the CD and providing a way of network booting the machine you’re setting up. LinuxCOE is not directly competing in that arena. The killer feature here lies in the configurability of each and every CD image it generates. Profiles can be saved so the same settings can be consistently applied to several images, value-adds can be dynamically selected and added to a base CD and much more. Another killer feature is distribution support: LinuxCOE supports generating images for over 100 GNU/Linux distributions.
But let’s get to the meaty part: How to install LinuxCOE 4. At the time of writing, there were only two ways to install: straight from the CVS version or from CVS snapshots. There were no file releases at SourceForge yet. Debian packages are also not available, but perhaps someone is already working on them.
Prerequisites
You need a system with the following tools:
- Web server with CGI support (Apache 2.0 is a good choice)
- Perl interpreter
- mkisofs
- The sudo system
All of these are available as packages for Debian.
1. Retrieving the installation files via CVS
After logging in to the CVS server (see CVS area on SourceForge for instructions), make sure you check out two things: the base SystemDesigner module and at least one distribution you would like to generate CDs for. In this example, I’m getting the package that generates Ubuntu CDs.
cvs -z3 -d:pserver:anonymous@linuxcoe.cvs.sourceforge.net:/cvsroot/linuxcoe co -P SystemDesigner
cvs -z3 -d:pserver:anonymous@linuxcoe.cvs.sourceforge.net:/cvsroot/linuxcoe co -P SystemDesigner-Ubuntu
2. Reading the README
You should have a SystemDesigner/README file now. It’s a good idea to take a short look at that, even though I am going to walk you through the installation steps.
3. Customize configuration file
Change into the SystemDesigner directory. There, edit the file config.site to suit your needs. You can specify where to install the files for the web site, which user your web server runs as etc. The defaults that HP chose are all correct for Debian. Be aware that LinuxCOE rather aggressively installs some parts of the system into /var/www, including the directory where ISO images are generated. If your /var partition is rather small, make sure to customize that particular option. Most of the system is installed under /usr/local/systemdesigner/4, so you won’t have to worry too much about cluttering up your filesystem.
4. Make sure that your web server user can mount things
This is a bit of a critical step: your web server user (normally www-data) needs to be able to mount things. This is accomplished via sudo without a password. If you plan to install LinuxCOE on a very public web server, make sure you either run LinuxCOE’s own web server as a completely seperate user or make other arrangements for security. Giving the regular old web user mount privileges might not be such a great idea, security-wise, on a typical web server.
Use visudo to give www-data these privileges and add an entry like so:
# LinuxCOE SystemDesigner needs to mount/unmount
www-data ALL = NOPASSWD: /bin/mount, /bin/umount
5. Install base system
SystemDesigner is not installed via the usual ./configure && make && make install run that we’re all used to. Instead, this is how it’s documented in the INSTALL file. In the SystemDesigner directory:
export CONFIG_SITE="./config.site" && ./configure --help
export CONFIG_SITE="./config.site" && ./configure
If all went well (watch error output), that will generate the makefiles we can now use. Let’s go ahead and install, still in the SystemDesigner dir:
make
make install
make integrate
That last step is crucial! Also, the last two steps only work as root, since they write to various system locations (unless otherwise specified).
6. Install distribution-specific files
Now we’ll have to integrate the distribution-specific files we’ve downloaded earlier. Change into their directory, in this example, SystemDesigner-Ubuntu, and follow the instructions in the INSTALL file:
export CONFIG_SITE="/usr/local/systemdesigner/4/etc/includes/config.state" && ./configure --help
export CONFIG_SITE="/usr/local/systemdesigner/4/etc/includes/config.state" && ./configure
make
make install
If you’ve installed the base system to somewhere other than /usr/local/systemdesigner/4, you’ll just have to change the path accordingly, of course.
7. Add a geographically closer mirror
We don’t want to pound your chosen distribution’s main download servers, so let’s add a closer mirror to the config file for your distribution. Again, the example uses Ubuntu. Edit /usr/local/systemdesigner/4/osvend.d/Ubuntu and add a mirror address:
OSVEND archive.ubuntu.com Ubuntu Feisty i386 HTTP /ubuntu/
OSVEND ch.archive.ubuntu.com Ubuntu Feisty i386 HTTP /ubuntu/
The second line is the new one in this case, the main archive.ubuntu.com entry is there by default.
8. Restart web server and look at website
Now you should be ready to restart your web server and look at the fruits of your labor:
/etc/init.d/apache2 restart
Browse to http://yourwebserveraddress/systemdesigner and start making CD images!
9. burn CDs and test
All the work up until now would have been idiotic if we weren’t going to use those CDs. So find an old PC or something else where you can wipe the drive clean, stick the CD in and watch the magic. It’s quite amazing to see a machine install itself without any form of human interaction, especially if you’re used to the way this is normally done, via network distribution and configuration management. The CD route is a slick alternative that offers its own range of flexible options.
Where to go from here?
The admin guide contains a more in-depth explanation of every feature, especially how to customize the CDs that are generated. Also, you can give your LinuxCOE installation its own distinct look via editing the template HTML files.
If you end up using the system a lot on your network, don’t forget to implement some way to save bandwidth. Pulling those file packages off the distributor’s mirrors is not just a little rude, but also unneccessary. Stick a caching proxy between your LinuxCOE server and the outside world or, even better, build your own mirror of the distro’s repository. If you have enough resources, you could even make your mirror available to others in your location. Whichever method you choose, I’m sure you’ll have a lot of fun with your own little flock of custom CDs.
Bergtagung 2007: Call for Entries and Invitation
May 25th, 2007
Ever gave a talk while standing on a juicy, grassy meadow, in the middle of the forest, in the middle of the night? Everything around you pitch black, ants crawling on your flipchart, a flashlight the only source of illumination? And all that in the Swiss Alps?
How about speaking in your own little classroom, talking about that FreeBSD-based fruit juicer you’ve invented, with your audience of geeks sitting on small wooden chairs made for six year olds? Well, I guess now you can have that.
........ Introducing the BERGTAGUNG ..
...... A Meeting of Free Spirits and Open Minds ...
..... Somewhere in the Swiss Alps .....
... Involving Beer (and probably cows) ......
--[ bergtagung.org ]--
We’re so bloody open that you can dump the abstract of your talk in our wiki yourself.
Submit your talk to: bergtagung.org/wiki/Talks
Invitation
If your mind is reasonably open and your software predominantly free, you’re the perfect participant. If that does not sound like you, don’t worry. We have decent mind-stretching equipment and there will surely be some nerds around to fix you up with a liberated operating system of your choice.
Whatever you do, don’t stay at home!
Bergtagung 2007
(First Annual!)
July 20 – 22
Siat, Switzerland
Coming? Register at: bergtagung.org/wiki/Participants
Moo!
From Zero to Virtualization: Linux-Vserver vs. OpenVZ
April 13th, 2007
We’re currently evaluating solutions for virtualizing GNU/Linux servers at the HGKZ in order to replace seriously aging hardware (700 MHz P3’s!). At the same time, we can be hip like you and use important-sounding words such as "machine consolidation", "hypervisor" and "cuttlefish".
Gino is evaluating OpenVZ while I’m looking at Linux-Vserver. Both solutions have a similar approach: Don’t create virtual machines. Instead, create virtual servers that are sealed away from each other, but running on the same kernel. This has its own set of advantages and disadvantages, but I won’t go into that, you can read about it elsewhere.
Here’s a handy comparison table of what we found out so far:
| Linux-Vserver | OpenVZ | |
| Kernel | Pre-patched kernel included with Debian | Need to patch your own or rely on outside sources for binaries |
| Networking | Networking for guests works out of the box | Needs IP forwarding on the host, custom network config on the guest |
| Developers | Funny Austrians | Creepy Russians |
| Documentation | Horrible mess in six different states of rebuild and decay | Well-structured, organized and maintained |
| Affiliation | Snuggles a bit with RPM-based systems sometimes | Spends entire weekends in RPM-based systems’ beds and refuses to leave come Monday |
| Debian | Is treated like a leper, but a very friendly one | Is the enemy |
| Guest configuration | Implicit but convoluted | Explicit but straightforward |
There, that’s hard scientific facts for you.
After all of this probing, compiling, tickling, testing and general mayhem, we have decided to go with OpenVZ. To reach that decision, we of course evaluated both solutions on many levels (don’t let that table fool you). There are clear philosophical and architectural differences between the two solutions, but one key factor in our decision was that for the administrator, both systems are almost too similar.
Yes, OpenVZ takes a more complicated approach to networking, but Linux-Vserver takes a more complicated approach to configuration. Yes, a Linux-Vserver host’s default config is mostly what you want and just seems to work out of the box, but this lowers your motivation for learning the details of the resource management system. And details, as you surely know, are nearly always ugly.
With OpenVZ, you are forced to learn these things up front, which presents a steeper learning curve but gifts you with a more solid grasp of the technology. You get to flex your math muscle to fit virtual servers into your actual hardware’s limitations without creating an impossible physical paradoxon that rips a hole into space-time, and that’s quite handy. With Linux-Vserver, these things might come back to haunt you later, when you’re trying to put vserver no. 22 onto your machine and discover your 16 GB of memory are already spent, and that’s when details bite a tasty chunk right out of your lower backside. The decrepit state of Linux-Vserver’s documentation does nothing to ease your fears in this department, either. Convoluted configuration is okay, as long as it’s well-documented convoluted configuration.
Now what if we are wrong, and within the next 8 months someone writes The Linux-Vserver Bible (Illustrated Swimsuit Edition) and SWSoft decides to pull the plug on support for OpenVZ, leaving us without any burly Russian engineers to take care of the code? That may seem sad, but it paves the way for such a beautiful pink-colored fluffy thought that it nearly makes my skull burst: We would still be fine. Both of the solutions are open. No proprietary formats. No secrets. We can migrate from one to the other at any time.
It’s the beauty of Free Software once again.
Windows Games Run Faster on Linux than on Windows Vista
April 12th, 2007
WINE is an implementation of the Win32 API on Linux, so that you can run programs written for Windows on your GNU/Linux installation. It normally reeks of office applications, spreadsheets and polka-dotted ties. Cedega, then, is a commercial version of WINE optimized for games. It runs a larger library of Windows games than plain WINE can. So far, so good, right? Now, common sense might dictate that the games should run slower in WINE or Cedega, because those things are busy translating the Windows-nonsense the games speak into something that GNU/Linux can understand, and that surely costs performance, no?
Common sense is wrong. These benchmarks say that GNU/Linux with Cedega or WINE runs the tested games 33 – 40% faster than Windows Vista. 40%! That’s not just the fraction of fps that the hardcore crowd lusts for, that’s a significant number.
So the best modern platform to play Windows games on is not Windows but GNU/Linux? Creepy.
Thin client handover at the Polytechnic, Malawi, Africa
December 6th, 2006
Alex Antener yesterday managed to wrap up this year’s stage of his Free Software project in Malawi, Africa with the official handover of the two complete thin client networks donated by the University of Applied Sciences and Arts, Zürich (my dear employer).
In the picture you see Martin Thawani, librarian of the Polytechnic Library, accepting the symbolical gift "Free as in Freedom". That book "interweaves biographical snapshots of GNU project founder Richard Stallman with the political, social and economic history of the free software movement". The GNU project is what makes GNU/Linux and the GNU tools possible in the way we know them today.
Alex Antener’s approach to helping the African nation cross the digital divide is different than that of many other organizations and individuals. Instead of dumping northern computer trash on poor schools that certainly won’t ask for something better, he flew across the continents with 70 kilograms of the latest geek toys in his hand baggage. Highly modern servers based on the Intel Core 2 Duo CPU as well as state of the art thin clients by Fujitsu-Siemens — machines newer than what is used in most European organizations! That’s what we use here, why should we cover Malawi in our outdated tech trash? That’s just a convenient way for northern nations to lighten their consciousness and their recycling budget.
Instead, Alex set up the thin client network based entirely on Free Software, then made it transparent how the whole thing works, how it’s maintained, what the nuts and bolts are and where to find help to help yourself. These things would be impossible or severely limited had he used proprietary software. Additionally, the servers he installed make sure that the Polytechnic gets the most out of its prohibitively expensive Internet connection. Firstly they offer proxy caching services, meaning that things downloaded from the Internet are downloaded only once, later the locally stored copy is served and the Internet connection is not taxed anymore. Secondly, the machines are immune to viruses, spyware, trojans and other malware, so the plague of bandwidth-swallowing infected machines is over.
I also took part in the project with some consulting, because I believe the way large western and northern corporations treat African nations of Malawi’s rank is appalling. Africa is often merely abused by private institutions and NGOs to siphon development aid money out of their own (or foreign) governments. Then there is the cultural pollution that comes from large companies like Microsoft and Cisco. They try to impose their proprietary technology, then teach their proprietary thoughts. It’s apparently easy to take a network engineering course in Malawi, but try to learn any other technology than Cisco’s and you will soon run full-speed into a concrete wall. Africa is not supposed to learn about its possibilities. Africa should be thankful! Thankful that we lower ourselves to its ridiculous level and teach it about our wonderful American products. Only Cisco routers shall they know, only Microsoft operating systems shall they use. Operating systems that African companies can gladly buy from us. Oh, don’t worry about payment, my friend, development aid has you covered.
Alex has demonstrated that there are other ways to bridge the gap, to give access to knowledge that is useful in any context, not just inside one single company’s little sandbox. Free Software was nothing new at the time. GNU/Linux and open standards like the ISO standard OpenDocument format were nothing new either. But the average Malawian computer user does not know about these things, even though it’s a Linux distribution by an African man’s company that is the most popular in the world at this moment.
The Polytechnic now has all this information, and it stands as an inspiring example of what is possible. A few hundred people have learned about their possibilities in these last two months, and thousands and ten thousands more still have the opportunity in the coming years.
Photo (C) 2006 Nathalie Bissig