Paul Boddie's Free Software-related blog

Archive for June, 2013

Norwegian Voting and the Illusion of “Open Source”

Sunday, June 30th, 2013

I was interested to read about the new Norwegian electronic voting administration system, EVA, and a degree of controversy about whether the system can scale up to handle the number of votes expected in Oslo during the upcoming parliamentary elections. What I find more controversial is the claim that the system has been made available as “open source” software. In fact, a quick look at the licence is enough to confirm that the source code is really only available as “shared source”: something which people may be able to download and consult, but which withholds the freedoms that should be associated with open source software (or Free Software, as we should really call it).

So, here is why the usage of the term “open source” is dishonest in this particular case:

  • The rights offered to you (as opposed to the Norwegian authorities) cover only “testing, reviewing or evaluating the code”. (The authorities have geographical and usage restrictions placed on them.)
  • The licence restricts use to “non-commercial purposes”.
  • Anything else you might want to do requires you to get “written approval” from the vendors of the different parts of the system in question.
  • The software is encumbered by patents, but there is no patent grant.

Lost in Translation

In fairness, the page covering the source code does say the following about the use of the term “open source” which I have translated from the Norwegian:

When we use the term “open source”, we do not mean that the entire solution can be regarded as “free software”, meaning that you can download the software, inspect it, change it, and use it however you like. Anyone can download and inspect the source code, but it may only be used to carry out Norwegian elections. The solution should be available for research, however, and you are allowed to develop it further for such academic purposes.

Why translate from Norwegian when the English is underneath in the same page? Here’s why:

When we are using the term “open source code”, we do not say that the source code as a whole is what’s known as “open source”, that is code that can be freely downloaded, examined, changed and used. Anyone can download our source code, but it is only permittable to use it for elections in Norway. However, research on the solution is allowed and you are hence allowed to develop the system for an academic purpose.

In other words, while there are people involved who are clearly aware of what Free Software is, they apparently reserve the right to misuse the term “open source” to mean something other than what it was intended to mean. By consulting the Norwegian text first, I even give them the benefit of the doubt, but the contradiction of saying that something both is and is not something else should have awakened a degree of guilt that this could be regarded as deception.

Open Season

Now, one may argue that the Open Source Initiative should have been more aggressive in upholding its own brand and making sure that “open source” really does act as a guarantee of openness, but the “open” prefix is perhaps one of the most abused terms in the field of computing, and thus one might conclude that the OSI were fighting a losing battle from the outset. Of course, people might claim that the term “free software” is ambiguous or vague, which is why I prefer to write it as Free Software: the use of capital letters should at least get English readers unfamiliar with the term to wonder whether there is more involved than the mere juxtaposition of two widely-used words.

In Norwegian, “fri programvare” communicates something closer to what is meant by Free Software: the “fri” does not generally mean “for free” or “gratis”, but rather communicates a notion of freedom. I imagine that those who translated the text quoted above need to improve their terminology dictionary. Nevertheless, this does not excuse anyone who takes advantage of the potential ambiguity in the common sense perception of the term to promote something that has only superficial similarities to Free Software.

The Good, the Bad, and the Evry

I suppose we should welcome increased transparency in such important systems, and we should encourage more of the same in other areas. Nevertheless, with a substantial amount of activity in the field of electronic voting, particularly in the academic realm and in response to high-profile scandals (with some researchers even apparently experiencing persecution for their work), one has to wonder why the Norwegian government is not willing to work on genuinely open, Free Software electronic voting systems instead of partnering with commercial interests who, by advertising patent coverage, potentially threaten research in this area and thus obstruct societal progress, accountability and democracy.

One might suggest that the involvement of EDB ErgoGroup, now known as Evry, provides some answers to questions about how and where taxpayer money gets spent. With substantial state involvement in the company, either through the state-owned postal monopoly or the partly-state-owned incumbent telephone operator, one cannot help thinking that this is yet another attempt to funnel money to the usual beneficiaries of public contracts and to pick winners in an international market for electronic voting systems. Maybe, with voting problems, banking system problems and the resulting customer dissatisfaction, public departments and ministries think that this vendor needs all the help it can get, despite being appointed to a dominant position in the nation’s infrastructure and technology industry to the point of it all being rather anti-competitive. When the financial sector starts talking of monopolies, maybe the responsible adults need to intervene to bring a degree of proper functioning to the marketplace.

But regardless of who became involved and their own rewards for doing so, the Norwegian ministry responsible for this deception should be ashamed of itself! How about genuinely participating in and advancing the research and development of voting systems that everyone on the planet can freely use instead of issuing dishonest press releases and patronising accounts of how Norway doesn’t really need to learn from anyone? Because, by releasing patent-encumbered “shared source” and calling it “open source”, some decision makers and their communications staff clearly need lessons in at least one area: telling the truth.

An Aside

I also find it distasteful that the documentation hosted on the government’s electoral site has adverts for proprietary software embedded in it, but I doubt that those working in the Microsoft monoculture even notice their presence. These people may be using taxpayer money to go shopping for proprietary products, but I do not see why they should then be advertising those products to us as well using our money.

Resource Autocompletion in Roundcube

Friday, June 21st, 2013

In a previous article, I described my experiences setting up Kolab for groupware functionality on Debian Wheezy. One of the problems I encountered was that of searching for resources when creating events, and it didn’t seem possible to start typing the name of a resource and to have the details autocompleted. Given that Kolab integrates Roundcube webmail with other services including LDAP directories, and given that Roundcube seemed happy to look up people in such directories, I suspected that fixing this problem would probably involve refining the search criteria for each search performed when a key is pressed in the participant field of the event dialogue (or in the recipient field of the compose mail screen).

After some digging in the source code for the purpose of getting some familiarity with what goes on inside Roundcube, I found a guide to LDAP address books in Roundcube that mentions some of the queries one might expect to happen when autocompletion is taking place. And, sure enough, such information can be found in the /etc/roundcubemail/ file provided by the Kolab-related Debian packages. So it then becomes a matter of specifying some other queries to permit resources to be found as well as people.

My solution to the problem, which may not be the most appropriate (so I welcome corrections and comments), is to add another address book provider as follows:

    $rcmail_config['ldap_public'] = array(
            'kolab_resources' => array(
                    'name'                      => 'Global Resources',
                    'base_dn'                   => 'ou=Resources,dc=example,dc=com',
                    'LDAP_Object_Classes'       => array("top", "mailrecipient"),
                    'required_fields'           => array("cn", "mail"),
                    'search_fields'             => array('cn', 'mail'),
                    'sort'                      => array('cn', 'mail'),
                    'filter'                    => '(objectClass=mailrecipient)',
                    'fieldmap'                  => Array(
                            // Roundcube        => LDAP
                            'name'              => 'cn',
                            'email:primary'     => 'mail',
                            'email:alias'       => 'alias',

    $rcmail_config['autocomplete_addressbooks'] = Array(

Here, the new entry for kolab_resources augments the existing kolab_addressbook entry (not shown), and it changes the nature of the search by modifying the base_dn to refer to the “Resources” organisational unit, where Kolab puts all the resources in the LDAP store. Since resources do not seem to provide various fields that people provide, some changes are also required to indicate which fields are provided, and in the fieldmap section the name expected by Roundcube is mapped to the cn provided by the LDAP store, thus enabling the name of each resource to appear alongside the mail address by which the resource is known.

With the new entry added, the autocomplete_addressbooks setting needs to be updated to include this new source of data in any future searching operations. And with that, it should be possible to specify a resource and have it autocompleted in Roundcube:

Resource autocompletion in Roundcube

Resource autocompletion in Roundcube

Evaluating Free Software Groupware: Kolab

Thursday, June 13th, 2013

I have recently had the inclination to evaluate Free Software groupware solutions in more detail, and perhaps the first that came to mind was Kolab: a long-running project that provides a range of groupware functions including e-mail, calendaring, address books, task management, and various other functions for a fairly wide range of organisation sizes. Of course, there are plenty of Free Software groupware projects offering complete and integrated solutions as well as individual components for use with existing infrastructure; the Debian Wiki page on groupware provides a fair (but probably incomplete) overview of the more interesting projects.

Installing and Configuring Kolab

Intrigued by accounts that Kolab is fairly easy to install on Debian Wheezy – the latest stable release of the Debian GNU/Linux software distribution – I set out to investigate, making use of my own tools to set up a User Mode Linux environment in which I could install the software. Initially, I tried to re-use an existing virtual environment, but a quick attempt to configure the software using the setup-kolab program was not successful, and a brief excursion via the #kolab IRC channel (on freenode), indicated that I might be better off starting with a completely fresh installation of Wheezy. Although I imagine it is possible to deal with the problems I encountered – setup-kolab did not like the presence of an existing LDAP server – the easiest way to troubleshoot is to start with a known configuration and see if things can be made to work from there.

Installation of Kolab 3.0 on Debian is fairly straightforward, as described both in the manual and more concisely in the blog article mentioned above (and also in older reports). The Kolab packages in Debian are set up to prefer the postfix packages to the apparent default of the exim4 packages and thus want to replace the latter. This might be a problem in some environments, and it may be possible to retain Exim for use with Kolab, but I haven’t investigated this. A somewhat undesirable feature of the currently available packages is that they are unsigned: Debian makes extensive use of package signatures to prevent tampering, and although it can be an annoyance to sign and publish packages and to publish the necessary keys for verification, hopefully Kolab will make its way into Debian as a collection of official packages once again.

Some Current Pitfalls

With a fresh system, setup-kolab seems fairly happy, and with the initial configuration performed it is possible to log into the administration interface, although it seems to be necessary to explicitly start the Apache server first. One strange problem with the Debian packages seems to be in the absence of a library file in the correct location, and this manifests itself in the administration interface as the absence of any way to add users. I fixed this for my system as follows:

ln -s /usr/lib/i386-linux-gnu/nss/ /usr/lib/

(Unlike the message linked above describing this fix, I still use a machine with the i386 architecture, not the x86_64 architecture, and the underlying problem seems to be related to the way that libraries are now stored to permit support multiple architectures on the same computer.)

I also noticed that some Kolab component, at least after some administrative tasks have been performed, tries to communicate with the IMAP server unsuccessfully but persistently. To reset their relationship, the following seemed to be required:

service cyrus-imapd restart

Some other complaints emerged on the console about mailbox creation, perhaps due to some resources I created, but it is possible to verify the state of the mailboxes as follows:

kolab list-mailboxes

I noticed that no matter which resource type I specified, the type of created resources would always be “Beamer”.

A resource created as "Car" but shown as "Beamer"

But it's a Porsche!

This probably doesn’t matter so much for actual resource booking, but I imagine that there’s a problem here needing to be fixed. It is possible that the Debian packages suffer from the above problems but that these problems have since been fixed in the project’s repository and in subsequent non-Debian package or distribution releases; I haven’t verified this, however.

Fun With Administration

Administration is never really much fun, but the administrative interface seems to provide a reasonable way of adding users and resources, populating the different information stores with user and mailbox details.

The main page of the administrative interface

The main page of the administrative interface

With the packaging issues mentioned above all sorted out, users can be added in the users section:

Adding a user in the administrative interface

Adding a user in the administrative interface

And resources can be added in the resources section:

Adding a resource in the administrative interface

Adding a resource in the administrative interface

Given that Kolab is based on conventional services like LDAP directories, IMAP mailboxes, and so on, if you needed to integrate with existing infrastructure and accommodate existing user populations, you probably wouldn’t spend much time in the administrative interface, but it is nice to see that an interface exists for quick edits to the system.

What About the Users?

With some users set up, one might be interested in seeing things from their perspective. Out of the box, the Debian packages provide a Roundcube webmail interface:

The Kolab Roundcube login page

The Kolab Roundcube login page

On the inside, the interface is much like the Roundcube many people have come to know. For instance, the mail interface is more or less what you would expect. Here, the folders on the left are IMAP folders that are also available to IMAP clients, but to start with there obviously aren’t any mails to look at:

The Kolab Roundcube mail interface

The Kolab Roundcube mail interface

Amongst the usual view buttons at the top of the window, featuring the mail, address book and settings, we find additional buttons for the calendar and tasks. First, the address book:

The address book in Roundcube

The address book in Roundcube

Here, it seems to pick up other users added via the administrative interface. Meanwhile, the calendar interface is probably slightly more interesting to look at because it’s something that you don’t usually get in Roundcube:

The calendar in the Kolab version of Roundcube

The calendar in the Kolab version of Roundcube

The calendar widgets seem to be rather familiar and those who do more JavaScript programming than I do will probably be able to identify the project that pioneered them. Nevertheless, they seem to behave mostly as I would expect from having used them elsewhere on other sites and services. One strange thing is the date numbering above the days in the week view (“Mon 6-10” meaning “Monday 10th June”, for example) which I imagine could be customised somewhere, although I didn’t see a setting to do exactly that.

Fun With Events

Given the existence of the calendar in Roundcube, and given that calendaring interests me already, I decided to make an attempt at creating a new event, inviting a participant, and requesting a resource. Dragging an area in the calendar caused the event dialogue to appear:

Adding a new event in Roundcube

Adding a new event in Roundcube

The location field appears to be non-autocompleted free text, but it would be nice to have a menu of recognised locations or resources, and perhaps there is some kind of setting or extension to provide that. With the main details filled out, on I went to the participants tab:

Finding participants for an event in Roundcube

Finding participants for an event in Roundcube

Just like the mail interface in Roundcube, the calendar also supports address lookups and offers autocompletion of names. However, I found that autocompletion didn’t take place for resources, so I ended up having to invite resources by using their full e-mail addresses (which were defined previously in the administrative interface). For example, for the “Forest” resource, I had to specify as a participant. Maybe this is also something that should be done another way, but I didn’t manage to figure it out.

Finding the availability of participants seems possible. Kolab does support the retention of free/busy information, so for those people making this information available to Kolab, their status should be visible in the user interface:

Participant availability in the event dialogue

Participant availability in the event dialogue

In principle, it should be possible for people to exchange free/busy information via e-mail and for the recipient to record this information and use it to schedule events, but I haven’t looked into whether Kolab or Roundcube support this at their respective levels. I found that in the availability view, it is possible to change the role of each participant by clicking on the icon next to their name, and this made it possible to give a resource the appropriate role. Again, if there were a better way of choosing a resource that I missed, maybe this wouldn’t be necessary.

With an event created and participants invited, Kolab manages to notify those participants, and to make things interesting I decided to configure Kontact in a KDE 4 environment (running in Debian Squeeze) to connect on behalf of the invited participant. Here is what that participant sees when they check their mail:

Kontact showing an event notification message

Kontact showing an event notification message

Although it is rather small in the above screenshot, Kontact shows a collection of links that allow the recipient to act on an incoming event notification. Here is a close-up:

The event invitation actions in a message

The event invitation actions in a message

For Kontact to be able to do this, it appears that the kdepim-groupware package is required, and indeed this functionality supports the iTIP technology mentioned above (here, in an invitation context instead of the free/busy context discussed above). It is important to understand that the open standards underpinning this workflow do not require that everyone have a login to a common server and manipulate information on that server directly: a critical feature of the iCalendar-related standards is that people are able to schedule events collaboratively without all being part of the same monolithic organisation and/or infrastructure. It is also interesting to see that where a recipient’s e-mail program cannot handle the workflow defined by iTIP, the message includes a link to the Roundcube webmail that can be used to signal a participant’s attendance or absence.

When a participant responds using one of the links provided in the message, the organiser gets a notification. Here, the Roundcube user gets to see a mail message telling them that the participant accepted the invitation:

A received acceptance of an event invitation

A received acceptance of an event invitation

Upon pressing the update button provided, the status of the event is updated in the calendar:

The updated event in the calendar

The updated event in the calendar

Here, the organiser is shown with a crown next to his name, the participant (using Kontact) has accepted the invitation to the event, and the resource has apparently been secured.

In Conclusion

There are obviously plenty of other experiments that could be performed here, as well as other features that could be explored. For instance, some more evaluation of the free/busy information, how local and remote users interact with it, and how well those with non-iTIP mail clients fare with over-the-Web notification of attendance or absence might be in order. Publishing calendars for over-the-Web consumption is also apparently supported, and it might be interesting to see how well Kolab supports the general “invite people you hardly know” event-planning paradigm that the likes of Doodle have been attempting to popularise.

It seems that Kolab at the very least supports basic calendar functionality in association with standards-compatible clients, and perhaps a brief investigation with Thunderbird (plus Lightning) and even more elementary mail and calendar clients might be informative. Since Kolab is Free Software, of course, the chances of resolving any shortcomings are increased for those willing and able to peruse and modify the code, and maybe I will take a closer look at that, too.

As noted above, calendaring and scheduling systems are already an interest of mine. The only problem now is that there’s just so much to look at and yet so little time to do so!

Site Licences and Volume Licensing: Locking You Both In… and Out

Sunday, June 9th, 2013

Once upon a time, back in the microcomputer era, if you were a reputable institution and were looking to acquire software it was likely that you would end up buying proprietary software, mostly because Free Software was not a particularly widely-known concept, and partly because any “public domain” or “freeware” programs probably didn’t give you or your superiors much confidence about the quality or maintenance of those programs, although there were notable exceptions on some platforms (some of which are now Free Software). As computers became more numerous, programs would be used on more and more computers, and producers would take exception to their customers buying a single “copy” and then using it on many computers simultaneously.

In order to avoid arguments about common expectations of reasonable use – if you could copy a program onto many floppy disks and run that program on many computers at once, there was obviously no physical restriction on the use of copies and thus no apparent need to buy “official” copies when your computer could make them for you – and in order to avoid needing to engage in protracted explanations of copyright law to people for whom such law might seem counter-intuitive or nonsensical, the concept of the “site licence” was born: instead of having to pay for one hundred official copies of a product, presumably consisting of one hundred disks in one hundred boxes with one hundred manuals, at one hundred times the list price of the product, an institution would buy a site licence for up to one hundred computers (or perhaps as many as the institution has, betting on the improbability that the institution will grow tenfold, say) and pay somewhat less than one hundred times the original price, although perhaps still a multiple of ten of that price.

Thus, the customer got the vendor off their back, the vendor still got more or less what they thought was a fair price, and everyone was happy. At least that is how it all seemed.

The Physical Discount Store

Now, because of the apparent compromise made by the vendor – that the customer might be paying somewhat less per copy – the notion of the “volume licence” or “bulk discount” arose: suddenly, software licences start to superficially resemble commodities and people start to think of them just like they do when they buy other things in bulk. Indeed, in the retail sector the average person became aware of the concept of bulk purchasing with the introduction of cash and carry stores, discount stores, and so on: the larger the volume of goods passing through those channels, the bigger the discounts on those goods.

Now, economies of scale exist throughout modern commerce and often for good reason: any fixed costs (or costs largely insensitive to the scale of output) in production and distribution can be diluted by an increased number of units produced and shipped, making the total per-unit cost less; commitments to larger purchases, potentially over a longer period of time, can also provide stability to producers and suppliers and encourage mutually-beneficial and lasting relationships throughout the supply chain. A thorough treatment of this topic is clearly beyond a blog post, but it is worthwhile to briefly explore how savings arise and how discounts are made.

Let us consider a producer whose factory can produce at most a million units of a product every year, it may not seek to utilise this capacity if it cannot be sure that all units will be sold: excess inventory may incur warehouse costs and also result in an uncompetitive product going unsold or needing to be heavily discounted in order to empty those warehouses and make room for more competitive stock. Moreover, the producer may need to reconsider their employment levels if the demand varies significantly, which in some places incurs significant costs both in reduction and expansion. Adding manufacturing capability might not be as easy as finding a spare factory, either. All this additional flexibility is expensive for producers.

However, if a large, well-known retailer like Wal-Mart or Tesco (to name but two that come to mind immediately) comes along and commits to buying most or all of the production, a producer now has more certainty that the inventory will be sold and that it will not be paying people to do nothing or to suddenly have to change production lines to make new products, and so on. Even things like product variations can be minimised by having a single customer or few customers, and this reduces costs for the producer still further. Naturally, Wal-Mart would expect some of the savings to be passed on to them, and so this relationship benefits both parties. (It also produces a potential discount to be passed on to retail customers who may not be buying in bulk after all, but that is another matter.)

The Software Discount Store?

For software, even though the costs of replication have been driven close to nothing, the production of software certainly has a significant fixed cost: the work required to develop a viable product in the first place. Let us say that an organisation wishes to make and sell a non-niche product but needs to employ fifty people for two years to do so (although this would have been almost biblical levels of manpower for some successful software companies in the era of the microcomputer); thus one hundred person-years are invested in development. To just remain in business while selling “copies” of the software, one might need to sell one hundred thousand individual copies. That is if the company wants to just sell “licences” and not do things like services, consulting, paid support, and so on.

Now, the cost of each copy can be adjusted according to the number of sales. If things go better than expected, the prices could be lowered because the company will cover its costs more quickly than anticipated, but they may also raise the prices to take advantage of the desirability of the product. If things go worse than expected, the prices might be raised to bring in more revenue per sale, but such pricing decisions also have to consider the customer reaction where an increased price turns away customers who can no longer justify the expense. In some cases, however, raising the price might make the product seem more valuable and make it more attractive to potential customers, despite the initial lack of interest from such customers.

So, can one talk about economies of scale with regard to software as if it were a physical product or commodity? Not really. The days of needing to get more disks from the duplicator, more manuals from the printer, and to send more boxes to distributors are over, leaving the bulk of the expense in employing people to get the software written. And all those people developing the product are not producing more units by writing more code or spending more time in the office. One can argue that by adding more features they are generating more sales, but it is doubtful that the relationship between features and sales is so well defined: after a while, a lot of the new features will be superfluous for all but “power users”. One can also argue that by adding more features they are making the product seem more valuable, and so a higher price can be justified. To an extent this may be the case, but the relationship between price and sales is not always so well defined, either (despite attempts to do so). But certainly, you do not need to increase your “production capacity” to fulfil a sales need: whether you make one hundred or one million sales (or generate a tenth of or ten times the anticipated revenue) is probably largely independent of how many people were hired to write the code.

But does it make sense to consider bulk purchasing of software as a way of achieving savings? Not really. Unlike physical production, there is no real limit to how many units are sold to customers, and so beyond a certain threshold demanded by profitability, there is no need for anyone to commit to purchasing a certain number of units. Especially now that a physical component of a software product is unlikely to be provided in any transaction – the software is downloaded, the manual is downloaded, there is no “retail box”, no truck arriving at the customer, no fork-lift offloading pallets of the product – there is also no inventory sitting in a warehouse going unsold. It might be nice if someone paid a large sum of money so that the developers could keep working on the product and not have to be moved to some other project, but the constraints of physical products do not apply so readily here.

Who Benefits from Volume Licensing?

It might be said, then, that the “economies of scale” argument starts to break down when software is considered. Producers can more or less increase supply at will and at a relatively low cost, and they need only consider demand in order to break even. Beyond that point, everything is more or less profit and they deliver units at no risk to themselves. Certainly, a producer could use this to price their products aggressively and to pass on considerable savings to customers, but they have no obligation and arguably little inclination to do so for profitability reasons alone. Indeed, they probably want to finance new products and therefore need the money.

When purchasers of physical goods choose to buy in bulk, they do so to get access to savings passed on by the producer, and for some categories of products the practice of committing larger sums of money to larger purchases carries little risk. For example, an organisation might buy a larger quantity of toilet paper than it normally would – even to the point of some administrator complaining that “this must be more than we really need!” – and as long as the organisation had space to store it, it would surely be used over time with very little money wasted as a result.

But for software, any savings passed on by the producer are more discretionary than genuine products of commerce, and there is a real risk of buying “more than we really need”: a licence for an office application will not get “used up” when someone has “reached the end” of another licence; overspending on such capacity is just throwing money away. It is simply not in the purchaser’s interest to buy too many licences.

Now, software producers have realised that their customers are sensitive to this issue. Presumably, the notion of the site licence or “volume licensing” arose fairly quickly: some customers may have indicated that their needs were not so well-defined that they could say that they needed precisely one hundred copies of a product, and besides, their computer users might not have all been using the software at the same time, and so it might not make sense to provide everyone with a copy of a program when they could pass the disks around (or in later times use “floating licences”). So, producers want customers to feel that they are getting value for money and not spending too much, and thus the site licence was presumably offered as a way of stopping them from just buying exactly what they need, instead getting them to spend a bit more than they might like, but perhaps a bit less than they would need to if money were no object and per-unit pricing was the only thing on offer. (The other way of influencing the customer is, of course, the threat of audits by aggressive proprietary software organisations, but that is another matter.)

Regardless of the theory and the mechanisms involved, do customers benefit from site licences? Well, if they spend less on a site licence than they do on the list price of a product multiplied by the number of active users of that product, then they at least benefit from savings on the licensing fees, certainly. However, there are other factors involved, introducing other broader costs, that we will return to in a moment.

Do producers benefit from site licences? Almost certainly. They allow companies to opportunistically increase revenue by inviting customers to spend a bit more for “peace of mind” and convenience of administration (no more having to track all by yourself who is using which product and whether too many people are doing so because a “helpful” company will take care of it for you). If such a thing did not exist, customers would probably choose to act conservatively and more closely review their purchases. (Or they might just choose to embrace Free Software instead, of course.)

All You Won’t Eat

But it is the matter of what the customer needs that should interest us here. If customers did need to review their purchases more closely, they might find it hard to justify spending large sums on volume licences. After all, not everyone might be in need of some product that can theoretically be rolled out to everyone. Indeed, some people might prefer another product instead: it might be much more appropriate for their kind of work, or it might work better on their platform (or even actually work on their platform where the already-bought product does not).

And where the organisation’s purse strings are loosened when buying a site licence for a product in the first instance, the organisation may not be so forthcoming with finance to acquire other products in the same domain, even if there are genuine reasons for doing so. “You already have an office program you can use; why do you want us to buy another?” Suddenly, instead of creating opportunities, volume licensing eliminates them: if the realm of physical products worked like this, Tesco would offer only one brand of toilet paper and perhaps not even a particularly pleasant one at that!

But it doesn’t stop there. Some vendors bundle products together in volume licensing deals. “Why not indulge yourself with a package of products featuring the ones you want together with some you might like?” This is what customers are made to ask themselves. Suddenly, the justification for acquiring a superior product from a competitor of the volume licensing provider is subject to scrutiny. “You already have access to an intranet solution; why do you want us to spend time and money on another?” And so the supposedly generous site licence becomes a mechanism to rein in spending and even the mere usage of alternatives (which may be Free Software acquired at no cost), all because the acquisition cost of things that people are not already actively using are wrongly perceived as being “free”. “Just take advantage of the site licence!” is what people are told, and even if the alternatives are zero cost, the pressure will still be brought to bear because “we paid for things we could use, so let’s use them!”

And the Winner is…

With such blinkered thinking the customer can no longer sensibly exercise choice: it becomes too easy to constrain an organisation’s strategy based on what else is in the lucky dip of products included in the multiple product volume licensing programme. Once one has bought into such a scheme, there is a disincentive to look elsewhere for other solutions, and soon every need to be satisfied become phrased in terms of the solutions an organisation has already “bought”. Need an e-mail system? The solution now has to be phrased in terms of a single vendor’s product that “we already have”. And when such extra purchases merely add to proprietary infrastructure with proprietary dependencies, that supposedly generous site licence is nothing but bait on the end of the vendor’s fishing line.

We know who the real winner is here. The real loser is anyone having to compete with such schemes, especially anyone using open standards in their products, particularly anyone delivering Free Software using open standards. Because once people have paid good money for something, they will defend that “investment” even when it makes no real sense: this is basic human psychology at work. But the customer is the loser, too: what once seemed like a good deal will just result in them throwing good money after bad, telling themselves that it’s the volume of usage – the chance to sample everything at the “all you can eat” buffet – that makes it a “good investment”, never mind that some of the food at the buffet is unhealthy, poor quality, or may even make people ill.

The customer becomes increasingly “locked in”, unable to consider alternatives. The competition becomes “locked out”, unable to persuade the customer to migrate to open-standards-based solutions or indeed anything else, because even if the customer recognised their dependency on their existing vendor, the cost of undoing the mess might well be less predictable and less palatable than a subscription fee to that “preferred” vendor, appearing as an uncomfortably noticeable entry in the accounts that might indicate strategic ineptitude or wrongdoing – that a mistake has been made – which would be difficult to acknowledge and tempting to conceal. But when the outcome of taking such uncomfortable remedial measures would be lower costs, truly interoperable systems and vastly increased choice, it would be the right thing to do.

One might be tempted to just sit back and watch all this unfold, especially if one has no connection with any of the organisations involved and if the competition consists only of a bunch of proprietary software vendors. But remember this: when the customer is spending your tax money, you are the loser, too. And then you have to wonder who apart from the “preferred” vendor benefits from making you part of the losing team.

Horseplay in Public Procurement? “Standards!”

Thursday, June 6th, 2013

There is a classic XKCD comic strip where the programmer, “slacking off” in the office and taking a break from doing work, clearly engaging in horseplay, issues the retort “Compiling!” to get his supervisor or peers off his back. It is seen as the ultimate excuse for not doing one’s work, immediately curtailing any further investigation of what really is going on in the corridor. Having recently been investigating some strategic public sector purchasing decisions, it occurred to me that something similar is going on in that area as well.

There’s an interesting case that came up a few years ago: Oslo municipality sought to acquire infrastructure for e-mail and related functionality. The scope of the tender covered “at least 30000 accounts” for client and server software, services and assistance, which is a pretty big tender but not unexpected given that the municipality is one of the largest single employers in Norway with almost 50000 employees (more statistics available here). Unfortunately, the additional documents are no longer available (and are generally not publicly available at the state procurement portal – you have to register as an interested party), but they are quoted in various places. Translating one particular requirement…

“Oslo municipality has standardised on Microsoft Office as office productivity software. It is therefore expected that solutions use MS Outlook 2003 and later as client.”

Two places where the offending requirements are reproduced are in complaints to the state procurement panel: 2009/124 and 2009/153. In these very similar complaints, it is pointed out that alternatives to Outlook can be offered as options (this is in the original tender), but that the municipality would only test proposed solutions with Outlook. As justification for insisting on Outlook compatibility, the municipality claimed that they had found “six different large companies providing relevant software in connection with the drafting of the requirements… all of which can be used together with Outlook”, and thus there was a basis for real competition. As a result, both complaints were rejected.

The Illusion of Compatibility

Now, one might claim that it is perfectly reasonable to want to acquire systems that work with the ones you already have. It is a bit like saying, “I’ve bought all this riding equipment: of course I want a horse!” The deeper issue here is whether anyone should be allowed to specify product compatibility to limit competition. In other words, when you just need transport to get around, why have you made your requirements so specific that you will only ever be getting a horse?

It is all very well demanding compatibility with a specific product, but when the means by which compatibility can be achieved are controlled by the vendor of that product, it is never going to be a fair competition for anyone trying to provide compatibility for their own separate products and solutions, especially when the vendor of the specified product is known to have used compatibility breakage to deliberately undermine the viability of competitors’ products. One response to this pitfall is to insist that those writing procurement tenders specify standards instead of products and that these standards must be genuinely open and not de-facto proprietary standards.

Unfortunately, the regulators of procurement do not seem to go even this far. The Norwegian government states that public sector institutions must support various standards, although the directorate concerned appears to have changed these obligations from the original directive and now insists that the dubious, forcibly- and incompletely-standardised Office Open XML document format must be accepted by the public sector in communications; they have also weakened the Internet publishing requirements for public sector institutions by permitting the use of various encumbered, cartel-controlled audio and video formats. For these changes, entertained in a review process, we can thank the likes of Statistics Norway who wanted “Word format” as well as OOXML to be permitted in the list of acceptable “standards”.

In any case, such directives only cover the surface of public sector activity, and the list of standards do not in general cover anything more than storage and interchange formats plus basic communications standards. This leaves quite a gap where established Internet standards exist but are not mandated, thus allowing proprietary protocols and technologies to insert themselves into infrastructure and pervert the processes of procurement and systems integration.

The Pretense of “Standards!”

But even if open standards were mandated in the public sector – a worthy and necessary measure – that wouldn’t mean that our work to ensure a level playing field – fairness in procurement – would be done. Because vendors can always advertise compliance with standards, they can still insist that their products be considered in any procurement contest, and even if those products do notionally support standards it does not mean that they will end up using them when deployed. For example, from the case of the Oslo municipality e-mail system, the councillor with responsibility for finance and development indicated the following:

“Oslo municipality is a complicated and comprehensive organisation and must take existing integration with specialist/bespoke systems into account. A procurement of other [non-Microsoft] end-user software will therefore result in unnecessary increases in costs for the municipality.”

In other words, even if existing software was acquired under the pretense that it supported standards, in deployment it may actually only function with other software using proprietary mechanisms, and the result of this is that newly-acquired software must also support these proprietary mechanisms. And so, a proprietary infrastructure grows, actively repelling components that employ open standards, with its custodians insisting that it is the fault of standards-compliant software that such an infrastructure would need to be dismantled almost in its entirety and replaced if even one standards-compliant component were to be admitted.

Who benefits the most from this? The vendor peddling the proprietary platforms and technologies that enable this morass of interdependency, of course. Make no mistake: any initial convenience promised by such a vendor fades away when the task of having to pursue an infrastructure strategy not dictated by outside interests is brought to bear on the purchaser. But such tasks are work, of course, and if there’s a way of avoiding it and insisting it doesn’t need attending to, a distraction can always be found.

And so, the horseplay continues under the excuse of “Standards!” when there is no real intent to uphold them or engage in the real work of maintaining a sustainable infrastructure that does not exclude open competition or channel public money to preferred vendors. Unlike the character in the comic strip whose code probably is still compiling, certain public sector institutions would have experienced a compilation error and be found out. It appears, unfortunately, that it is our job to peer around the cubicle partition and see what is happening on screen and perhaps to investigate the noises coming from the corridor. After all, our institutions don’t seem to be particularly concerned about doing so.