The future of quality and testing in ownCloud

At the last ownCloud sprint in Berlin we ran a workgroup on QA and testing. In my capacity as an ownCloud systems developer, myself and four others spent a day working on how to improve the quality and stability of ownCloud in future, with a particular focus on tools and automation of checks and reviews.

The result was a set of proposals which I presented to sprint attendees on the last evening. Only 24 hours after we had come up with these ideas, some had already been implemented, and by the end of the sprint, one of the most important ones — a peer-review based work-flow — had been agreed and adopted. This is testament to the benefits of centralised sprints – with so many important people in the same room we were able to develop ideas, share them, and even execute some of them in a single weekend!

Below are the original proposals made at the sprint. Look out for status indicators like [DONE] and [IN PROGRESS]. Please note that these are just proposals, and not an agreed roadmap. Some of the ideas are good and have been implemented, but others may not be so good, and may not be implemented.

Share your own views either in the comments or on the ownCloud public mailing list.

Proposals

Issue tracker management

  • We can Improve communication of where to report bugs
    • Add a link to the GitHub issue tracker in main nav-bar and/or at top of developer homepage nav
    • Remove “bug reports” and “feature requests” section of the forum [IN PROGRESS]
    • Identify and consolidate contact platforms and support them equally well, or remove
  • We can make reported bugs more useful
    • Define labels on github issues [DONE]
    • Use issue milestones
    • Publish who maintains which apps/features (see New feature introduction)
    • Formalise bug triaging process – consider appointment of an official bug quality controller

New feature introduction

  • App / feature mini-specifications
    • Adopt convention of writing explanations of key features / goals of apps
    • Contain a few paragraphs or bullet points only
    • Include them in info.xml using <spec> tag?
    • This way other developers can spot high-level opportunities and issues early on
  • Implement a code review system
    • Short term: use GitHub’s pull request system for all commits to ‘Stable’ and ‘Master’ branches, perform code reviews before changes are merged, and integrate the ownCloud CI server into the pull request workflow [DONE]
    • Long term: consider using a stand-alone web-based code review system like Gerrit
    • Potential benefits:
      • Ensure all changes are checked by humans before
        commit (chain of responsibility)
      • ‘Stable’ branch doesn’t get broken
      • Integrated peer review
      • Integrated automated test suites
      • More open governance
      • A cleaner git history (easier identification of
        issue origin)
      • Easy extendability of checks (security)
      • Automated code quality checks
      • Specifically, by integrating ownCloud’s existing JenkinsCI server into the process we can:
        • Ensure all changes pass existing automated tests before commit (PHPUnit, Cucumber, others)
        • Define additional tests for common issues (code sniffers, correct use of API etc.)
        • Check for security problems (e.g. error_log(), file upload, echo in templates)
        • Give contributors automated feedback before another human has to review it (simple errors are automatically spotted and notifications sent)

Isolation of functionality and apps

  • Long term goal:
    • Move towards object-oriented code, away from static methods (see coding standards)
  • Short term goal:
    • Make internals more independently testable by decoupling static dependencies (dependency injection) [DONE]
    • Abstract away static method calls one level to allow the use of alternatives in unit tests (mocking objects )
    • Instead of OC_User::getUser('foo'); use $api->OC_User->getUser('foo');
    • Potential benefits:
      • Enables isolation of functionality under test
      • Doesn’t affect existing code (usage is voluntary)
      • Supports multiple versions of internal ownCloud API (provides a compatibility layer between apps etc.)
      • Very simple to use (just add $api-> before existing class names)

Bugfixing workflow

  • Proposed bug-fixing procedure (for PHP):
    • Reproduce the bug
    • Write a unit test which isolates the behaviour
    • Check the test fails and confirms the issue
    • Fix the bug
    • Check the unit test now passes
    • Mark bug fixed

Release procedure, security announcements

  • Promote the security list, push public to use it instead of bugtracker via website links
  • Ensure release procedure includes checking CI server test status
  • Expand existing unit tests (see Isolation of functionality and apps & New feature introduction)
  • Ensure all necessary apps are enabled on CI server for tests leading up to new release

Coding standards (link)

  • Add code-sniffers to CI to identify problematic code (see New feature introduction)
  • Add recommendation for object oriented code (see Isolation of functionality and apps)
  • Add requirement of mini-spec files (see New feature introduction)
  • Add requirement of DocBlock comments for all classes and methods
  • Disallow use of dangerous functions in templates [DONE]
  • Add policy on word order of “public static function”

Parliamentary candidates challenged to digital debate in Manchester

19.00, 7th Nov at the Dancehouse Theatre, Manchester, UK – Free Entry

Manchester Central candidates will be challenged to explain how they will defend citizen’s privacy and free speech online.

Recent arrests for “offensive tweets” and proposals to put the whole UK population under Internet surveillance and data collection have thrown digital issues into the forefront of political debate.

Candidates will be asked how they will protect Manchester’s voters from dangerous Internet policies, and also ensure that the local digital economy is supported.

Newly formed Manchester Open Rights Group has teamed up with NUJ New Media, No2ID Manchester and Free Software Foundation Europe to run the event which is being widely promoted.

Presentations will also be followed by a Q and A and debate giving the public a chance to quiz candidates on issues effecting one of Manchester’s largest and fastest growing industries.

As well as the digital economy, we invite comments and questions on the use of the Internet for social change and views on the proposed increase in powers of surveillance.

Free Tickets are available from http://www.thedancehouse.co.uk

There is a booking cost if you book online – But you can get a free reservation by ringing the Dancehouse Box Office on 0161 2379753

Speakers

  • Tom Dylan – Green Party
  • Marc Ramsbottom- Liberal Democrats
  • Loz Kaye – Pirate Party
  • Lucy Powell (invited) – Labour
  • Matthew Sephton (invited) – Conservatives

With support from

  • Peter Bradwell – Open Rights Group
  • Martin Bryant – Managing Editor, The Next Web
  • John Robb – Louder than War

Supporting organisations

  • National Union of Journalists
  • New Media Industrial Council
  • Free Software Foundation Europe
  • Manchester Open Rights Group
  • Manchester No2ID.

Press contacts: Mick Chesterman 0044 7913 882193, Tom Chiverton 0044 7967 672404

Install Google Dart language on Fedora 17

The JavaScript related Dart language has a compiler and IDE provided by Google. The project developers work on Ubuntu, and so in order to get these tools working on Fedora we have to make a couple of preparations. These instructions are designed for Fedora 17 64-bit, and were tested using Dart version 14167.

Dart is non-copyleft Free Software using a BSD license.

  1. Install the required packages:
    yum install subversion pkgconfig python perl gcc-c++ bison \
    flex gperf nss-devel nspr-devel gtk2-devel glib2-devel freetype-devel \
    atk-devel pango-devel cairo-devel fontconfig-devel GConf2-devel \
    dbus-devel alsa-lib-devel libX11-devel expat-devel bzip2-devel \
    dbus-glib-devel elfutils-libelf-devel libjpeg-devel \
    mesa-libGLU-devel libXScrnSaver-devel \
    libgnome-keyring-devel cups-devel libXtst-devel libXt-devel pam-devel
  2. Link your bzip2 library so that Dart can find it:
    ln -s /lib/libbz2.so /lib/libbz2.so.1.0
  3. Download and extract Dart from http://www.dartlang.org/
  4. Open a terminal and cd into your Dart directory, e.g.:
    cd ~/Downloads/Dart
  5. Run Dart:
    ./DartEditor
  6. Open one of the included demo projects, open a .dart file, click ‘Run’ (green ‘play’ button in toolbar), and see Dartium open the freshly compiled project.

Install Android Emulator in Fedora 17

Android emulator allows you to test apps and websites from an Android user’s perspective, without the need for a physical Android device. The Emulator does not work “out of the box” in Fedora however, and only a version for 32-bit machines is provided. Additional packages are required, and the GUI must be started from the command line. These steps are designed for 64-bit machines, but they should also work on 32-bit ones as well.
android emulator screenshot

  1. Install the additional packages which the emulator depends upon:
    yum install libstdc++.i686 glibc.i686 ncurses-libs.i686 libstdc libstdc++.i686 libzip.i686 libX11.i686 libXrandr.i686 SDL.i686
  2. Download the Linux edition of the emulator from Google: http://developer.android.com/sdk/index.html
  3. Extract the downloaded archive (open your Downloads folder right click on the file, “extract here”)
  4. Open a terminal, change directory to the folder you have just extracted:
    cd ~/Downloads/android-sdk-linux/
  5. Run the emulator:
    tools/android
  6. Once the repository has automatically updated itself, download the packages that have been automatically selected for you (click “install packages”)
  7. Open the Virtual Device manage window from the menu: “tools” > “Manage AVDs”
  8. Click “new” to create a new AVD, use default settings
  9. Highlight your new device and click “start”, then “launch”

The emulator should now boot up after a few seconds and allow you to use the device.

Install Webgen in Fedora 17 with Ruby 1.8.6 and RVM

Webgen does not work with new versions of Ruby. Using Webgen installed using ruby and rubygems from Fedora repos results in errors including “obsolete and deprecated Config”.

To get webgen to work, we will use RVM (Ruby Version Manager). This allows you install multiple versions of ruby side by side. RVM is not available in Fedora repositories, and must be manually installed.

  1. If you have already installed ruby from a repository, remove it:

    yum remove ruby ruby-libs rubygems

  2. As root, install RVM, and follow the on-screen instructions:

    \curl -L https://get.rvm.io | bash -s stable --ruby
    Note: the leading backslash (\) is intentional and could be important

  3. Install Ruby 1.8.6:

    rvm install 1.8.6
    Note: you can see what versions are available to install using: rvm list known

  4. Switch to use Ruby 1.8.6 by default:

    rvm use 1.8.6
    Note: You can check that it worked using: ruby -v
    Note: If you want this to be permanent, for all shells, and not just the terminal you’re currently using, execute: rvm use 1.8.6 –default

  5. Install webgen:

    gem install webgen

  6. That’s it! Good luck.

Results: Free Software voice & video testing

Last weekend on Software Freedom Day the Manchester FSFE Fellowship group, assisted by additional participants in Britain and Germany, spent the afternoon testing Free Software alternatives to Skype.

Results

The 25 sets of results were recorded, and can be browsed, sorted, and searched on the news article edition of this post.

Six audio tests successfully passed (24%), as did five video tests (20%). Mumble was the most successful client, passing 100% of tests (audio only, video is not yet supported). XMPP passed four out of 14 audio tests, whereas SIP passed only one out of ten (both video and audio). Of nine apps tested, only Mumble, Pidgin, Jitsi, and Google Talk’s web client achieved passes.

results table
Chat testing results table

Conclusions

The clients tested performed more poorly than expected, probably due to network problems. One of the difficulties in testing was that generally there was little or no information about why the test had failed.

SIP clients couldn’t connect successfully except when both testers used the same client, and had accounts on the same SIP server. This was surprising, especially considering that the accounts used for testing were is many cases paid for and commercially supported.

The only client and protocol which consistently did what it promised was Mumble, which had 100% test pass rate. Unlike all other clients, Mumble uses its own protocol, and also offers audio conferencing and text-to-speech by default. Mumble users are constrained to using the same server however, unlike SIP and XMPP users who should theoretically each be able to use a separate server of their choice.

It would have been useful to have a local SIP and XMPP server on the same network as the testers in order to better identify network related problems. This could have helped determine whether failures stemmed from the client, network, or server..

Examining STUN and ICE configurations was beyond the scope of our tests, but as these technologies seem critical to whether calls succeed or fail, they merit careful examination when choosing or configuring SIP and XMPP servers.

FSFE Fellows will be pleased to note that XMPP calls using FSFE’s own XMPP server were often successful: several volunteers could see and hear each other when both were using their @jabber.fsfe.org accounts.

A Skype alternative?

Skype uses a variety of notorious methods to punch through network obstacles like firewalls, and to ensure it can locate the intended call recipient wherever they may be. It also benefits from being a centralised system where, if necessary, all roads lead to Rome (or a Skype managed server, at least).

Contrastingly, Free Software systems play by conventional rules of network traffic, and try and connect callers using established procedures. They also have to communicate with a variety of different servers, which may operate in ways they don’t know or expect. In addition to this, some of the chat clients that we tested have only recently added support for more than text-based chatting (Gajim in Debian testing is too old for video, for example), and probably add a few problems of their own into the mix.

Based on our test results, those looking for a Free Software Skype alternative should use either Mumble, or SIP with all callers on the same server and using the same client.

Moving Forward

We had hoped to publish a conclusive compatibility table of chat clients, providing a reference for people needing to know “what works with what”, and “what is easiest”. Instead, we have table of subjective results, and a list of questions:

  • How can we determine whether an XMPP server supports audio and video (‘Jingle‘)?
  • How can we gather more detailed information about why a call failed?
  • How can we determine whether a SIP or XMPP server supports STUN (and should therefore be less prone to network issues)?

If you have answers to these questions, please share them in the comments!

Install Rar in Fedora 17: make rar archive files

Plenty of resources explain how to install unrar support in Fedora, but what about when you need to create rar files, instead of merely extracting them?

Rar has licensing issues which is why it isn’t available in standard Fedora repositories. The package ‘Unrar’ is available in rpmfusion repositories, but to get rar compression support you can install one of these two non-Free packages:

rar-3.8.0-1.el6.rf.i686.rpm (32 bit)
rar-3.8.0-1.el6.rf.x86_64.rpm (64 bit)

Ultra-secure social networking promised by Secure Share

Secure Share is a very interesting project that aims to replace existing social networks with a far, far more secure platform upon which developers can build new distributes apps and services. Think of it a bit like a new version of Bit Torrent with an easy API for sending messages, relationships, streams, and status updates, and a prototype desktop client for bringing these together in a social networking interface.

The Secure Share team seem to be correct in their core beliefs that Federated networking is part of the privacy problem rather than the solution, and that the web browsers are incapable of delivering secure communication. They certainly appear to know what they’re talking about, and the amount of research and planning evident in the many articles on the website is impressive. The project lead, Carlo Von Lynx, has promising talents, with a background in security and scalability (compare this to the team that founded diaspora).

Code already exists, mainly for the back end libraries and daemons. There is a user interface component, but it hasn’t been worked on since April, and it’s intended to be a desktop social network app, which may be impractical for most users as it requires local installation.

All in all the project feels more like a research project than a software project at this stage. Presumably their recent requests for  funding is intended to ease the transition to usable software.

I’m very sceptical about the requirement for Secure Share users to install something on their local machine. In a time when Google is determined to turn all consumer computers into thin clients, and the web browser is becoming “the only app you need”, it seems unlikely that the future social networking app used by “all humanity” will run from a binary downloaded from secureshare.org.

Secure Share developers also don’t seem certain how it would be possible for their users to access their accounts on more than one machine. Being able to login from university, or on a friend’s laptop, seems like a critical requirement however. There’s some vague talk of using Secure Share on smartphones, but if this is to become a reality then surely it should a priority right from the start. Making smartphones a primary target platform seems like a no-brainer now that they outsell traditional personal computers.

It seems to me that Secure Share is a long term project with a lot of intellectual value, but comparatively little practical potential, unless it receives a lot of interest and funding. Diaspora had $200,000, and what they achieved was approximately 5% of what Secure Share have in mind. That said, Carlo is a heck of a lot more capable than the Diaspora developers, and he’s already written a significant chunk of the Secure Share back end. Secure Share is also a second-generation project, building upon successes of the Psyc project, showing that those involved know how to deliver production quality applications.

I have high hopes for Secure Share. If it becomes user-friendly and feature-rich enough, perhaps it could lure developers into taking it to new platforms, and users into jumping ship from Diaspora, and maybe even Facebook. I wish the team the best of luck.

Gtimelog: editing logs

Gtimelog is very simple. It provides no method of editing the log of how your time was spent.

Fortunately the program’s simplicity extends to its system files, which encourage easy manual editing, without the need for a separate interface for doing so. The file format of log files benefits from some explanation however.

In Gtimelog click File -> Edit timelog.txt to view the log for the current day. Each task has only one time associated with it, and that is the time that the task ended. The time that any given task started is taken from the end of the preceeding task.

2012-06-15 09:00: Start
2012-06-15 12:00: Eating mailing list spam
2012-06-15 13:00: **Lunch

In the above timelog.txt, the eating of spam commenced at 09.00, and ended at 12.00.

View web pages on a local web server from a virtual machine on Fedora 17

These instructions will allow you to view web pages that are served by your local web development server from within a virtual machine.

Example scenario: you run a local web server on Fedora for development, and you want to test your local site pages from within another operating system which is running in a virtual machine, e.g. to test compatibility with Internet Explorer 9. The below method saves you from having to upload your development files somewhere remote – you can keep all your development local and also try out your pages on all platforms.

1. Install VirtualBox from https://www.virtualbox.org/. Download the .rpm and install it via GUI or commandline

Note: Using either Virtual Machine Manager, or VirtualBox from the rpmfusion repositories does not work.

2. Install and boot your desired OS in VirtualBox (I’ll assume that this is Windows)

3. Edit and save the hosts file of the guest OS to include entries for each of the local websites that you wish to access from the virtual machine. The site names should be identical to the ones in your Fedora /etc/hosts file:

10.0.2.2 local.mysite example.devsite

Note: on Windows saving the hosts file takes effect immediately. On Windows the file is located at c:\windows\system32\drivers\etc\hosts

Note: Windows 7 requires an additional step to edit the hosts file

4. Start Apache on your host machine, if it isn’t already running

5. On the virtual machine visit example.devsite in a web browser

UPDATE: If you encounter problems when starting your virtual machine relating to kernel modules, try the following:

yum install gcc kernel-devel kernel-headers kernel-PAE-devel

/etc/init.d/vboxdrv setup