Simple R package for UN Comtrade API

uncomtrade-logo.pngI have recently been doing a lot of work in R, a statistical programming language, and have started accumulating a small collection of functions that I regularly use. One of the tools I have particularly come to rely on is this function to obtain data from the United Nation’s Comtrade database, a great source for country-to-country trade statistics. Since there seemed to be no R package that provides access to their new API (which is still in beta), I simply wrote a function for myself based on the sample code on their homepage and kept it with my personal files. Motivated by posts like this, however, I decided to put the code into an R package and share it here for others to use.

You can get the source file here: comtrade.tar.gz, MD5 sum: 22ce86b845d8a8ec7af98b0377c5f885

The package is identical to the sample code on the UN homepage and simply adds documentation. Note that they say that their API may change at any point, and since I have no plans to keep this updated (for now, at least), this may simply stop working in the future. Nevertheless, here is how to set up the package directly from within R:

download.file(url = "", destfile = "comtrade.tar.gz")

If the MD5 checksum corresponds to what you see above, go ahead and install the file and start pulling data from the UN using the get.Comtrade() function call:

install.packages("comtrade.tar.gz", repos = NULL, type = "source")
(s1 <- get.Comtrade(r = "842", p = "124,484"))

Gallery app for Android 5 Lollipop

gallery-aosp_100.pngIf you have recently updated your Android device to the latest Android version 5 “Lollipop“, you may have noticed that the “Gallery” app has been replaced by Google’s brand-new “Photos” app which depends on Google Plus. If you don’t like the idea of linking your photos to Google’s social network, or if you have disabled/uninstalled the Google Plus app altogether, there is a way to get back the familiar Gallery app without compiling it yourself by pulling it from a CyanogenMod image. Here is how.

  1. Go to and find the device corresponding most closely to your phone. This shouldn’t matter, really, since the .apk file should be the same everywhere. But it can’t hurt, either.
  2. On the device page, click the big green icon that reads “Release channel
  3. Download the latest “snapshot” version of the CyanogenMod image, or anything above that (i.e. release candidate or stable), to make sure that the file contains everything we need.
  4. Compare the MD5 checksum of the image file you downloaded against that given on the download page, to make sure your download is safe and has not been tampered with. On Linux and Mac OS X, you do this by running md5sum cm-*.zip from a terminal.
  5. Now open the image file in your favourite archive manager, navigate to /system/app/ and get the file “Gallery2.apk” from there. Transfer the .apk file to your phone.
  6. On your phone, use your favourite file manager to locate the .apk file you just transferred and click on it to start installation. Make sure you enable “Unknown Sources (Allow installation of apps from sources other than the Play Store)” from Settings -> Security first, and turn it off again when you’re done (it’s a good idea to keep it disabled most of the time).

Set up Fcitx for Chinese and Japanese language input on Ubuntu Trusty 14.04

fcitx.pngAfter recently upgrading to Ubuntu 14.04 Trusty Tahr (LTS), I decided to give Fcitx, the default input method framework on Ubuntu’s Chinese sister project Ubuntu Kylin, a try and I was pleasantly surprised. Not only is Fcitx rock-solid and actively developed, it also offers input methods for Japanese, Korean, Vietnamese and a bunch of other languages in addition to the default Chinese input methods. Here is how you get it to work on Ubuntu 14.04.

  1. Make sure you remove any IBus input methods you may have been using previously: Go to System Settings -> Text Entry, and remove all entries in the pane below “Input sources to use”, except the one representing your default keyboard layout.
    Also make sure to remove the keyboard shortcuts for “Switch to next source” and “Switch to previous source” by clicking inside the corresponding boxes on the right and hitting Backspace.
  2. Install the following packages via Software Center, Synaptic or sudo apt-get install:
    fcitx fcitx-pinyin fcitx-sunpinyin fcitx-googlepinyin fcitx-anthy fcitx-mozc
  3. Use a terminal (Ctrl + Alt + T) or a Run dialogue (Alt + F2) to run: im-config
    When asked if you want to explicitly update the user preferences, select “Yes”, then select fcitx as the default input method framework in the dialogue which pops up.
  4. Log off and back on again, or better yet, reboot.
  5. You should now see a keyboard icon in your system tray in the upper right corner of your screen. Click the icon and choose “Configure Current Input Method”, which should the Input Method Configuration. Clicking the plus “+” sign in the lower left corner of that window will open a small dialogue window. Uncheck the box “Only Show Current Language” and add the input methods you need (more on that below).
  6. Close the settings window, open your favourite text editor, bring up your desired input method via the Fcitx tray icon -> Input Method or the keyboard trigger shortcut: Ctrl + Space. You can also use Ctrl + Shift to move to the next input method.

There are a wide variety of options to configure the default behaviour of Fcitx and its looks. I like to preserve the input method state across windows, for example, which can be set under: Fcitx tray icon -> Configure Current Input Method -> Global Config.

Recommended Input Methods

sunpinyin.pngAs for input methods, I recommend Sun Pinyin for Chinese, and Mozc (which originates from Google Japanese Input) for Japanese. Both are excellent input methods that work very reliably in everyday use and are at least on par with what you get on Windows or Mac. Unfortunately the version of Sun Pinyin found in the Ubuntu 14.04 repositories suffers from a bug that prevents you from entering “yue” or “qiong”. To get around this, you have to update the package for “sunpinyin-data”, either by compiling it yourself or by getting a more recent version from the Debian repos. If you opt for the latter, run the following in your terminal:


Then compare the various checksums against the numbers you find on

md5sum sunpinyin-data_0.1.22+20131212-1_amd64.deb
openssl sha1 sunpinyin-data_0.1.22+20131212-1_amd64.deb
sha256sum sunpinyin-data_0.1.22+20131212-1_amd64.deb

If they match, go ahead and install the package via: sudo dpkg -i sunpinyin-data_0.1.22+20131212-1_amd64.deb


Finding local extrema in Matlab and Octave via fminbnd()

octave_logo-small.pngI recently had to work on a Matlab assignment that required the use of fminbnd() to find local extrema. As I typically work in Octave rather than Matlab I ran into some problems getting my code to work within both programmes. As it turned out, Matlab and Octave handle the function slightly differently, so I thought I’d share my findings to save others some headache.

First of all, some background information: fminbnd() is a Matlab/Octave function to find a local minimum of a continuous function within a specified interval (check out this excellent page for a simple example). By default, the function is used like this: [xval, yval, flag] = fminbnd(myfunction, startx, endx, options); This will search for local minima of the function myfunction within the interval of x = [startx, endx] and return the x-value of the minimum xval, the corresponding y-value at the minimum yval as well as an exit flag, which should be “1″ if there is a solution. Although fminbnd() is designed to search for minima, it can search for maxima by reversing the function parameter, i.e. -myfunction. It’s quite a handy function since it can greatly reduce the time required to determine the position of extrema when the rough location and type are already known.

The difficulty in getting fminbnd() to work on Octave stemmed from the fact that the function in question required a large number of parameters and the way both programmes handle this syntactically. To illustrate the problem, have a look at the following piece of code:

function y = poly2(x,a,b)
y = a*x.^2+b*x;

opts = optimset('Display','off');
[xval, yval, flag] = fminbnd('poly2',-2,2,opts,1,-2);

This apparently worked in Matlab, but didn’t in Octave. The basic idea was to have Matlab iterate over the first parameter x to find the local minimum, so in order to do that, the second and the third required parameters of poly2() are appended in the call to fminbnd(). Looking at the definition of fminbnd() given above, it seems natural enough that this would fail, and Octave does indeed give an error: error: Invalid call to fminbnd. Correct usage is: -- Function File: [X, FVAL, INFO, OUTPUT] = fminbnd (FUN, A, B, OPTIONS). The way to make this work in Octave is by using an anonymous function to define a wrapper which pre-defines the second and third parameter, so we get a new function that only requires one input parameter x.

poly2W = @(x) poly2(x,1,-2);
[xval, yval, flag] = fminbnd(poly2W,-2,2,opts);

This gives xval = 1.000000 yval = -1.000000 flag = 1. Defining poly2W() in this way allows Octave to complete the search and determine the point (1,-1) as the minimum of the original function poly2() for a = 1 and b = -2 within the interval [-2,2]. The nice thing about anonymous functions is that they inherit arguments from the enclosing scope, so in case your parameter values are determined dynamically within the computation, you can simply call these variables when defining your anonymous function. The only downside that remains is that fminbnd() appears to be considerably slower in Octave than in Matlab, but how much that matters depends on your project :-)


Block unauthorized OpenVPN logins using fail2ban

fail2ban.pngMonitoring a server can be a lot of work, but thankfully handy tools like fail2ban or logwatch make the task a lot easier. Fail2ban, for example, monitors the log files of services running on your system and blocks incoming connections when it detects a break-in attempt (using iptables or hosts.deny). These need to be defined using a regex filter, and while a great number of templates are already available for the most-used services (Apache, SSH, etc.), OpenVPN thus far has not been included. Setting this up isn’t too difficult, though.

Create a file openvpn.conf in /etc/fail2ban/filter.d/with the following content:

failregex = [a-b]*ovpn-server.*:.<HOST>:[0-9]{4,5} TLS Auth Error:.*
     [a-b]*ovpn-server.*:.<HOST>:[0-9]{4,5} VERIFY ERROR:.*
     [a-b]*ovpn-server.*:.<HOST>:[0-9]{4,5} TLS Error: TLS handshake failed.*

Set up a local configuration file for fail2ban by running cp -ivra /etc/fail2ban/jail.conf /etc/fail2ban/jail.local and open /etc/fail2ban/jail.local and add the following at the end of the file:

enabled = true
port = 1194
protocol = udp
filter = openvpn
logpath = /var/log/syslog
maxretry = 3

Finally, run /etc/init.d/fail2ban restart to restart fail2ban and make the changes take effect. Note that this set-up assumes that your OpenVPN server logs go to syslog. Also note that, in case you want to modify the filter rules, each failregex line must contain the <HOST> tag, otherwise even valid regex rules will not work, since fail2ban won’t know which address to block (use the fail2ban-regex tool to check if your detection rules are working: fail2ban-regex logfile.log /etc/fail2ban/filter.d/openvpn.conf).

You can set up fail2ban to email you each time there has been a break-in attempt by further editing the parameters in jail.local. Personally, however, I prefer a less intrusive solution based on logwatch. Logwatch is another programme that monitors log files on your system, but its job is to email you daily (or weekly, monthly, etc.) summaries of them. A simple way to make this set-up both convenient and secure is by setting up logwatch to monitor fail2ban logs, deliver summaries to a local inbox and run a little batch script via a cron-job to fetch these messages, encrypt them and send them to your actual email address.


You may also want to check out refiddle to help you with those regular expressions.