vanitasvitae's blog


Archive for June, 2017

GSoC Week 4.1

Wednesday, June 28th, 2017

This is a quick update post to my post from yesterday. The GSoC part came a little short because of my OMEMO reflections and because I only worked on one topic the last week, so this is intended as a kind of supplement.

Today I got my SOCKS5 code working again! I can send and receive both from and to local and external proxy servers. It was previously not working because I missread the XEP. I thought that one party only connects to a proxy after the activation element was received, while in fact the connection must be established beforehand. Also on the sender side I closed my OutputStream after writing the data on it. This caused the receiver to fail opening an InputStream because the socket was closed. It took some time and Googling to find out that closing the OutputStream also closes the socket. Unfortunately I’m not having a “established” implementation which I can test my code against. If you know of an app/messenger which supports Jingle FT 5, please let me know :)

My next steps will be writing integration tests. At some point I lost track of tests, so I’m a little bit behind on that front. Creating those tests should greatly improve the stability of the code and make breaking changes more obvious. Also I’ll create a little NIO test application to make myself familiar with the NIO framework. Currently my focus lays on getting things to work in the first place, but in the end it would be nice to have data streams implemented using NIO. Another thing that I’ll tackle this week is getting jingles fallback mechanism in case of a failed transport to work. This should be pretty straight forward.

Then again I’ll hopefully soon spent some more time again on my plans for Jingle Encrypted Transfers. I hope to get a working prototype after the general Jingle File Transfer is finished.

Thats all for now. Happy Hacking :)

Fourth week of GSoC and OMEMO thoughts

Wednesday, June 28th, 2017

GSoC

Evaluation phase is there! Time went by faster than I expected. That’s a good sign, I really enjoy working on Smack (besides when my code does not work :D ).

I spent this week to work on the next iteration of my Jingle code. IBB once again works, but SOCKS5 still misses a tiny bit, which I struggle to find. For some reason the sending thread hangs up and blocks just before sending begins. Its probably a minor issue, which can be fixed by changing one line of code, nevertheless I’m struggeling to find the solution.

Apart from that it turned out, that a bug with smack-omemo which I was earlier (unsuccessfully) trying to solve resurrected from the land of closed bug reports. Under very special conditions pubsub results seem to only arrive in Smack exactly after the result listener timed out. This keeps smack-omemo from fetching device bundles on some servers. I originally thought this was a configuration issue with my server, but it turned out that aTalk (a Jitsi for Android fork which is working on including OMEMO support) faces the exact same issue. Looks like I’ll have to investigate this issue once more.

OMEMO – Reflections

It appears that soon the council will decide on the future of OMEMO. I really hope, that the decision will make everyone happy and that the XMPP community (and – perhaps more important – the users) will benefit from the outcome.

There are multiple options for the direction, which the development of OMEMO can take. While everybody aggrees, that something should happen, it is not clear what.

OMEMO is already rather well deployed in the wild, so it is obviously not a good idea to break compatibility with existing implementations. A few months ago, OMEMO underwent a very minor protocol change to mitigate a vulnerability and even though Conversations (birthplace to OMEMO) made a smooth transition over multiple versions, things broke and confused users. While things like this probably cannot be avoided in a protocol which is alive and changing, it is desirable to avoid breaking stuff for users. Technology is made for users. Keeping them happy should be highest priority.

However, at the moment not every user can benefit from OMEMO since the current specification is based on libsignal, which is licensed under the GPLv3. This makes the integration in premissively licensed software either expensive (buying a license from OWS), or laborious (substitute libsignal with other double ratchet library). While the first option is probably unrealistic, the second option has been further investigated during discussions on the standards mailing list.

Already the “official” specification of OMEMO is based on the Olm library instead of libsignal. Olm more or less resembles OWSs double ratchet specification, which has been published by OWS roughly half a year ago. Both Olm and libsignal got audited, while libsignal got significantly more attention both in the media and among experts. One key difference between both libraries is, that libsignal uses the Extended Triple Diffie Hellman (X3DH) key exchange using the XEdDSA signature scheme. The later allows a signature key to be derived from an encryption key, which spares one key. In order to use this functionality, a special conversion algorithm is used. While apparently this algorithm is not too hard to implement, there is no permissive implementation available in any established, trusty crypto library.

In order to work around this issue, one proposal suggests to replace X3DH completely and switch to another form of key exchange. Personally I’m not sure, whether it is desirable to change a whole (audited) protocol in order to replace a single conversion algorithm. Given the huge echo of the Signal protocol in the media, it is only a matter of time until the conversion algorithm makes its way into approved libraries and frameworks. Software follows the principle of supply and demand and as we can conclude from the mailing list discussion, there is quite a lot of demand even alone in the world of XMPP.

I guess everybody aggrees that it is inconvenient, that the “official” OMEMO XEP does not represent, what is currently implemented in the real world (siacs OMEMO). It is open to debate now, how to continue from here on. One suggestion is to document the current state of siacs OMEMO in a historical XEP and continue development of the protocol in a new one. While starting from scratch offers a lot of new possibilities and allows for a quicker development, it also most definitely implies to completely break compatibility to siacs OMEMO at users expense. XMPP is nearly 20 years old now. I do not believe that we are in a hurry :) .

I think siacs OMEMO should not get frozen in time in favor of OMEMO-NEXT. Users are easily confused with names, so I fear that two competing but incompatible OMEMOs are definitely not the way to go. On the other hand, a new standard with a new name that serves the exact same use case is also a bad idea. OMEMO works. Why sould users switch? Instead I’d like to see a smooth transition to a more developer friendly, permissively implementable protocol with broad currency. Already there are OTR and OpenPGP as competitors. We don’t need even more segmentation (let alone the same name for different protocols). I propose to ditch the current official OMEMO XEP in favor of the siacs XEP and from there on go with Andreas’ suggestion, which allows both libsignal as well as Olm to be used simultaneously. This allows permissive implementations of OMEMO, drives development of a permissively licensed conversion algorithm and does not keep users standing in the rain.

To conclude my reflections: Users who use OMEMO will use OMEMO in two or three years. It is up to the community to decide, whether it will be frozen “historical” siacs OMEMO, or collectively developed, smoothly transitioned and unified OMEMO.

Thank you for your time :)

Third Week of GSoC

Tuesday, June 20th, 2017

Another week is has passed and the first evaluation phase slowly approaches. While I already fulfilled my goals (Jingle File Transfer using InBandBytestreams and SOCKS5Bytestreams), I still have a lot of work to do. The first working implementation I did is only so much – working. Barely. Now its time to learn from mistakes I made while I constructed my prototype and find better ways to do it in the next iteration. This is what I was up to in the past week and what will keep me from my usual sleep cycle for the coming week(s).

I spent the past week doing ground work and writing utility classes which will later allow me to send jingle actions in a clean way. The prototype implementation had all constructions of Jingle elements inside of the control flow, which made reading the code very hard. This will change in the next iteration.

While I worked on my implementation(s), I detected some errors in the XEPs involved and created pull requests against the xsf/xeps repository. In other spots I found some unclarities, but unfortunately my questions on the xsf chat were left unanswered. In some cases I found the solution myselves though.

Also I began upstreaming some changes and additions to the Smack repository. Parsers and elements of IBB have already been merged, as well as some more additions to the HashManager (XEP-0300) I created earlier, and some tests and fixes for the existing Jingle framework. Still open are my PR for SOCKS5 parsers and the first parts of the Jingle file transfer package.

I also dedicated a tiny little bit of my spare time to a non-GSoC project around a blog post on how to create an OMEMO capable chat client using Smack in less than 200 lines of code. The source code of the example application can be found in the FSFE’s brand new git repository. Unfortunately I also found a small bug in my OMEMO code that I have to fix sometime in the next weeks (nothing crucial, just some annoying faulty behavior).

I plan to spend the coming week working on my Jingle code, so that I have a mostly working framework when the evaluation phase begins.

Thats all for now. Happy Hacking :)

Tutorial: Home-made OMEMO client

Wednesday, June 14th, 2017

The german interior minister conference recently decided that the best way to fight terrorism is passing new laws that allow the government to demand access to communication from messengers like WhatsApp and co. Very important: Messengers like WhatsApp. Will even free software developers see requests to change their messengers to allow government access to communications in the future? If it comes so far, how are we then still possible to protect our communications?

The answer could be: Build your own messenger. I want to demonstrate, how simple it is to create a very basic messenger that allows you to send and receive end-to-end encrypted text messages via XMPP using Smack. We will use Smacks latest new feature – OMEMO support to create a very simple XMPP based command line chat application that uses state of the art encryption. I assume, that you all know, what XMPP is. If not, please read it up on Wikipedia. Smack is a java library that makes it easy to use XMPP in an application. OMEMO is basically the Signal protocol for XMPP.

So lets hop straight into it.
In my example, I import smack as a gradle dependency. That looks like this:

gradle.build

apply plugin: 'java'
apply plugin: 'idea'

repositories {
    mavenCentral()
    maven {
        url 'https://oss.sonatype.org/content/repositories/snapshots'
    }
}

ext {
    smackVersion="4.2.1-SNAPSHOT"
}

dependencies {
    compile "org.igniterealtime.smack:smack-java7:$smackVersion"
    compile "org.igniterealtime.smack:smack-omemo-signal:$smackVersion"
    compile "org.igniterealtime.smack:smack-resolver-dnsjava:$smackVersion"
    compile "org.igniterealtime.smack:smack-tcp:$smackVersion"
}

//Pack dependencies into the jar
jar {
    from(configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }) {
    exclude "META-INF/*.SF"
    exclude "META-INF/LICENSE"
    }
    manifest {
        attributes(
            'Main-Class': 'Messenger'
        )
    }
}

Now we can start the main function of our client. We need to create a connection to a server and log in to go online. Lets assume, that the user passes username and password as arguments to our main function. For sake of simplicity, we’ll not catch any errors like wrong number of parameters etc. Also we want to get notified of incoming chat messages and we want to send messages to others.

Messenger.java

public class Messenger {

    private AbstractXMPPConnection connection;
    private static Scanner scanner;

    public static void main(String[] args) throws Exception {
        String username = args[0];
        String password = args[1];
        Messenger messenger = new Messenger(username, password);

        scanner = new Scanner(System.in);
        while(true) {
            String input = scanner.nextLine();

            if (input.startsWith("/quit")) {
                break;
            }
            if (input.isEmpty()) {
                continue;
            }
            messenger.handleInput(input);
        }
    }

    public Messenger(String username, String password) throws Exception {
        connection = new XMPPTCPConnection(username, password);
        connection = connection.connect();
        connection.login();

        ChatManager.getInstanceFor(connection).addIncomingListener(
                (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message)
        );

        System.out.println("Logged in");
    }

    private void handleInput(String input) throws Exception {
        String[] split = input.split(" ");
        String command = split[0];

        switch (command) {
            case "/say":
                if (split.length > 3) {
                    String recipient = split[1];
                    EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient);

                    StringBuilder message = new StringBuilder();
                    for (int i=2; i<split.length; i++) message.append(split[i]);

                    ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(message);
                }
                break;
        }
    }
}

If we now compile this code and execute it using credentials of an existing account, we can already log in and start chatting with others using the /say command (eg. /say bob@marley.jm Hi Bob!). But our communications are unencrypted right now (aside from tls transport encryption). Lets change that next. We want to use OMEMO encryption to secure our messages, so we utilize Smacks new OmemoManager which handles OMEMO encryption. For that purpose, we need a new private variable which will hold our OmemoManager. Also we make some changes to the constructor.

Messenger.java

private OmemoManager omemoManager;

public Messenger(String username, String password) throws Exception {
    connection = new XMPPTCPConnection(username, password);
    connection = connection.connect();
    connection.login();

    //additions begin here
    SignalOmemoService.acknowledgeLicense();
    SignalOmemoService.setup();
    //path where keys get stored
    OmemoConfiguration.setFileBasedOmemoStoreDefaultPath(new File("path"));
    omemoManager = OmemoManager.getInstanceFor(connection);

    //Listener for incoming OMEMO messages
    omemoManager.addOmemoMessageListener(new OmemoMessageListener() {
        @Override
        public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage,
                        Message wrappingMessage, OmemoMessageInformation omemoInformation) {
            System.out.println("(O) " + encryptedMessage.getFrom() + ": " + decryptedBody);
        }

        @Override
        public void onOmemoKeyTransportReceived(CipherAndAuthTag cipherAndAuthTag, Message message,
                        Message wrappingMessage, OmemoMessageInformation omemoInformation) {
            //Not needed
        }
    });

    ChatManager.getInstanceFor(connection).addIncomingListener(
            (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message)
    );
    omemoManager.initialize();
    //additions end here.
    System.out.println("Logged in");
}

Also we must add two new commands that are needed to control OMEMO. /omemo is similar to /say, but will encrypt the message via OMEMO. /trust is used to trust an identity. Before you can send a message, you have to decide, whether you want to trust or distrust an identity. When you call the trust command, the client will present you with a fingerprint which you have to compare with your chat patner. Only if the fingerprint matches, you should trust it. We add the following two cases to the handleInput’s switch case environment:

Messenger.java

case "/omemo":
    if (split.length > 2) {
        String recipient = split[1];
        EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient);

        StringBuilder message = new StringBuilder();
        for (int i=2; i<split.length; i++) message.append(split[i]);

        //encrypt
        Message encrypted = null;
        try {
            encrypted = OmemoManager.getInstanceFor(connection).encrypt(recipientJid, message.toString());
        }
        // In case of undecided devices
        catch (UndecidedOmemoIdentityException e) {
            System.out.println("Undecided Identities: ");
            for (OmemoDevice device : e.getUntrustedDevices()) {
                System.out.println(device);
            }
        }
        //In case we cannot establish session with some devices
        catch (CannotEstablishOmemoSessionException e) {
            encrypted = omemoManager.encryptForExistingSessions(e, message.toString());
        }

        //send
        if (encrypted != null) {
            ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(encrypted);
        }
    }
    break;

case "/trust":
    if (split.length == 2) {
        BareJid contact = JidCreate.bareFrom(split[1]);
        HashMap<OmemoDevice, OmemoFingerprint> fingerprints =
                omemoManager.getActiveFingerprints(contact);

        //Let user decide
        for (OmemoDevice d : fingerprints.keySet()) {
            System.out.println("Trust (1), or distrust (2)?");
            System.out.println(OmemoKeyUtil.prettyFingerprint(fingerprints.get(d)));
            int decision = Integer.parseInt(scanner.nextLine());
            if (decision == 1) {
               omemoManager.trustOmemoIdentity(d, fingerprints.get(d));
            } else {
                omemoManager.distrustOmemoIdentity(d, fingerprints.get(d));
            }
        }
    }
    break;

Now we can trust contact OMEMO identities using /trust bob@marley.jm and send them encrypted messages using /omemo bob@marley.jm Hi Bob!. When we receive OMEMO messages, they are indicated by a “(O)” in front of the sender.
If we want to go really fancy, we can let our messenger display, whether received messages are encrypted using a trusted key. Unfortunately, there is no convenience method for this available yet, so we have to do a small dirty workaround. We modify the onOmemoMessageReceived method of the OmemoMessageListener like this:

Messenger.java

@Override
public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage,
            Message wrappingMessage, OmemoMessageInformation omemoInformation) {
    //Get identityKey of sender
    IdentityKey senderKey = (IdentityKey) omemoInformation.getSenderIdentityKey().getIdentityKey();
    OmemoService<?,IdentityKey,?,?,?,?,?,?,?> service = (OmemoService<?,IdentityKey,?,?,?,?,?,?,?>) OmemoService.getInstance();

    //get the fingerprint of the key
    OmemoFingerprint fingerprint = service.getOmemoStoreBackend().keyUtil().getFingerprint(senderKey);
    //Lookup trust status
    boolean trusted = omemoManager.isTrustedOmemoIdentity(omemoInformation.getSenderDevice(), fingerprint);

    System.out.println("(O) " + (trusted ? "T" : "D") + " " + encryptedMessage.getFrom() + ": " + decryptedBody);
}

Now when we receive a message from a trusted identity, there will be a “T” before the message, otherwise there is a “D”.
I hope I could give a brief introduction on how to use Smacks OMEMO support. You now have a basic chat client, that is capable of exchanging multi-end-to-multi-end encrypted messages with other XMPP clients that support OMEMO. All took less than 200 lines of code! Now its up to you to add additional features like support for message carbons, offline messages and co. Spoiler: Its not hard at all :)
You can find the source code of this tutorial in the FSFE’s git repository.

When the government is unable or simply not willing to preserve your privacy, you’ll have to do it yourself.

Happy Hacking :)

GSoC – Second week of coding

Wednesday, June 14th, 2017

The second week of GSoC is over! My Jingle implementation progresses.

Most of my efforts went into designing the state machine behind the Jingle and Jingle File Transfer protocol. Because I never really worked with asynchronous communication, let alone network code before, it takes some time to get my head around that.

I’m heavily utilizing the water fall development model – I code until I get stuck at some point I did not consider at all, then I create a new class and start over again. This is very tideous, but I make slow progress towards working Jingle Socks5 Bytestream transports!

All in all I predict, that it’ll take its time to fully complete the Jingle implementation so that it covers every corner case.

Introducing JET!

While working on my Jingle code, I also started writing down my plans for Jingle Encrypted Transfers (jet). My goal is to keep that specification as simple as possible while providing a reasonable way to exchange encrypted data. I decided, that hiding metadata is not in the scope of this document for now, but can later be specified in a seperate document. Contributions and thoughts regarding encrypted Jingle file transfer are welcome :)

Happy Hacking!

Smack v4.2 Introduces OMEMO Support!

Tuesday, June 6th, 2017

This blogpost doubles as a GSoC update, as well as a version release blog post.

OMEMO Clownfish logo.

OMEMO Clownfish logo (conversations.im)

I have the honour to announce the latest release of Smack! Version 4.2 brings among bug fixes and additional features like Explicit Message Encryption (XEP-0380) and Message Processing Hints (XEP-0334) support for OMEMO Multi-End-Message-and-Object encryption (XEP-0384). OMEMO was developed by Andreas Straub for the Conversations messenger (also as a Google Summer of Code project) in 2015. Since then it got quite popular and drew a lot of attention for XMPP in the media. My hope is that my efforts to develop an easy to use Smack module will result in an even broader adoption.

OMEMO is a protocol for multi-end to multi-end encrypted communication, which utilizes the so called Double Ratchet algorithm. It fulfills amongst the basic requirements of encrypted communication (confidentiality, authenticity and integrity) also the properties of deniability and forward secrecy as well as future secrecy. Smacks implementation brings support for encrypted single and group chats including identity management and session renegotiation.

Current implementations (as well as this one) are based upon the libsignal library developed by OpenWhisperSystems for their popular Signal (formerly TextSecure) messenger. Smacks OMEMO support is structured in two modules. There is smack-omemo (APL licensed), which contains the logic specified in the XEP, as well as some basic cryptographic code. The other module smack-omemo-signal (GPLv3 licensed) implements some abstract methods defined by smack-omemo and encapsulates all function calls to libsignal.

Currently smack-omemo-signal is the only module available that implements the double ratchet functionality, but there has been a lot of discussion on the XMPP Standards Foundations mailing list regarding the use of alternative (more permissively licensed) libraries for OMEMO (like for example Olm, a double ratchet implementation from our friends over at the [matrix] project). So once there is a new specification that enables the use of other libraries, it should be pretty easy to write another module for smack-omemo enabling OMEMO support for clients that are not GPLv3 compatible as well.

Smack’s OMEMO modules are my first bigger contribution to a free software project and started as part of my bachelors thesis. I’m quite happy with the outcome :)

Smack Logo

Also Smack has a new Logo!

That was a lot of talking about OMEMO. Now comes the second functioning of this blog post, my GSoC update.

My project of implementing Jingle File Transfer (XEP-0234) for Smack is going relatively well. I’m stuck at some points where there are ambiguities in the XEP or things I don’t know yet, but most of the time I find another construction site where I can continue my work. Currently I’m implementing stanza providers and elements needed for file transfer. Along the way I steadily create Junit tests to keep the code coverage at a high level. Already it pays off when there are fiddly changes in the element structure.

It’s a real pleasure to learn all the tools I never used before like code coverage reports or mocking and I think Flow does a good job introducing me to them one by one.

That’s all for now. Happy hacking :)