vanitasvitae's blog


Archive for the ‘xmpp’ Category

GSoC Week 4.1

Wednesday, June 28th, 2017

This is a quick update post to my post from yesterday. The GSoC part came a little short because of my OMEMO reflections and because I only worked on one topic the last week, so this is intended as a kind of supplement.

Today I got my SOCKS5 code working again! I can send and receive both from and to local and external proxy servers. It was previously not working because I missread the XEP. I thought that one party only connects to a proxy after the activation element was received, while in fact the connection must be established beforehand. Also on the sender side I closed my OutputStream after writing the data on it. This caused the receiver to fail opening an InputStream because the socket was closed. It took some time and Googling to find out that closing the OutputStream also closes the socket. Unfortunately I’m not having a “established” implementation which I can test my code against. If you know of an app/messenger which supports Jingle FT 5, please let me know :)

My next steps will be writing integration tests. At some point I lost track of tests, so I’m a little bit behind on that front. Creating those tests should greatly improve the stability of the code and make breaking changes more obvious. Also I’ll create a little NIO test application to make myself familiar with the NIO framework. Currently my focus lays on getting things to work in the first place, but in the end it would be nice to have data streams implemented using NIO. Another thing that I’ll tackle this week is getting jingles fallback mechanism in case of a failed transport to work. This should be pretty straight forward.

Then again I’ll hopefully soon spent some more time again on my plans for Jingle Encrypted Transfers. I hope to get a working prototype after the general Jingle File Transfer is finished.

Thats all for now. Happy Hacking :)

Fourth week of GSoC and OMEMO thoughts

Wednesday, June 28th, 2017

GSoC

Evaluation phase is there! Time went by faster than I expected. That’s a good sign, I really enjoy working on Smack (besides when my code does not work :D ).

I spent this week to work on the next iteration of my Jingle code. IBB once again works, but SOCKS5 still misses a tiny bit, which I struggle to find. For some reason the sending thread hangs up and blocks just before sending begins. Its probably a minor issue, which can be fixed by changing one line of code, nevertheless I’m struggeling to find the solution.

Apart from that it turned out, that a bug with smack-omemo which I was earlier (unsuccessfully) trying to solve resurrected from the land of closed bug reports. Under very special conditions pubsub results seem to only arrive in Smack exactly after the result listener timed out. This keeps smack-omemo from fetching device bundles on some servers. I originally thought this was a configuration issue with my server, but it turned out that aTalk (a Jitsi for Android fork which is working on including OMEMO support) faces the exact same issue. Looks like I’ll have to investigate this issue once more.

OMEMO – Reflections

It appears that soon the council will decide on the future of OMEMO. I really hope, that the decision will make everyone happy and that the XMPP community (and – perhaps more important – the users) will benefit from the outcome.

There are multiple options for the direction, which the development of OMEMO can take. While everybody aggrees, that something should happen, it is not clear what.

OMEMO is already rather well deployed in the wild, so it is obviously not a good idea to break compatibility with existing implementations. A few months ago, OMEMO underwent a very minor protocol change to mitigate a vulnerability and even though Conversations (birthplace to OMEMO) made a smooth transition over multiple versions, things broke and confused users. While things like this probably cannot be avoided in a protocol which is alive and changing, it is desirable to avoid breaking stuff for users. Technology is made for users. Keeping them happy should be highest priority.

However, at the moment not every user can benefit from OMEMO since the current specification is based on libsignal, which is licensed under the GPLv3. This makes the integration in premissively licensed software either expensive (buying a license from OWS), or laborious (substitute libsignal with other double ratchet library). While the first option is probably unrealistic, the second option has been further investigated during discussions on the standards mailing list.

Already the “official” specification of OMEMO is based on the Olm library instead of libsignal. Olm more or less resembles OWSs double ratchet specification, which has been published by OWS roughly half a year ago. Both Olm and libsignal got audited, while libsignal got significantly more attention both in the media and among experts. One key difference between both libraries is, that libsignal uses the Extended Triple Diffie Hellman (X3DH) key exchange using the XEdDSA signature scheme. The later allows a signature key to be derived from an encryption key, which spares one key. In order to use this functionality, a special conversion algorithm is used. While apparently this algorithm is not too hard to implement, there is no permissive implementation available in any established, trusty crypto library.

In order to work around this issue, one proposal suggests to replace X3DH completely and switch to another form of key exchange. Personally I’m not sure, whether it is desirable to change a whole (audited) protocol in order to replace a single conversion algorithm. Given the huge echo of the Signal protocol in the media, it is only a matter of time until the conversion algorithm makes its way into approved libraries and frameworks. Software follows the principle of supply and demand and as we can conclude from the mailing list discussion, there is quite a lot of demand even alone in the world of XMPP.

I guess everybody aggrees that it is inconvenient, that the “official” OMEMO XEP does not represent, what is currently implemented in the real world (siacs OMEMO). It is open to debate now, how to continue from here on. One suggestion is to document the current state of siacs OMEMO in a historical XEP and continue development of the protocol in a new one. While starting from scratch offers a lot of new possibilities and allows for a quicker development, it also most definitely implies to completely break compatibility to siacs OMEMO at users expense. XMPP is nearly 20 years old now. I do not believe that we are in a hurry :) .

I think siacs OMEMO should not get frozen in time in favor of OMEMO-NEXT. Users are easily confused with names, so I fear that two competing but incompatible OMEMOs are definitely not the way to go. On the other hand, a new standard with a new name that serves the exact same use case is also a bad idea. OMEMO works. Why sould users switch? Instead I’d like to see a smooth transition to a more developer friendly, permissively implementable protocol with broad currency. Already there are OTR and OpenPGP as competitors. We don’t need even more segmentation (let alone the same name for different protocols). I propose to ditch the current official OMEMO XEP in favor of the siacs XEP and from there on go with Andreas’ suggestion, which allows both libsignal as well as Olm to be used simultaneously. This allows permissive implementations of OMEMO, drives development of a permissively licensed conversion algorithm and does not keep users standing in the rain.

To conclude my reflections: Users who use OMEMO will use OMEMO in two or three years. It is up to the community to decide, whether it will be frozen “historical” siacs OMEMO, or collectively developed, smoothly transitioned and unified OMEMO.

Thank you for your time :)

Third Week of GSoC

Tuesday, June 20th, 2017

Another week is has passed and the first evaluation phase slowly approaches. While I already fulfilled my goals (Jingle File Transfer using InBandBytestreams and SOCKS5Bytestreams), I still have a lot of work to do. The first working implementation I did is only so much – working. Barely. Now its time to learn from mistakes I made while I constructed my prototype and find better ways to do it in the next iteration. This is what I was up to in the past week and what will keep me from my usual sleep cycle for the coming week(s).

I spent the past week doing ground work and writing utility classes which will later allow me to send jingle actions in a clean way. The prototype implementation had all constructions of Jingle elements inside of the control flow, which made reading the code very hard. This will change in the next iteration.

While I worked on my implementation(s), I detected some errors in the XEPs involved and created pull requests against the xsf/xeps repository. In other spots I found some unclarities, but unfortunately my questions on the xsf chat were left unanswered. In some cases I found the solution myselves though.

Also I began upstreaming some changes and additions to the Smack repository. Parsers and elements of IBB have already been merged, as well as some more additions to the HashManager (XEP-0300) I created earlier, and some tests and fixes for the existing Jingle framework. Still open are my PR for SOCKS5 parsers and the first parts of the Jingle file transfer package.

I also dedicated a tiny little bit of my spare time to a non-GSoC project around a blog post on how to create an OMEMO capable chat client using Smack in less than 200 lines of code. The source code of the example application can be found in the FSFE’s brand new git repository. Unfortunately I also found a small bug in my OMEMO code that I have to fix sometime in the next weeks (nothing crucial, just some annoying faulty behavior).

I plan to spend the coming week working on my Jingle code, so that I have a mostly working framework when the evaluation phase begins.

Thats all for now. Happy Hacking :)

Tutorial: Home-made OMEMO client

Wednesday, June 14th, 2017

The german interior minister conference recently decided that the best way to fight terrorism is passing new laws that allow the government to demand access to communication from messengers like WhatsApp and co. Very important: Messengers like WhatsApp. Will even free software developers see requests to change their messengers to allow government access to communications in the future? If it comes so far, how are we then still possible to protect our communications?

The answer could be: Build your own messenger. I want to demonstrate, how simple it is to create a very basic messenger that allows you to send and receive end-to-end encrypted text messages via XMPP using Smack. We will use Smacks latest new feature – OMEMO support to create a very simple XMPP based command line chat application that uses state of the art encryption. I assume, that you all know, what XMPP is. If not, please read it up on Wikipedia. Smack is a java library that makes it easy to use XMPP in an application. OMEMO is basically the Signal protocol for XMPP.

So lets hop straight into it.
In my example, I import smack as a gradle dependency. That looks like this:

gradle.build

apply plugin: 'java'
apply plugin: 'idea'

repositories {
    mavenCentral()
    maven {
        url 'https://oss.sonatype.org/content/repositories/snapshots'
    }
}

ext {
    smackVersion="4.2.1-SNAPSHOT"
}

dependencies {
    compile "org.igniterealtime.smack:smack-java7:$smackVersion"
    compile "org.igniterealtime.smack:smack-omemo-signal:$smackVersion"
    compile "org.igniterealtime.smack:smack-resolver-dnsjava:$smackVersion"
    compile "org.igniterealtime.smack:smack-tcp:$smackVersion"
}

//Pack dependencies into the jar
jar {
    from(configurations.compile.collect { it.isDirectory() ? it : zipTree(it) }) {
    exclude "META-INF/*.SF"
    exclude "META-INF/LICENSE"
    }
    manifest {
        attributes(
            'Main-Class': 'Messenger'
        )
    }
}

Now we can start the main function of our client. We need to create a connection to a server and log in to go online. Lets assume, that the user passes username and password as arguments to our main function. For sake of simplicity, we’ll not catch any errors like wrong number of parameters etc. Also we want to get notified of incoming chat messages and we want to send messages to others.

Messenger.java

public class Messenger {

    private AbstractXMPPConnection connection;
    private static Scanner scanner;

    public static void main(String[] args) throws Exception {
        String username = args[0];
        String password = args[1];
        Messenger messenger = new Messenger(username, password);

        scanner = new Scanner(System.in);
        while(true) {
            String input = scanner.nextLine();

            if (input.startsWith("/quit")) {
                break;
            }
            if (input.isEmpty()) {
                continue;
            }
            messenger.handleInput(input);
        }
    }

    public Messenger(String username, String password) throws Exception {
        connection = new XMPPTCPConnection(username, password);
        connection = connection.connect();
        connection.login();

        ChatManager.getInstanceFor(connection).addIncomingListener(
                (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message)
        );

        System.out.println("Logged in");
    }

    private void handleInput(String input) throws Exception {
        String[] split = input.split(" ");
        String command = split[0];

        switch (command) {
            case "/say":
                if (split.length > 3) {
                    String recipient = split[1];
                    EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient);

                    StringBuilder message = new StringBuilder();
                    for (int i=2; i<split.length; i++) message.append(split[i]);

                    ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(message);
                }
                break;
        }
    }
}

If we now compile this code and execute it using credentials of an existing account, we can already log in and start chatting with others using the /say command (eg. /say bob@marley.jm Hi Bob!). But our communications are unencrypted right now (aside from tls transport encryption). Lets change that next. We want to use OMEMO encryption to secure our messages, so we utilize Smacks new OmemoManager which handles OMEMO encryption. For that purpose, we need a new private variable which will hold our OmemoManager. Also we make some changes to the constructor.

Messenger.java

private OmemoManager omemoManager;

public Messenger(String username, String password) throws Exception {
    connection = new XMPPTCPConnection(username, password);
    connection = connection.connect();
    connection.login();

    //additions begin here
    SignalOmemoService.acknowledgeLicense();
    SignalOmemoService.setup();
    //path where keys get stored
    OmemoConfiguration.setFileBasedOmemoStoreDefaultPath(new File("path"));
    omemoManager = OmemoManager.getInstanceFor(connection);

    //Listener for incoming OMEMO messages
    omemoManager.addOmemoMessageListener(new OmemoMessageListener() {
        @Override
        public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage,
                        Message wrappingMessage, OmemoMessageInformation omemoInformation) {
            System.out.println("(O) " + encryptedMessage.getFrom() + ": " + decryptedBody);
        }

        @Override
        public void onOmemoKeyTransportReceived(CipherAndAuthTag cipherAndAuthTag, Message message,
                        Message wrappingMessage, OmemoMessageInformation omemoInformation) {
            //Not needed
        }
    });

    ChatManager.getInstanceFor(connection).addIncomingListener(
            (from, message, chat) -> System.out.println(from.asBareJid() + ": " + message)
    );
    omemoManager.initialize();
    //additions end here.
    System.out.println("Logged in");
}

Also we must add two new commands that are needed to control OMEMO. /omemo is similar to /say, but will encrypt the message via OMEMO. /trust is used to trust an identity. Before you can send a message, you have to decide, whether you want to trust or distrust an identity. When you call the trust command, the client will present you with a fingerprint which you have to compare with your chat patner. Only if the fingerprint matches, you should trust it. We add the following two cases to the handleInput’s switch case environment:

Messenger.java

case "/omemo":
    if (split.length > 2) {
        String recipient = split[1];
        EntityBareJid recipientJid = JidCreate.entityBareFrom(recipient);

        StringBuilder message = new StringBuilder();
        for (int i=2; i<split.length; i++) message.append(split[i]);

        //encrypt
        Message encrypted = null;
        try {
            encrypted = OmemoManager.getInstanceFor(connection).encrypt(recipientJid, message.toString());
        }
        // In case of undecided devices
        catch (UndecidedOmemoIdentityException e) {
            System.out.println("Undecided Identities: ");
            for (OmemoDevice device : e.getUntrustedDevices()) {
                System.out.println(device);
            }
        }
        //In case we cannot establish session with some devices
        catch (CannotEstablishOmemoSessionException e) {
            encrypted = omemoManager.encryptForExistingSessions(e, message.toString());
        }

        //send
        if (encrypted != null) {
            ChatManager.getInstanceFor(connection).chatWith(recipientJid).send(encrypted);
        }
    }
    break;

case "/trust":
    if (split.length == 2) {
        BareJid contact = JidCreate.bareFrom(split[1]);
        HashMap<OmemoDevice, OmemoFingerprint> fingerprints =
                omemoManager.getActiveFingerprints(contact);

        //Let user decide
        for (OmemoDevice d : fingerprints.keySet()) {
            System.out.println("Trust (1), or distrust (2)?");
            System.out.println(OmemoKeyUtil.prettyFingerprint(fingerprints.get(d)));
            int decision = Integer.parseInt(scanner.nextLine());
            if (decision == 1) {
               omemoManager.trustOmemoIdentity(d, fingerprints.get(d));
            } else {
                omemoManager.distrustOmemoIdentity(d, fingerprints.get(d));
            }
        }
    }
    break;

Now we can trust contact OMEMO identities using /trust bob@marley.jm and send them encrypted messages using /omemo bob@marley.jm Hi Bob!. When we receive OMEMO messages, they are indicated by a “(O)” in front of the sender.
If we want to go really fancy, we can let our messenger display, whether received messages are encrypted using a trusted key. Unfortunately, there is no convenience method for this available yet, so we have to do a small dirty workaround. We modify the onOmemoMessageReceived method of the OmemoMessageListener like this:

Messenger.java

@Override
public void onOmemoMessageReceived(String decryptedBody, Message encryptedMessage,
            Message wrappingMessage, OmemoMessageInformation omemoInformation) {
    //Get identityKey of sender
    IdentityKey senderKey = (IdentityKey) omemoInformation.getSenderIdentityKey().getIdentityKey();
    OmemoService<?,IdentityKey,?,?,?,?,?,?,?> service = (OmemoService<?,IdentityKey,?,?,?,?,?,?,?>) OmemoService.getInstance();

    //get the fingerprint of the key
    OmemoFingerprint fingerprint = service.getOmemoStoreBackend().keyUtil().getFingerprint(senderKey);
    //Lookup trust status
    boolean trusted = omemoManager.isTrustedOmemoIdentity(omemoInformation.getSenderDevice(), fingerprint);

    System.out.println("(O) " + (trusted ? "T" : "D") + " " + encryptedMessage.getFrom() + ": " + decryptedBody);
}

Now when we receive a message from a trusted identity, there will be a “T” before the message, otherwise there is a “D”.
I hope I could give a brief introduction on how to use Smacks OMEMO support. You now have a basic chat client, that is capable of exchanging multi-end-to-multi-end encrypted messages with other XMPP clients that support OMEMO. All took less than 200 lines of code! Now its up to you to add additional features like support for message carbons, offline messages and co. Spoiler: Its not hard at all :)
You can find the source code of this tutorial in the FSFE’s git repository.

When the government is unable or simply not willing to preserve your privacy, you’ll have to do it yourself.

Happy Hacking :)

GSoC – Second week of coding

Wednesday, June 14th, 2017

The second week of GSoC is over! My Jingle implementation progresses.

Most of my efforts went into designing the state machine behind the Jingle and Jingle File Transfer protocol. Because I never really worked with asynchronous communication, let alone network code before, it takes some time to get my head around that.

I’m heavily utilizing the water fall development model – I code until I get stuck at some point I did not consider at all, then I create a new class and start over again. This is very tideous, but I make slow progress towards working Jingle Socks5 Bytestream transports!

All in all I predict, that it’ll take its time to fully complete the Jingle implementation so that it covers every corner case.

Introducing JET!

While working on my Jingle code, I also started writing down my plans for Jingle Encrypted Transfers (jet). My goal is to keep that specification as simple as possible while providing a reasonable way to exchange encrypted data. I decided, that hiding metadata is not in the scope of this document for now, but can later be specified in a seperate document. Contributions and thoughts regarding encrypted Jingle file transfer are welcome :)

Happy Hacking!

Last week of GSoC Community Bonding

Friday, May 26th, 2017

This is my report for the last week of community bonding. On next tuesday the coding phase officially begins \o/.

I spent my week like I did the one before; writing tests, increasing the codecoverage of my Smack OMEMO module. Today the coverage finally reached the same level as the main codebase, meaning my PR wouldn’t decrease the percentage anymore. Apart from that I’ve read some tips on java.nio usage for file transfer and made myself more familiar with its non-blocking IO concepts.

Throughout the week I took the one or other chance to distract myself from work by pariticipating in OMEMO related discussions on the standards mailing list. I’m quite happy, that there’s a vital discussion on the topic which seems to have a solution in prospect which is acceptable for everyone. Specifying the OMEMO XEP in a way that enables implementations using different crypto libraries is definitely a huge step forward which might bring a broader adoption without leaving those who pioneered and developed the standard standing in the rain (all my subjective opinion). I was really surprised to see developers of the Matrix project participating in the discussion. That reminded me of what the spirit of floss software really is :)

I plan to spent the last days before the coding phase sketching out my projects structure and relaxing a little before the hard work begins. One of my goals is to plan ahead and I really hope to fulfill this goal.

Happy Hacking :)

Vanitasvitae

GSoC: Second, third week of community bonding

Saturday, May 20th, 2017

Hi all!

This is my report for the second, as well as the first half of the third week of GSoC community bonding, which I spent again working on finalizing my OMEMO code.

I dug deeper into writing test cases, mainly integration tests and I found quite a lot of small, undetectable bugs this way (Yay!). This strengthens my plan to work test driven during my GSoC project. The OMEMO code I currently work on was started as part of my bachelor thesis about 4 to 5 months ago and at this time, I was more concerned about having working code in the end, so I wrote no tests at all. Deploying test cases AFTER the code is already written is not only a tideous task, but its also often very difficult (because the code is not structured properly). So I learned my lesson the hard way :D

During testing I also found another bug in an XMPP server software, which prevents Smack from creating accounts on the server on the fly. Unfortunatelly this bug will not get fixed anymore for the version I use (installed from debian testing repository, which I thought was *reasonable* new), which keeps me from doing proper testing the way its meant to be done. I don’t have the time to compile the server software myselves. Instead, I work around this issue by creating the accounts manually everytime I run the test suite using a small bashscript.

I also had to deal with a really strange bug with file writing and reading. smack-omemo has a set of 4 integration tests, which all write data into a temporary directory. After each test, the directory is deleted to prevent tests influencing eachother. The issue was, that only the first test could read/write to the test directory. All subsequent tests failed for some reason. It took me a long time to notice, that there were two folders created (one in the working directory, another one in the subdirectory of the integration test framework). I am still not really sure what happened. The first folder was logged in all debug output, while files were written (by the first test) to the second filder. I guess it was caused by the temp directory being specified using a relative path, which messed up the tests, which were instanciated by the test framework using reflection. But I’m really not sure about this. Specifying the directory using an absolute path fixed the issue in the end.

Last but not least, me and Flow worked out some more details about my GSoC project (Implementing (encrypted) Jingle file transfer for Smack). The module will most likely be based upon java.nio to be scalable in the future. Flow also emphasized that the API should be as easy to use as possible, but at the same time powerful and extensible, which is a nice challenge (and probably a common one within the XMPP community). My initial plan was to create a XEP for OMEMO encrypted Jingle file transfer. We decided, that it would be of more value, to specify the XEP in a way, which allows arbitrary encryption techniques instead of being OMEMO exclusive.

Currently there is a little bit of tension in the community regarding the OMEMO specification. I really hope (and believe) there is a solution which is suitable of making everybody happy and I’m looking forward to participate in an open discussion :)

Happy hacking!

GSoC: First week of community bonding

Wednesday, May 10th, 2017

The first week of community bonding is nearly over and already it’s quite an experience for me. Me and Alameyo were very nicely welcomed by members of the igniterealtime project which really took care of making us able to jump right into the project.

I spent most of the time making myself more familiar with the Smack codebase by working on my OMEMO pull request, which comes closer and closer to a mergeable state. Currently I’m working together with my mentor Flo to make the API of the module as easy to use as possible. I learned the hard way, that singletons and generics are a pain when used together, but in the end it worked out quite well I’d say, although there is still some work left to do.

During the week, I also came across some bugs in XMPP server software, which were a great opportunity to come in contact with other developers outside of the igniterealtime project. Everybody was nice and helpful. Its really pleasant to work with the community.

Lastly, I also got in touch with some other GSoC students, which is super fun. I really enjoy meeting people from other countries and it turns out, that everybody puts their pants on the same way.

I’ll post updates roughly on a weekly basis to document the course of events. Untill then :)

Happy Hacking!

My GSoC introduction

Sunday, May 7th, 2017

Hi!

My name is Paul Schaub (vanitasvitae), I’m a computer science student from M√ľnster, Germany, I’m 23 years old and right now I’m in my 8th semester.

I’m very close to getting my bachelors (undergraduate) degree with only one subject missing (Yay!). That’s why I got some spare time, which I am very happy to be able to spend as a GSoC (Google Summer of Code) student on the Smack project \o/!

 

In my free time, I’m involved in some free software projects mostly around Android apps. I’m currently one maintainer of the unofficial diaspora* app dandelion*. I also wrote the most likely unknown Android app EnigmAndroid which is a simulation of the Enigma machine.

 

My bachelors thesis was about implementing the OMEMO encryption protocol for the Smack library. The result is currently available as an open pull request (there’s still some work to do ). I chose this topic, because I believe that strong, easy to use encryption is very important for society. I’m very happy that the project worked out so great so far.

 

Within the GSoC, I’ll implement Jingle file transfer for Smack as well as finish up the OMEMO PR. Also I’ll try to find an elegant solution for OMEMO encrypted file transfer, which I’ll try to formalize into a (Proto-)XEP. Here you can find my GSoC proposal.

 

I’m always interested in making new contacts in the FOSS world, so if you have any questions or want to chat, don’t hesitate to contact me . You can find my contact information on my github page, but I’m also hanging out in the open_chat MUC.

 

Happy hacking!

Vanitasvitae