Opened 17 months ago

Last modified 5 weeks ago

#16801 new enhancement

Add OMEMO Encryption support to XMPP

Reported by: thenktor Owned by: deryni
Milestone: Component: XMPP
Version: 2.10.11 Keywords: OMEMO, TextSecure, Axolotl, encryption
Cc: doctorlard, doczkal, drzraf, herbsmn, Mkaysi, calestyo, laclaro, dllud, chaoscommander, bird.dandruff, blipp, pravi, Moarc, mancho, dakra

Description

Please add support for the new OMEMO draft-XEP: http://conversations.im/xeps/multi-end.html

OMEMO is an XMPP Extension Protocol (XEP) for secure multi-client end-to-end encryption: http://conversations.im/omemo/ It offers Forward Secrecy and deniability while allowing you to keep the benefits of message synchronization and offline delivery.

OMEMO uses the Axolotl ratchet to establish secure sessions between every combination of devices: https://github.com/trevp/axolotl/wiki

Here is a corresponding ticket in Gajim's trac: https://trac.gajim.org/ticket/8161

Change History (38)

comment:1 Changed 16 months ago by doctorlard

Seems to be this can be developed using:

  • the Whisper Systems libaxolotl-c library; note that it only requires OpenSSL (BSD-licensed) for its unit tests, and
  • support in the client for XEP-0163 (Personal Eventing Protocol), which is already in Pidgin according to the wiki.

The main advantage of OMEMO over OTR is that it works in multi-user chats, and between multiple devices/clients when combined with support for XEP-0280 Message Carbons #15508 and XEP-0313 Message Archive Management #15653. Conversations implements this on Android, and I have to say it works beautifully well with my devices and a Prosody 0.9.x server with pep, mam and carbons modules enabled.

I wish I had more time, I'd give this a crack myself! Good to see progress being made on some of these XEP tickets - especially the recent GSoC work on XEP-0280 Carbons support, well done Koosha :)

Last edited 16 months ago by doctorlard (previous) (diff)

comment:2 Changed 15 months ago by doctorlard

Whoops, I lied about the whole OMEMO multi-user chat thing; my misunderstanding. There is however a review of the mpOTR paper ("multi-party OTR") and the current Whisper Systems multi-user approach here.

comment:3 Changed 14 months ago by doczkal

Is this implemented already?

comment:4 Changed 14 months ago by self

Maybe somebody can port it from Gajim https://trac.gajim.org/ticket/8161

comment:5 Changed 12 months ago by herbsmn

This is important. Is anyone working on it?

Conversations just got encryped OMEMO group / conferences working a few weeks ago.

Here's their github issue where they talk about it: https://github.com/siacs/Conversations/issues/1372

Here's where they explain how it works in their github README: https://github.com/siacs/Conversations#how-does-the-encryption-for-conferences-work

CryptoCat? just got OMEMO working too. https://crypto.cat/security.html

OMEMO is the future. Will Pidgin be a part of it?

comment:6 Changed 11 months ago by herbsmn

The Conversations App, Chatsecure, and Monal devs are discussing creating a non-GPL licenced version of OMEMO that can be implimented in iOS XMPP clients here: https://github.com/anurodhp/Monal/issues/9

Seems like they are looking to base it off of the Apache 2.0 licenced OLM ratchet implementation: https://matrix.org/git/olm/ https://github.com/chrisballinger/OLMKit/tree/olmkit/xcode

It might be worth it for Pidgin to wait for this OMEMO fork to be created before implimenting the current version of OMEMO since the fork will be incompatable with the current version.

It'd be swell if Pidgin users were able to communicate via XMPP with iOS clients using a ratchet based encyrption protocal, no?

comment:7 Changed 11 months ago by doctorlard

Yeah, it just needs some clever sausage to write it.

comment:8 Changed 11 months ago by thenktor

@herbsmm: XMPP with OMEMO may be nice but XMPP clients cannot receive messages while they are not running in foreground or while the device screen is off. So you have to start the app yourselve to look for new messages. So XMPP sadly is no alternative to mobile IMs on iOS (Signal, Threema, Whatsapp, ...).

comment:9 Changed 11 months ago by herbsmn

That's not exactly correct.

https://chatsecure.org/blog/#push

https://chatsecure.org/blog/#on-the-roadmap

https://github.com/chatsecure/chatsecure-push-server

"Background sockets can be enabled easily if you don't mind compiling yourself. Look out for blog post soon about our new push stuff" https://twitter.com/ChatSecure/status/720116067885019137

Sounds like this problem is getting fixed by the same people that are working on this new version of OMEMO. The future in this area looks bright.

Limited Push for Chatsecure is in Beta right now! Sign up and try it out! https://twitter.com/ChatSecure/status/712033573503680512

comment:10 Changed 11 months ago by herbsmn

I was told by Daniel Gultsch who is the main OMEMO guy that, "This is not going to be a fork. This is just going to be the next step in the standardization process". "This is simply going to be OMEMO."

So.... Pidgin should get cracking! OMEMO is the future and is being actively developed to add XMPP encryption support for iOS.

If you use an iPhone and Pidgin, wouldn't you like to send OMEMO encrypted messages back and forth btw both devices? The ability to do this is coming in the short term future.

If you use an android device, you could already send OMEMO encrypted messages to and from your phone right now with Conversations.im but not with Pidgin! You'd have to use Gajim.org on your desktop or laptop instead.

What are the blockers to getting this going? Can anyone document what needs to happen to add support for OMEMO?

comment:11 Changed 11 months ago by doctorlard

thenktor: "XMPP clients cannot receive messages while they are not running in foreground or while the device screen is off."

That's simply not true. Modern XMPP clients can support a combination of XEPs: Stream Management, Client State Indication, and Message Archive Management. e.g. https://conversations.im/#optimizations

comment:12 Changed 11 months ago by thenktor

@doctorland: That may be fine for Android, but for iOS AFAIK the only way to deliver a message while the device is in standby is to use the Apple push notification service. Apps are not allowed to run in background.

But we should go back to topic ;-)

comment:13 Changed 11 months ago by herbsmn

@doctorland and @thenktor here's a write up on the iOS XMPP push stuff that chatsecure is doing: https://chatsecure.org/blog/chatsecure-v32-push/

Also, back to topic, here's ChatSecure? talking about OMEMO's future: https://chatsecure.org/blog/chatsecure-v32-push/#the-road-ahead

If you are reading this, and you know the technical details of what needs to be done for Pidgin to impliment OMEMO, can you please enumerate the subtasks that need to be done to reach the goalline? OTR is showing its age and we need to give Pidgin users the option to use the best encryption standards that are out there, namely OMEMO. Let's strike while the iron is hot.

comment:14 Changed 11 months ago by calestyo

CCing myself,... this seems to be a much better goal than #16537 :)

comment:15 Changed 10 months ago by laclaro

Hi,

is someone already working on this feature? I think OMEMO has to implemented as soon as possible in all majow XMPP clients. And I think pidgin is one of the most important XMPP-clients out there now.

Why? Because the possibility to have multiple ressources/clients on multiple platforms (laptop, phone, car, desktop, ...) is one major strength of XMPP. OMEMO makes it easily possible to encrypt traffic to all these resources (unlike OTR). Modern mobile chat apps nowadays also tend to use end-to-end-encryption, but usually they are designed to work with one platform only. If XMPP wants to compete with them, we have to play the decentralized-server-multi-platform-card even more.

Then one could tell: "We encrypt as good as you are, but additionally you can choose your platform and your client, we are decentralized, non-spying and open-source. What have you got?"

Without modern encryption, we cannot play that card that well, because people would just say: "Well, I WANT an easy-to-use encryption, otherwise I cannot send my nude pictures securely. Thank's, I'll stick with app XYZ."

Best wishes Henning

Last edited 10 months ago by laclaro (previous) (diff)

comment:16 Changed 10 months ago by EionRobb

Looks like another implementation of Axolotl aka Signal Protocol called olm at https://matrix.org/git/olm/about/ (Apache licensed)

Is there an XEP for OMEMO yet that goes beyond "use Axolotl"?

comment:17 Changed 10 months ago by herbsmn

the Olm / Matrix person and the Conversations, Monal, and ChatSecure? persons were all talking OMEMO in this github thread if you guys wanna ask questions to all of them at once. I don't know if there's a different "hub" elsewhere you can connect with them all. https://github.com/anurodhp/Monal/issues/9

comment:18 Changed 8 months ago by bird.dandruff

besides message carbons (#15508) and stream management (#14252) this is one of the most important missing features...

comment:19 follow-up: Changed 6 months ago by herbsmn

It is pretty shameful that Pidgin isn't working on this. Gajim now has their omemo plugin in debian: https://packages.debian.org/sid/gajim-omemo Pidgin doesn't seem to even want to start developing an OMEMO plugin. Can a pidgin dev please respond in here and explain the logic of ignoring this nearly year old ticket? Is it too much to ask that work should begin to add support for the encryption protocol of the future?

comment:20 follow-up: Changed 6 months ago by herbsmn

Maybe consider changing your project's name to Ostrich, since your heads are obviously in the sand when it comes to adding support for the new, audited, state of the art encryption standard. Your devs won't even articulate a reason why you won't look at the standard. Shameful.

comment:21 in reply to: ↑ 19 ; follow-up: Changed 6 months ago by EionRobb

Replying to herbsmn:

It is pretty shameful that Pidgin isn't working on this. Gajim now has their omemo plugin in debian: https://packages.debian.org/sid/gajim-omemo Pidgin doesn't seem to even want to start developing an OMEMO plugin. Can a pidgin dev please respond in here and explain the logic of ignoring this nearly year old ticket? Is it too much to ask that work should begin to add support for the encryption protocol of the future?

a) the OMEMO XEP hasn't been approved as a valid XEP to implement (has it even been assigned an XEP number yet?)

b) us Pidgin devs are working on other things - we are unpaid volunteers working on this in our spare time.

c) there's nothing stoping you from writing it yourself if there's a feature you're hoping to see implemented. e.g. the OTR plugin is a 3rd party plugin not written by any Pidgin dev

comment:22 in reply to: ↑ 20 Changed 6 months ago by dx

Replying to herbsmn:

Maybe consider changing your project's name to Ostrich, since your heads are obviously in the sand when it comes to adding support for the new, audited, state of the art encryption standard. Your devs won't even articulate a reason why you won't look at the standard. Shameful.

Hello herbsmn, thanks for your interest in OMEMO.

I'm the main developer of bitlbee (which can use libpurple as backend), and occasional contributor to pidgin itself. As a bitlbee developer I had to deal with its own XMPP and OTR implementations, and sometimes patching issues in libotr itself. Turns out that, despite the attempt to have a minimal and easy to use API, dealing with libotr required handling several annoying undocumented edge cases and which required a deep understanding of how otr works.

These issues don't necessarily apply to libsignal-protocol - but others may. Its interface is significantly lower level than libotr - instead of placing a handful of hooks to feed libotr whole messages and let it send its own, you have to do all the session handling yourself. That means even more places where things could go wrong.

It could be an interesting challenge, but an extremely time consuming one, and most importantly, something that can't be done by anyone. Being a C project, our options are more limited - it's a language that requires additional attention to detail, and good C developers with plenty of free time are hard to come by, especially because those tend to be employed by companies that pay proportionally to their skills.

You can contact me at dx@… to get my hourly rates.

comment:23 in reply to: ↑ 21 Changed 6 months ago by bird.dandruff

Replying to EionRobb:

a) the OMEMO XEP hasn't been approved as a valid XEP to implement (has it even been assigned an XEP number yet?)

no XEP number yet, but the latest progress on the topic can be read here: https://mail.jabber.org/pipermail/standards/2016-September/031429.html

comment:24 Changed 3 months ago by pravi

OMEMO is now a released standard http://www.xmpp.org/extensions/xep-0384.html

comment:25 Changed 3 months ago by pravi

Now OMEMO is based on Olm. If we build a purple-olm, xmpp and matrix (via purple-matrix) can reuse it.

comment:26 Changed 2 months ago by herbsmn

Both OMEMO and OLM have been audited by third parties: ​https://conversations.im/omemo/audit.pdfhttps://www.nccgroup.trust/us/our-research/matrix-olm-cryptographic-review/

Some of this content is outdated, but a lot of documentation was written a few months ago about OMEMO here: ​https://we.riseup.net/riseup/xmpp

OMEMO is being ported to Profanity.im as well ​https://github.com/boothj5/profanity/issues/658

Usability for the only desktop client that supports OMEMO currently, Gajim, is not perfect. ​https://current.workingdirectory.net/posts/2017/encrypted-mucs/

I understand that certain pidgin devs don't want to work on this with out getting paid. I'm probably going to keep updating this ticket with information in case someone changes their mind or someone decides to pay for the development work.

Perhaps Tails OS would be interested in collaborating or funding this: https://labs.riseup.net/code/issues/11541#change-64941

comment:27 Changed 2 months ago by EionRobb

Nothing at all to do with getting paid. It's just up to whoever has enough interest to do it.

That said, there are two different people who have said they're working on it as part of their thesises, one on the developer mailing list who's working on his master's thesis, one in the #pidgin irc channel for his bachelor's thesis. Unfortunately neither has made their source code public yet so no one else can help them out.

comment:28 Changed 2 months ago by herbsmn

This is encouraging! Do you mind mentioning their usernames?

Please let them know that InstantBird? and Tor Messenger, which I think both share some code with you, both have open tickets about adding OMEMO also:

https://bugzilla.mozilla.org/show_bug.cgi?id=1237416 https://trac.torproject.org/projects/tor/ticket/17457

comment:29 Changed 2 months ago by blipp

Hi, speaking for one of the parties interested in implementing OMEMO for Pidgin, actually we already started. But we might suspend our implementation efforts in favor of the person working on it for their thesis and might move to another client – currently discussing this with them. I am blipp on #pidgin, but currently not so active on IRC.

comment:30 Changed 2 months ago by mancho

Hello, speaking for the other one of the parties, I'm working on it as part of my bachelor thesis. It's somewhat usable now. At least after one has got past the initial configuration.

I'm now entering the writing phase of my work. After that I plan to migrate the repository to GitHub? an develop more openly once I'm done at the university, which should happen fairly soon ;)

You can find the code here https://git.imp.fu-berlin.de/mancho/libpurple-omemo-plugin

I know it's not the best, but it's enough as a prototype to base my paper on. The real thing comes later this Spring, I hope :)

comment:31 Changed 2 months ago by herbsmn

thanks for the update mancho and blipp!

blipp: are you considering moving to InstantBird? or Tor Messenger or some other client?

mancho: thanks for posting your repo! best of luck writing your paper! looking forward to seeing the real this this Spring, as you hope.

comment:32 Changed 2 months ago by bird.dandruff

Hi mancho,

great to hear of your project and progress! Thank you for doing this!

After entering the phase for the "real thing" you may consider to swap libsignal-protocol-c for libolm as Olm is the official reference implementation described in the OMEMO-XEP.

@blipp: telepathy could also use some love ;)

comment:33 follow-up: Changed 8 weeks ago by psi1337

I really would appreciate OMEMO in Pidgin! Perhaps we could file a bounty? I'm willing to put some coints in.

comment:34 in reply to: ↑ 33 Changed 8 weeks ago by Robby

Replying to psi1337:

I really would appreciate OMEMO in Pidgin! Perhaps we could file a bounty? I'm willing to put some coints in.

Here: https://www.bountysource.com/issues/28287404-add-omemo-encryption-support-to-xmpp

comment:35 follow-up: Changed 7 weeks ago by RoPe93

comment:36 in reply to: ↑ 35 Changed 7 weeks ago by EionRobb

Replying to RoPe93:

Omemo plugin for Pidgin done (not by me): https://github.com/gkdr/lurch

https://twitter.com/Fr333k/status/825767756113047556

Not yet GPL-compatible unfortunately (relies on OpenSSL) although _riba has been talking in the #pidgin irc channel about changes to resolve this

comment:37 Changed 5 weeks ago by _riba

Just a quick update: The license issues were resolved, and it now also compiles on Windows and MacOS. Thanks to EionRobb?, there is even a precompiled windows build.

comment:38 Changed 5 weeks ago by DreamFlasher

Thank you for your great work @_riba! Make sure to collect your bounty! https://www.bountysource.com/issues/28287404-add-omemo-encryption-support-to-xmpp

Note: See TracTickets for help on using tickets.
All information, including names and email addresses, entered onto this website or sent to mailing lists affiliated with this website will be public. Do not post confidential information, especially passwords!