Michael McConville (mmcc, mmcco)

I'm a rising senior at Swarthmore College working as Pidgin's resident "Maintenance Hero" through the 2015 Google Summer of Code. My main goal is to help get our 3.0.0 release out the door. I'm also working on security improvements and end-to-end encryption.

Contents

1. Contact
2. Clean-up ideas
3. Some 3.0 blockers
4. Building Pidgin 3.0
   1. GPlugin
      1. Moonscript
   2. libnice
   3. Farstream
   4. GStreamer
   5. Anecdotes
   6. "I'm having trouble building 3.0. I'll just make my additions to 2.x.y."
   7. "Help!"
5. Voice/video library requirements
6. Building a statically linked Pidgin binary
7. Real-time text
8. Privilege separation
   1. The value of multiprocess design
   2. Breakages
   3. Pidgin's processes
9. GTK 3 migration
10. OpenBSD port

Contact

• Jabber: mmcc@jabber.at (OTR preferred)
• Email: mmcco ~a~ mykolab.com

Clean-up ideas

• Drop many #if 0 blocks that imply distracting possibilities or long-ago-deprecated TODOs.
• It seems like the typing status functionality could be condensed a lot, or that "API" could be defined to something more useful/broad. The IM conversation API (found in libpurple/conversationtypes.h) consists of:
  • 2 core functions
  • 2 icon related functions
  • 11 typing-status-related functions.

Some 3.0 blockers

This list is to help me organize what remains between us and a 3.0 beta release. Many have to do with the plugin API, which changed drastically when the GObject code was merged a year or so ago. We need documentation for these changes, and they caused major regressions in some of our included plugins.

• the History plugin renders its log history below, not above, the current conversation
• at least on OpenBSD, a bug in the plugin unloading logic causes a segfault in GPlugin on quit
• pushed messages (messages which were sent when the recipient was offline) are displayed with only the time sent, not the date
• the typing status logic seems pretty thoroughly broken
  • for example, each backspace will cause a new XMPP typing update, with no rate limiting
  • statuses also don't time out - a 3.0 user will often be listed as "stopped typing" for hours, even when the recipient is using 2.x.y
• the ExtPlacement plugin's config window is broken

Honestly, a 3.0 alpha release could probably be done at this point. I think it would be a good idea to drop GStreamer 0.10 and GTK+ 2 support before we do, though, to ensure that no one is building with them.

Building Pidgin 3.0

Pidgin 3.0 has many dependencies, including some new ones and some that are a little tricky to get set up properly. Below are notes and suggestions on how to reliably build Pidgin 3.0. Don't take them as gospel - this is simply what I've done to make the builds work reliably.

I've been doing my dev work on Ubuntu 15.04 (the most recent release at the moment). Because of that, much of the information below is OS-specific. However, the dependencies' Debian and Ubuntu packages are often very similar or even identical, and the two OSs make up a large share of our user base. Obviously, compatible versions of all dependencies are going to have to be available in the Debian and Ubuntu repositories before 3.0 can be effectively shipped.

All commentary on package versions is from early June 2015.

I install all manually compiled packages into the prefix $HOME/env/ to keep /usr/ clean and Ubuntu-specific. I strongly recommend this - it makes it easier to specify which version of a library should be used, and it prevents linker-related headaches and system reinstalls when you start getting library-related errors. I've added $HOME/env/ to the following environment variables to make it fully usable:

• PATH
• LIBDIR
• LIBRARY_PATH
• LD_LIBRARY_PATH
• C_INCLUDE_PATH
• CPLUS_INCLUDE_PATH
• PKG_CONFIG_PATH

These packages should cover most of the requirements. I probably missed a few, but the configuration failure should make it clear what you need:

sudo apt-get install gtk+3 libtool intltool libenchant-dev libxss-dev libjson-glib-dev libgstreamer1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-plugins-base libgstreamer-plugins-good1.0-dev libgstreamer-plugins-base1.0-dev libidn11-dev libmeanwhile-dev network-manager-dev libnss3-dev libgnome-keyring-dev

GPlugin

​GPlugin was added as a Pidgin dependency in May 2015. It's meant to allow plugins to be written in any programming language through the use of ​GObject introspection. There were build problems in 0.18 involving Mozilla's gjs library (apparently only C++ headers were available while C headers were needed), but that's been removed as a dependency for now.

GPlugin is new, and isn't yet included in Debian or Ubuntu's package repos. You therefore have to build from source. Additionally, I wasn't able to find which (if any) environment variable determines the search path for .gir files. I therefore grudgingly installed it without my build environment prefix. I've since been told that such an environment variable does exist, though - I'll share it here once I find it.

Moonscript

GPlugin requires Moonscript, a Lua library that isn't available in package repos. To get it, just install LuaRocks and call luarocks --local install moonscript. It will use $HOME/.luarocks as the prefix, so add its subdirs to your env vars as needed.

As of GPlugin 0.21, the compile-time Moonscript tests are disabled by default. However, you'll need it to run Moonscript plugins.

libnice

​libnice is a library implementing ICE and STUN which is used by the voice/video stack.

It's Ubuntu package is derived from the Debian package, which is maintained by the ​Debian Telepathy Maintainers. This package is currently at version 0.1.7-1 in the repo and 0.1.4-1 in Ubuntu 15.04. The latest libnice release is 0.1.13, and we need at least 0.1.8 for the latest version of Farstream. 0.13.0 builds and tests successfully on Ubuntu 15.04, so I'm working on an update to submit.

Farstream

​Farstream is a library built atop GStreamer that offers easy, self-contained voice/video protocol implementations accessible through GObject APIs.

There are two major versions of Farstream: 0.1 and 0.2. Both are available Ubuntu 15.04's repos, but the 0.2 package's minor version is too old for Pidgin 3.0. It therefore needs to be built manually. It depends on a new version of libnice, so build that first.

GStreamer

The GStreamer library is the basis of Pidgin's voice/video support.

There are two major GStreamer versions: 0.10 and 1.0. Pidgin currently supports both, but you should use GStreamer 1.0. If possible, use dpkg --get-selections to find and uninstall all GStreamer 0.10 packages. If some software you use depends on them, just check the ./configure output or config.log to make sure that GStreamer 1.0 is being used. The minor version available in Ubuntu 15.04's packages is currently new enough to compile Pidgin 3.0, so just install from there.

Anecdotes

• Another GSoC student, who was using GTK+2, reported weird GUI errors like missing input boxes that were unreported in the debug log. GTK+3 is your best bet - sudo apt-get install gtk+3 should get you at least most of the way there.
• Run Pidgin from a terminal window (just type pidgin -d, or the full pathname) so that you can see loader warnings and other such non-fatal console reports.

"I'm having trouble building 3.0. I'll just make my additions to 2.x.y."

Please don't.

Your changes are likely to be inapplicable or non-translatable, meaning that they'd have to be completely rewritten to be added to 3.0. The fundamental library internals are quite different, and almost everything wasn't overhauled is renamed and shuffled into a different file. I learned this the hard way.

Additionally, I'm finding 3.0 much more pleasant to work with.

"Help!"

If you're having trouble building Pidgin 3.0, contact me - I'm happy to help.

Remember, config.log is your friend. Often, configure script errors (particularly involving voice/video and GTK+) are reported very generically to stdout but have specific and helpful error messages in config.log.

Also, if you come across an obstacle on your platform of choice, please report it to the relevant package maintainer with as much useful information as possible. Keeping major dependencies' packages stable and up-to-date will allow us to continue improving voice/video, desktop environment integration, etc.

Voice/video library requirements

As of June 2015, Pidgin supports both GStreamer 0.10 and GStreamer 1.0. GStreamer 0.10 has been ​officially unmaintained since March 2013 - more than two years ago. The announcement mentions that the 0.10 branch had been largely abandoned a while before: "there are many hundreds of bugs that have only been fixed in 1.x, and many more are fixed every week." However, package and port maintainers may still patch bugs.

Pidgin's voice/video has problems with mysterious failures and sometimes segfaults because of GStreamer plugin incompatibilities. Restricting ourselves to a single current and supported version of the relevant libraries would make the code more reliable and easier to test and maintain.

On the other hand, Tomasz pointed out that GStreamer 1.0 only reintroducted device selection in version 1.4, the most recent minor release. Because Windows and some Linux distributions may not support 1.4 for a while, we may need to implement that ourselves.

GStreamer 0.10 and Farstream 0.1 are coupled, as are GStreamer 1.0 and Farstream 0.2. When determining the compatibility of an operating system release, be sure to check for both. For example, Debian Wheezy currently supports GStreamer 1.0 but only Farstream 0.1.

Requiring GStreamer 1.0 and Farstream 0.2 would mean that Pidgin 3.0 would not support certain LTS operating systems such as Ubuntu 12.04, Debian Squeeze (6) and potentially Wheezy (7), and RHEL/CentOS 6.

Of course, these releases are unlikely to backport Pidgin 3.0, considering that they haven't backported GStreamer 1.0. If they do, they can always just build with --disable-vv.

Building a statically linked Pidgin binary

The command for this is, for example, ./configure --enable-static --disable-shared --with-static-prpls=jabber. In this case, only Jabber/XMPP is supported. The build system will only statically link libraries compiled by Pidgin - all system libraries will be dynamically linked. You can verify this by running ldd in the new Pidgin binary (pidgin/pidgin) after the build.

There are third-party tools that can create what are effectively statically linked binaries using the LD_PRELOAD environment variable. These could be useful for creating a testing binary for distribution, but I haven't had the need for that.

Real-time text

Real-time text (RTT) is a cool feature. It's generally useful, and it's particularly great for deaf people who want to better emulate verbal conversation. For more information, see ​RealTimeText.org

I've tried adding it as specified in ​XEP-0301 twice: first in 2.x.y and then in 3.0. I got as far as a working backend prototype that could send and receive real-time text stanzas.

However, RTT is a large and difficult feature to implement robustly. The format of the data, when it is transmitted and received, and how the GUI responds to it are all fundamentally different than IMs or group chats. Libpurple's core infrastructure is divided between IMs and group chats, with an individual "stack" for each. It seems to me that the only way to properly implement RTT is to add a third stack for RTT.

On one hand, this stack can be much more minimalist than that of IMs. For example, there is no concept of typing status (a feature which demands a lot of code) or HTML in RTT, and RTT demands far fewer GUI hooks. However, adding a full additional stack demands a lot of boilerplate and code duplication. Additionally, I find the XEP overengineered, which may partially explain why (to my knowledge) no major chat application supports it yet.

I still hope that RTT gets added to Pidgin, but it may not make it for the 3.0 release. If you're interested in working on it, let me know and I'll be happy to help. If you're interested in funding RTT work, check out the ​Bountrysource entry.

Privilege separation

Privilege separation is one of the easiest ways to improve Pidgin's security.

After initial library loading, there are four types of file access for Pidgin:

• access to libpurple-specific files in ~/.purple/
• access to files in arbitrary locations for file transfers
• GTK-related files (icons, etc.) from /usr/share
• access to binaries (these should probably be replaced or removed)

Of course, the third does not apply to libpurple.

This suggests a solution: we can have a process for Pidgin's core that's restricted to ~/.purple/ and /usr/share, and a file transfer helper process with arbitrary file access.

Later, additional daemons could be added for things like logging and restricted to subdirectories of ~/.purple/. This would further protect user credentials and OTR keys.

The value of multiprocess design

There isn't a simple, intuitive, widely-used, and easy-to-learn Mandatory Access Control (MAC) or similar framework yet. Options include FreeBSD's Capsicum, Linux's AppArmor, and OpenBSD's newly released tame syscall. To further complicate things, these frameworks are generally baked into the kernel and are therefore platform-specific.

​Tails, an anonymity-focused operating system based on Tor and Debian, includes Pidgin and OTR by default. They've written an AppArmor profile for Pidgin that's now included in the Debian/Ubuntu package apparmor-profiles-extra.

All existing MAC frameworks are pretty cumbersome and have a slow learning curve. (tame is trying to buck this trend, but it's far too new and rarely used to be an option yet.) So, the best model is:

• program developers make their code multiprocess and refrain from using unnecessary privileges
• packagers and OS/distro developers use this to write good MAC profiles

Anecdotally, it'd be nice to start the convention of using a tag like PRIVSEP in code to help packagers find points of potential lockdown. Lacking these, searching for fork() and exec()-family functions with cscope or something similar is a good approach. Beyond that, running the program in a debugger and breaking on new process creation is educational, as is understanding its initialization and main() logic well.

Breakages

There are, of course exceptions. Below is a (likely incomplete) list of files outside of ~/.purple/ that are accessed and where the access occurs.

• /usr/bin/gscore
  • pidgin/plugins/musicmessaging/musicmessaging.c
• /usr/bin/tzc
  • libpurple/protocols/zephyr/zephyr.c
• gconftool-2
  • libpurple/proxy.c

Pidgin's processes

A little annotation of processes that Pidgin 3.0 spawns, where, and why. This will help us know where and when to chroot and what might break.

• gconftool-2 is used to query proxy settings from GNOME
  • see ​libpurple/proxy.c:244
• DNS queries are done from forked processes
  • these don't seem to use any significant access and could probably be heavily locked down
  • reliably, one seems to die on startup - is this bug?
  • see ​purple_dnsquery_resolver_run()

GTK 3 migration

This project ballooned, and I had to abandon it in order to get more pressing things done. #16715 and #16716 document some of what I found.

OpenBSD port

Porting Pidgin 3 to OpenBSD was a valuable experience. OpenBSD builds with strong compiler hardening and loads executables with strong ASLR and memory sanitization, so memory corruption bugs that usually fly on other platforms are often exposed. I found at least four of these, three of which I've already fixed upstream.

Porting to a different build environment also helps tease out bugs. I found an issue in our pkg-config logic and another in an Automake script.