Michael McConville (mmcc, mmcco)

I'm a senior at Swarthmore College who worked as Pidgin's resident "Maintenance Hero" through the 2015 Google Summer of Code. I focused on bug fixes, code removal, and security improvements.

Contents

1. Contact
2. Clean-up ideas
3. Some 3.0 blockers
4. Building Pidgin 3.0
   1. "I'm having trouble building 3.0. I'll just make my additions to 2.x.y."
   2. GPlugin
      1. Moonscript
   3. Voice/video library requirements
      1. libnice
      2. Farstream
      3. GStreamer
   4. Anecdotes
   5. "Help!"
5. Building a statically linked Pidgin binary
6. Real-time text
7. Privilege separation
   1. The value of multiprocess design
   2. Initialization
   3. Code annotations
   4. Breakages
   5. Pidgin's processes
8. GTK 3 migration
9. OpenBSD port

Contact

• Jabber: mmcc@jabber.at (OTR preferred)
• Email: mmcco ~a~ mykolab.com

Clean-up ideas

• It seems like the typing status functionality could be condensed a lot, or that "API" could be defined to something more useful/broad. The IM conversation API (found in libpurple/conversationtypes.h) consists of:
  • 2 core functions
  • 2 icon-related functions
  • 11 typing-status-related functions.

Some 3.0 blockers

This list is to help me organize what remains between us and a 3.0 beta release. Many have to do with the plugin API, which changed drastically when the GObject code was merged a year or so ago. We need documentation for these changes, and they caused major regressions in some of our included plugins.

• the History plugin renders its log history below, not above, the current conversation
• at least on OpenBSD, a bug in the plugin unloading logic causes a segfault in GPlugin on quit
• pushed messages (messages which were sent when the recipient was offline) are displayed with only the time sent, not the date
• the typing status logic seems pretty thoroughly broken
  • for example, each backspace will cause a new XMPP typing update, with no rate limiting
  • statuses also don't time out - a 3.0 user will often be listed as "stopped typing" for hours, even when the recipient is using 2.x.y
• the ExtPlacement plugin's config window is broken
• there are no guides to writing plugins with the new API
  • many/most of the major undocumented changes were merged with the GObject branch

Honestly, a 3.0 alpha release could probably be done at this point. It seems acceptably stable on Ubuntu.

Building Pidgin 3.0

Pidgin 3.0 has many dependencies, including some new ones and some that are a little tricky to get set up properly. Below are notes and suggestions on how to reliably build Pidgin 3. Don't take them as gospel - this is simply what I've done to make the builds work reliably.

I've mostly been working on Ubuntu 15.04 (the most recent release at the moment). Because of that, much of the information below is OS-specific. However, the dependencies' Debian and Ubuntu packages are often very similar or even identical, and the two OSs make up a large share of our user base. Obviously, compatible versions of all dependencies are going to have to be available in the Debian and Ubuntu repositories before Pidgin 3 can be effectively shipped.

I install all manually compiled packages into the prefix $HOME/env/ to keep /usr/ clean and Ubuntu-specific. I strongly recommend this - it makes it easier to specify which version of a library should be used, and it prevents linker-related headaches and system reinstalls when you start getting library-related errors. I've added $HOME/env/ to the following environment variables to make it fully usable:

• PATH
• LIBDIR
• LIBRARY_PATH
• LD_LIBRARY_PATH
• C_INCLUDE_PATH
• CPLUS_INCLUDE_PATH
• PKG_CONFIG_PATH

These packages should cover most of the requirements to build with all features enabled (this maximizes test coverage). I probably missed a few, but the configuration failure should make it clear what you need:

sudo apt-get install gtk+3 libtool intltool libenchant-dev libxss-dev libjson-glib-dev libgstreamer1.0-dev gstreamer1.0-plugins-bad gstreamer1.0-plugins-good gstreamer1.0-plugins-ugly gstreamer1.0-plugins-base libgstreamer-plugins-good1.0-dev libgstreamer-plugins-base1.0-dev libidn11-dev libmeanwhile-dev network-manager-dev libnss3-dev libgnome-keyring-dev libqt4-dev kdelibs5-dev

"I'm having trouble building 3.0. I'll just make my additions to 2.x.y."

Please don't.

Your changes would likely be inapplicable. The core library internals are quite different, and almost everything wasn't overhauled is renamed and shuffled into a different file. I learned this the hard way.

GPlugin

​GPlugin was added as a Pidgin dependency in May 2015. It's meant to allow plugins to be written in any programming language through the use of ​GObject introspection. There were build problems in 0.18 involving Mozilla's gjs library (apparently only C++ headers were available while C headers were needed), but that's been removed as a dependency for now.

GPlugin is new, and isn't yet included in Debian or Ubuntu's package repos. You therefore have to build from source. Additionally, I wasn't able to find which (if any) environment variable determines the search path for .gir files. I therefore grudgingly installed it without my build environment prefix. I've since been told that such an environment variable does exist, though - I'll share it here once I find it.

Moonscript

GPlugin requires Moonscript, a Lua library that isn't available in package repos. To get it, just install LuaRocks and call luarocks --local install moonscript. It will use $HOME/.luarocks as the prefix, so add its subdirs to your env vars as needed.

As of GPlugin 0.21, the compile-time Moonscript tests are disabled by default. However, you'll need it to run Moonscript plugins.

Voice/video library requirements

As of December 2015, Pidgin 3 voice/video support requires GStreamer 1.0 and Farstream 0.2. GStreamer 0.10 has been officially unmaintained ​since March 2013. The 0.10 branch had been largely abandoned a while before: "there are many hundreds of bugs that have only been fixed in 1.x, and many more are fixed every week."

GStreamer 0.10 and Farstream 0.1 are coupled, as are GStreamer 1.0 and Farstream 0.2. When determining the compatibility of an operating system release, be sure to check both. For example, Debian Wheezy currently supports GStreamer 1.0 but only Farstream 0.1.

Requiring GStreamer 1.0 and Farstream 0.2 means that Pidgin 3.0 doesn't support certain LTS operating systems such as Ubuntu 12.04, Debian Squeeze, RHEL/CentOS 6, and potentially Debian Wheezy. Of course, these releases are unlikely to backport Pidgin 3.0, considering that they haven't backported GStreamer 1.0. If they do, they can always just build with voice/video disabled (--disable-vv).

libnice

​libnice is a library implementing ICE and STUN which is used by the voice/video stack.

It's Debian/Ubuntu package is maintained by the ​Debian Telepathy Maintainers. This package is currently at version 0.1.7-1 in the repo and 0.1.4-1 in Ubuntu 15.04. The latest libnice release is 0.1.13, and the latest version of Farstream requires at least 0.1.8. libnice 0.13.0 builds and tests successfully on Ubuntu 15.04.

The libnice project seems inactive.

Farstream

​Farstream is a library built atop GStreamer that offers easy, self-contained voice/video protocol implementations accessible through GObject APIs.

The Farstream 0.2 package's minor version in Ubuntu 15.04 is too old for Pidgin 3. It therefore needs to be built manually. It depends on a new version of libnice, so build that first.

GStreamer

The GStreamer library is the basis of Pidgin's voice/video support.

Back when we supported both 0.1 and 1.0, the 0.1 files would sometimes cause conflicts. I don't know whether this is the case now that we've dropped 0.1 support. The minor version available on Ubuntu 15.04 is currently new enough to compile Pidgin 3.0.

Anecdotes

• When testing a build, run Pidgin from a terminal window (just type pidgin -d, or the full pathname) so that you can see loader warnings and other such non-fatal console reports.

Additionally, I'm finding 3.0 much more pleasant to work with.

"Help!"

If you're having trouble building Pidgin 3.0, contact me - I'm happy to help.

Remember, config.log is your friend. Often, configure script errors (particularly involving voice/video and GTK+) are reported very generically to stdout but have specific and helpful error messages in config.log.

Also, if you come across an obstacle on your platform of choice, please report it to the relevant package maintainer with as much useful information as possible. Keeping major dependencies' packages stable and up-to-date will allow us to continue improving voice/video, desktop environment integration, etc.

Building a statically linked Pidgin binary

The command for this is, for example, ./configure --enable-static --disable-shared --with-static-prpls=jabber. In this case, only Jabber/XMPP is supported. The build system will only statically link libraries compiled by Pidgin - all system libraries will be dynamically linked. You can verify this by running ldd in the new Pidgin binary (pidgin/pidgin) after the build.

There are third-party tools that can create what are effectively statically linked binaries using the LD_PRELOAD environment variable. These could be useful for creating a testing binary for distribution, but I haven't had the need for that.

Real-time text

Real-time text (RTT) is a cool feature. It's generally useful, and it's particularly great for deaf people who want to better emulate verbal conversation. For more information, see ​RealTimeText.org

I've tried adding it as specified in ​XEP-0301 twice: first in 2.x.y and then in 3.0. I got as far as a working backend prototype that could send and receive real-time text stanzas.

However, RTT is a large and difficult feature to implement robustly. The format of the data, when it is transmitted and received, and how the GUI responds to it are all fundamentally different than IMs or group chats. Libpurple's core infrastructure is divided between IMs and group chats, with an individual "stack" for each. It seems to me that the only way to properly implement RTT is to add a third stack for RTT.

On one hand, this stack can be much more minimalist than that of IMs. For example, there is no concept of typing status (a feature which demands a lot of code) or HTML in RTT, and RTT demands far fewer GUI hooks. However, adding a full additional stack demands a lot of boilerplate and code duplication. Additionally, I find the XEP overengineered, which may partially explain why (to my knowledge) no major chat application supports it yet.

I still hope that RTT gets added to Pidgin, but it may not make it for the 3.0 release. If you're interested in working on it, let me know and I'll be happy to help. If you're interested in funding RTT work, check out the ​Bountrysource entry.

As of this time of writing (September 15th, 2015), the bounty for RTT is currently at $250.

The author of the RTT XEP-0301 spec, Mark Rejhon, says he is very happy to help with questions and implementation ideas regarding XEP-0301; email mark(at)realjabber.org

Privilege separation

Privilege separation is one of the easiest ways to improve Pidgin's security.

After initial library loading, there are four types of file access for Pidgin:

• access to libpurple-specific files in ~/.purple/
• access to files in arbitrary locations for file transfers
• GTK-related files (icons, etc.) from /usr/share
• access to binaries (these should probably be replaced or removed)

Of course, the third does not apply to libpurple.

This suggests a solution: we can have a process for Pidgin's core that's restricted to ~/.purple/ and /usr/share, and a file transfer helper process with arbitrary file access.

Later, additional daemons could be added for things like logging and restricted to subdirectories of ~/.purple/. This would further protect user credentials and OTR keys.

The value of multiprocess design

Privilege revocation of large, monolithic programs like Pidgin requires complicated access control frameworks. Options include FreeBSD's Capsicum and Linux's AppArmor. These frameworks are baked into the kernel and are therefore platform-specific.

​Tails, an anonymity-focused operating system based on Tor and Debian, includes Pidgin and OTR by default. They've written an AppArmor profile for Pidgin that's now included in the Debian/Ubuntu package apparmor-profiles-extra.

Because of these frameworks' complexity, the best model is:

• program developers make their code multiprocess and refrain from using unnecessary privileges
• packagers and OS/distro developers write MAC profiles

This method allows for more specialization and is therefore far more efficient.

Initialization

OpenBSD's new privilege revocation system call ​pledge(2) is based on the concept that programs do most of their privileged operations during initialization, eventually dropping into a main loop that is "boring and full of buffer overflows". The pledge(2) model therefore allows the process all privileges until it makes the system call. This is fundamentally different from approaches such as AppArmor's, which have a static privilege list stored outside the program that is always enforced.

The pledge(2) model makes more sense for network programs like Pidgin and is easier to implement. In these cases, we trust the binary to not be malicious, but we don't trust the remote network entities it interacts with, and we don't trust the binary to protect us from them. Why lock down the process before it interacts with the network? Many other access control frameworks attempt to protect the system from potentially malicious binaries, which can add complicating rigidity.

This sort of approach can be used in AppArmor's through its aa_change_hat and aa_change_profile functions. However, this is less elegant and seems rarely used.

Code annotations

Anecdotally, it'd be nice to start the convention of using a tag like PRIVSEP in code to help packagers find points of potential lockdown. Lacking these, locating fork() and exec()-family functions with cscope or something similar is a good approach. Running the program in a ​debugger and breaking on new process creation is also educational.

Breakages

There are, of course exceptions. Below is a (likely incomplete) list of files outside of ~/.purple/ that are accessed and where the access occurs.

• /usr/bin/gscore
  • pidgin/plugins/musicmessaging/musicmessaging.c
• /usr/bin/tzc
  • libpurple/protocols/zephyr/zephyr.c
• gconftool-2
  • libpurple/proxy.c

Pidgin's processes

A little annotation of processes that Pidgin 3.0 spawns, where, and why. This will help us know where and when to chroot and what might break.

• gconftool-2 is used to query proxy settings from GNOME
  • see ​libpurple/proxy.c:244
• DNS queries used to be done from forked processes, but we now use GLib's GIO

GTK 3 migration

We still use many deprecated GTK 2 features and get thousands of deprecation warnings when building. I've managed to replace some, but others - including the entire icon system - remain. This project ballooned, and I had to abandon it in order to get more pressing things done. #16715 and #16716 document some of what I found.

OpenBSD port

Porting Pidgin 3 to OpenBSD was a valuable experience. OpenBSD uses compile-time hardening such as a stack protector, as well as strong ASLR and memory sanitization. This exposes memory corruption bugs that usually fly on other platforms. I found at least four of these, three of which I've already fixed.

Porting to a different build environment also helps tease out bugs. I found an issue in our pkg-config logic and another in an Automake script.

Let me know if you want my Makefile. The port currently doesn't need any patches, which speaks well of Pidgin's portability and standards compliance.