Support XMPP Invisibility

[js]

When using multiple accounts and switching the status to invisible, nothing is done for Jabber at all. If someone was away and switches to invisible, the away status will be kept for XMPP. You should either implement invisibility for XMPP or set the status to online for XMPP then.

[rekkanoryo]

We do not implement any of the existing invisibility methods for XMPP, therefore it is impossible to set an invisible status on an XMPP account.

[rekkanoryo]

I forgot to update the component here.

[js]

Which is why I suggested: Implement it or set the status back to online.

[rekkanoryo]

I'll note here that seanegan has already stated he has no desire to support XMPP invisibility.

[js]

No, this is *not* the request of this bug report. The request is to either support XMPP invisibility or if that's not done set the status to online, so the away status won't be kept. To sum it up: The ideal solution would be to support invisibility if. If that's not possible, set the status for XMPP to online and for all other protocols to invisible.

[Menedas]

Why will invisibility not be implemented?

How to set it manually in the XMPP console? Is it also possible to set the mode only for some users in the contact list?

[deryni]

Up until recently there was not any viable, useful invisibility support for XMPP at a protocol level, this has only just recently become reasonable to support. It has not yet happened because no one with the time, interest, and knowledge has written it.

The invisibility support that pidgin will accept (and which I hope to eventually have time to implement) will be that specified by XEP-0186 (unless that is retracted or superseded by the time I get to it). You are free to try the snippets provided by that XEP in the XMPP Console, but realize that your server may not support it. Furthermore, pidgin will still automatically respond to certain requests even once you have sent the invisible command and that will betray your online presence.

Alternatively, it is possible to send a presence stanza of type unavailable to the server which *may* convey some/many of the invisibility 'benefits' but also may do things you don't want.

Neither of those methods allow for being invisible only to specific buddies however.

[Menedas]

But it seems there is a posibility given to set invisible to specific buddies.

User Becomes Invisible to Selected Contact: http://www.xmpp.org/extensions/xep-0018.html#sect-id2252739

And for me this seems to work over the XMPP-Console.

For the global invisible like this: <iq from='bilbo@…/shire'

id='inv1' type='set'>

<invisible xmlns='urn:xmpp:tmp:invisible'/>

</iq>

I only get a <error code='503' type='cancel'>

[js]

XEP-0186 support is pretty useless, as no server currently implements that.

What's in active use and does not break the core RFC (unlike type='invisible') is to use privacy lists for invisibility. Every server supports that and nearly every client supports it either - I don't see why Pidgin should be an exception here. Plus we would FINALLY have a working Privacy Dialog for XMPP. It's a shame to have privacy lists for proprietary garbage like ICQ, but not for XMPP.

For XEP-0018: Stay the hell away from it! It breaks the core RFC and thus has been rejected!

[js]

Oh, plus we use privacy lists for invisibility in ICQ as well. So it's not an argument that using privacy lists for that might be hacky.

[deryni]

Menedas: XEP-0018 is rejected, which means it is not suitable for use by XMPP clients and servers, thus making anything it suggests invalid. You get that error code because (as I suggested) most server do not support XEP-0186 yet (as it is still relatively new).

js: The fact that no servers support it at the moment doesn't make supporting it useless, it does mean that supporting it doesn't have any effect at the moment, but until and unless something better comes up that is what the current invisibility suggestion is for XMPP.

Do not let the fact that the name 'privacy list' is used for both ICQ and XMPP fool you, they are not at all the same thing.

The privacy lists in ICQ are (as far as I have ever understood) much *much* simpler than the XMPP feature of the same name. In ICQ there is exactly one visible list and exactly one invisible list, those lists are simple boolean lists (either a name is on the list or it is not), those lists are usable by id and can be guaranteed to exist, they only block presence notifications (not messages), and (historically) clients could only be signed on to an account from any one location at one time.

Privacy lists in XMPP are incredibly more complicated than that: as many lists as the user wishes to create can exist; the lists can support entries by jid, by domain, by group, or by subscription status; the lists are usable by name (not ID) and cannot be guaranteed to exist (XEP-0126 was an attempt to standardize a name so that clients could start to depend on the lists existing, it never made it very far); the lists can block presence (in each direction separately), messages (in each direction separately), and iq stanzas (in each direction separately); and clients have always been able to be logged in to more than one location (thus requiring client-to-client synchronizing of invisibility state and rule composition). A few last things, XMPP privacy lists are stackable and given the extensibility of XMPP the client (and the server) need to handle far more corner-cases in terms of valid and invalid responses than I believe ICQ clients (and the server) need to deal with.

There is a reason that, despite privacy lists being in the core XMPP RFCs, so few clients and servers really support them in a reasonable manner and that at least three attempts at simplifying, explaining, or replacing them as a means for invisibility have occurred. (I challenge you to show me multiple clients and servers that correctly and usefully support privacy list usage for invisibility purposes.)

[js]

I know privacy lists pretty well and it's not necessary to have a full implementation of that to have the Privacy thing in pidgin working and for invisibility. In fact, that'd be pretty easy. I'd even prefer that approach to invisible command as invisible command is experimental and not recommended for production use yet.

[deryni]

It is necessary to have a more-or-less full implementation of privacy lists to be able to reasonably support it given anything beyond a single client from a single computer with only serial permission changes. Not to mention the fact that pidgin is not in any way at the moment set up to control when it responds to incoming requests from people who happen to be on the privacy list (which last I checked was something the clients needed to do to be sure nothing leaked). If you think it is pretty easy to implement then I eagerly await the patch you are going to provide us.

Also, experimental status means that it is not yet final and may change, not that it shouldn't be implemented, in fact quite the opposite it is intended that people implement it so that any necessary changes or clarifications can be made so that it may advance to draft status.

[js]

For invisible to work, we could just create a list, call it invisible and activate it. That are two stanzas.

For privacy lists, we can have predefined lists like blocked, visible, etc. These are 2 stanzas. They could be created on connecting and deleted after that.

No, Pidgin does not need to know which client to answer and which not, as privacy lists are server based. If you block outgoing presence, outgoing presence is blocked. If you block incoming messages, you won't have any message you can answer to to leak presence.

[shreg]

If invisibilty is not implemented for XMPP, it could be usefull to WARN the user that pidgin does not support invisibility when the user set invisibility ON.

A message on top of the contact windows, "like the one for new email" for example.

my 2 cents to this small bug in this great software

[deryni]

No, we cannot "just create a list, call it invisible and activate it". That will overwrite any existing list with the name we use, which is a really bad idea if people already have a list set up in some other client. So we now need to also fetch the names of all the existing lists, fetch the definition of the list we want to edit, make the change we want locally and then set and activate it.

Constantly creating and deleting lists is not a good idea, especially not if we want to support any sort of ability to restore previous lists or allow finer grained lists. It likewise plays not at all well with any other clients trying to handle privacy lists, and disallows multiple clients to collaborate on a list.

Yes, privacy list blocking is server side and will not require pidgin to control when it does and doesn't respond to stanzas, but the other privacy/invisible mechanisms do require that.

And no blocking messages will not block IQ stanza, or presence stanzas. Which means that the UI needs to allow for each type of blocking separately.

shreg: Yes, it would be good of pidgin to notify users when the selected status cannot be applied, and is something I would be quite happy to accept a patch for that sort of thing. File a new ticket with that as a feature request please.

[js]

Yes, we can just create and activate a list and call that libpurple_invisible for example. And if you fear that two Pidgins could override their lists, we could attach an UUID to the name. We could create this list as not activated by default, but activate it when we want to be invisible.

[deryni]

Yes, and that removes the ability to have multiple clients update a single list, so you can't keep invisibility consistent between multiple locations, it also removes the ability for non-pidgin/libpurple clients to use the list, etc. As I said, 'proper' privacy list support is not easy and is required for anything even remotely resembling useful invisibility support, without it people will complain. I have no intention of cobbling together something broken and half-finished.

Not to mention the fact that using privacy lists for invisibility is generally seen as an improper use of them, which is why the other XEPs were attempted.

[jozeck]

But there are some workarounds of invisible status in XMPP, i.e.: http://www.igniterealtime.org/community/blogs/ignite/2008/02/19/playing-casper-in-openfire-350

What about this implementation?

[deryni]

'Invisibility' of that sort requires a fair amount of manual user work or the client doing a fair amount of guesswork as to what the user wants (when to send directed available presence, and when to send directed unavailable presence). That being said it should be possible for a plugin watching the incoming and outgoing jabber-* signals to filter outgoing presence broadcast and to send/revoke available presence to specific users given whatever criteria the plugin supports (opening and closing of conversation windows, right-click menu items for given buddies, for given groups, etc.). Such an implementation would not be ideal but should function at least somewhat.

Since I last commented on this ticket, pidgin has gained support for XEP-0191 (at least partially, and a patch exists for what should be the rest of it, though I haven't looked at the XEP recently). The relevant tickets are #7670 and #8045. Not that that is directly relevant but was something I figured might be worth mentioning.

[pidgin]

you should reopen the gtalk invisibility #5828 ticket, because gtalk implemented the invisibilty in a different way. gtalk invisibility should be implemented because its also supported via their own gtalk client.

implementing a fully standards compliant xmpp invisibility, like in this ticket is a different thing.

[darkrain42]

Ticket #9392 has been marked as a duplicate of this ticket.

[sundaymorning]

It seems that gtalk invisibility discussion should be done here. That's my interpretation after #9392 have been marked as a duplicate of this bug.

I present you with a less-than-perfect patch. It has been working fine for my needs and it could benefit others, that's why I'm uploading it. I don't think it should be integrated into the mainstream code because it's hacky and doesn't give the user "strong" invisibility. Feedback and suggestions are welcome, though.

My patch allows the user to be invisible while still being able to see who logs in and out, send messages and receive messages while invisible. The invisibility is not strong in the sense that someone who's really interested into knowing if you're online is able to find that out.

Google's implementation of invisibility also allows for someone who really wants to find out if you are invisible to do so. Therefore, I think my patch is no worse than google implementation in that sense.

[E1]

Hi, sundaymorning!

Would you be kind enough to tell us how we can use this patch of yours? I for one have no idea of how to compile nor have the capability and necessary instruments to do it. A simple instruction perhaps (or the link to the finish good) would be brilliant.

TIA, E1

[darkrain42]

Replying to sundaymorning:

It seems that gtalk invisibility discussion should be done here. That's my interpretation after #9392 have been marked as a duplicate of this bug. I present you with a less-than-perfect patch. It has been working fine for my needs and it could benefit others, that's why I'm uploading it. I don't think it should be integrated into the mainstream code because it's hacky and doesn't give the user "strong" invisibility. Feedback and suggestions are welcome, though. My patch allows the user to be invisible while still being able to see who logs in and out, send messages and receive messages while invisible. The invisibility is not strong in the sense that someone who's really interested into knowing if you're online is able to find that out. Google's implementation of invisibility also allows for someone who really wants to find out if you are invisible to do so. Therefore, I think my patch is no worse than google implementation in that sense.

This patch is worse than Google's implementation, to me. The polling is unacceptably bad, especially since it's doing it every 15 seconds.

I'm also worried that it will break of its own accord due to the hackery of sending <message><active/></message> in order for messages to be receivable (if Google changes the way their servers handle this situation).

Presence subscription requests don't seem to be sent to a resource who has "gone invisible"; I didn't test IQ stanzas of any sort, nor messages sent to the bare JID.

There is also a race condition between when someone signs on and when they're able to send you a message (which is of course then also tied to the polling frequency, which is already too fast, so this would simply be made worse by decreasing the polling frequency).

You should also be using the GHashTable js->buddies, not purple_find_buddies. The former has one entry per person, the latter may contain duplicates if a buddy is in multiple groups.

Edited: forgot to finish a sentence.

[bernmeister]

Should this be marked as Patches Needing Improvement?

[deryni]

Not to mention that constantly sending presence probes sort of defeats the entire purpose of invisibility in that everyone knows you are there.

A patch implementing xep-0186 would be interesting and would at least be on the right path for being acceptable. Very little else will be.

[sundaymorning]

My last patch was a very bad hack as pointed out by darkrain. I have studied a little more the jabber protocol and in particular google extensions to it and I wrote this new patch (I attached it to the ticket replacing the old one) wich works really well. I'm confident that google's client works in the same way as my patch.

It's not a full implementation on google shared status, but it does enough as to support invisibility (which is the only feature of shared-status I really care about). I think it may need some polishing by someone more experienced with pidgin but I think this new patch could make it to pidgin's mainstream version.

This is not http://xmpp.org/extensions/xep-0186.html but it implements google's approach on it. I'm not sure if there are jabber server implementing xep 0186 nowadays, but google server sure doesn't.

[deryni]

Enabling the shared status stuff means that pidgin status changes will not be seen, as the server will rewrite all presence stanzas (at least according to the documentation). Which means it will become impossible to set a status message for the Google Talk account from pidgin, since that will get rewritten by the server to the current shared message (why Google Talk only overwrites the message and not the type I have no idea, but that's what it looks like).

Also, it looks like this patch is going to break anyone who actually uses the shared status stuff because according to the docs when you set a shared status you need to push the status-list settings you last received or you will clear them. What exactly those lists do I'm not at all sure as the docs don't really say. They seem to indicate they are a list of messages associated with the status but don't indicate how (or when) they get used or why there are multiple entries.

Additionally, by way of patch specific comments. Manually setting <iq> stanza id:s is a very bad idea, you shouldn't do it. It isn't clear to me why you request the current shared status settings to then ignore (and remove) most of them when you set them in the callback handler. And lastly, jabber_send_status_request doesn't appear to be used at all.

[kk2]

the patch doesn't work anymore in pidgin 2.6.1

[ayanc]

I have a patch (to the pidgin 2.6.3 source) that implements the google shared-status system in google talk, which is what supports the "invisible" mode in the google talk client. The patch will also honor messages from the server asking the client to update its status ---- so now if you change your status on the gtalk client, this will be reflected in your pidgin status.

The uploaded file patches both libpurple, and to a minor extent the pidgin UI code. I think the libpurple patch is reasonably clean --- it switches to shared-status if the the XMPP server returns "google:shared-status" as one of its features, so it should not affect non-gtalk based jabber accounts.

The patch for the pidgin code is just an addition to the handler that is supposed to update the status box when an account's status changes without user initiation (in our case, when the gtalk server sends a status update message). It still doesn't seem to always update the actual text away message in the status box (only the mode: available/Busy/invisible) --- I'm not completely sure how the UI source works, so I haven't tried fiddling with it too much.

Ayan Chakrabarti [ayan.chakrabarti [at] gmail] ---

If you're new to this sort of thing, here's how to apply the patch:

1. Untar the source in some directory, and switch to it

$ tar xjvf pidgin-2.6.3.tar.bz2

$ cd pidgin-2.6.3

2. Apply the two patch files (unzip from attachment)

$ cat /path/to/patches/gtalk-sharedstatus.patch | patch -p1

3. ./configure, make, make install as usual.

[ayanc]

Patch for pidgin-2.6.3 to support google shared-status

[zx2c4]

@ayanc Thanks for making this.

Any chance of rebasing this for 2.6.4 and including it in the release of 2.6.5?

[ayanc]

@zx2c4: Including it in 2.6.5 isn't up to me :). The patch is for 2.6.3 because that's what I currently use.

I'll install monotone sometime soon, rebase this for the latest version and submit an "official" patch ticket.

[darkrain42]

Ticket #11061 has been marked as a duplicate of this ticket.

[QuLogic]

Ticket #11094 has been marked as a duplicate of this ticket.

[dlenski]

@ayanc,

Fantastic patch :-) I hope that we'll have proper Google Talk shared status support in Pidgin soon, thanks to you.

By the way, I got the patch to apply to 2.6.5 with very minimal changes. I built binary packages for Ubuntu, and they work just great with shared status and invisibility working correctly.

[dlenski]

@ayanc, One small issue is that the tray icon doesn't update to reflect the status change. This is probably a simple fix to the UI code...

[ayanc]

@dlenski: Thanks, you should put up a link to the binary packages here. I've had some people using Ubuntu e-mailing me, and unfortunately I haven't been able to help them much since I use Gentoo.

Sorry about the tray icon --- the UI generally does not seem to expect the status to be reset from the server, and therefore the pidgin code (as opposed to the libpurple code) doesn't always update everything correctly. I put in a fix to the status bar code, but since I don't use the tray icon (I have ratpoison as my window manager :) ), I missed that.

I'm not really sure whether there are any plans by the pidgin devs to include this patch (or shared status support in general). Going by the comments in

http://developer.pidgin.im/ticket/5828

it appears that there aren't :). If I upgrade to a later version of pidgin, I'll update the patch for that version and put it up here as well.

[dlenski]

@ayanc: Yeah, I know the devs have been hostile to Shared Status support in the past... though I don't really understand why since your patch implements it nicely without interfering with XMPP servers that don't support it.

I haven't been able to figure out how to get the tray icon to update... there's a public routine pidgin_docklet_update_icon() in pidgin/gtkdocklet.c, but it doesn't seem to do what I want.

[QuLogic]

Ticket #11433 has been marked as a duplicate of this ticket.

[moxfyre]

Patch for pidgin-2.6.6 to support google shared-status

[moxfyre]

@ayanc, I built packages for Pidgin 2.6.6 (for Ubuntu 9.10 karmic) and uploaded them to Launchpad: https://launchpad.net/~lenski/+archive/pidgin-gss

Anyone should be able to install these in Ubuntu now.

I've also attached the slightly updated patch that applies cleanly to Pidgin 2.6.6.

Still haven't worked out the tray icon update bug, though.

[deryni]

Our reason for disliking Google's shared status stuff has never been because we thought it couldn't be done without causing problems for other XMPP servers, it has been because we dislike the model it uses. It removes one of the main benefits to being able to be logged in from multiple locations which is that they can have different statuses. It means the status you see in the status selector may or may not be accurate (and that's not something that can be fixed when multiple accounts are enabled). Etc.

Do you all actually want shared status support in pidgin or do you just want invisibility support? Because despite the way Google has them implemented they really are two different things. We have no problem support invisibility (though we think it is a stupid feature) it is the shared status stuff we have no desire to support.

[js]

This bug report has been terribly hijacked (and I get quite spammed with mails about a GTalk issue which does not interest me the least). Can you please create a new bug report for the GTalk stuff? As this bug report has nothing to do with GTalk at all. It's about either implementing XMPP invisibility (read _XMPP_ Invisibility, NOT GTalk invisibility, which is something else!!) or settings the status back to online when connected with multiple protocols and invisible is chosen as the global status.

[deryni]

Google Talk Shared Status ticket (at least for the time being): #11433

[moxfyre]

@deryni, as I understand it, the way that Google Chat supports invisibility is *NOT* separate from Shared Status. The correct way to signal invisibility to Google Talk is via a shared status settings:

<iq type='get' to='gmail.com'> <query xmlns='http://jabber.org/protocol/disco#info'/> </iq>
<iq type='get' id='ss-1' to='me@gmail.com'> <query xmlns='google:shared-status' version='2'/> </iq>
<iq type='set' id='ss-1' to='me@gmail.com'> <query xmlns='google:shared-status' version='2'><invisible value='true'/></query> </iq>

I know there are some hacky ways to emulate it, such as by simply sending out "unavailable" presence messages, but those don't work very well.

So, unfortunately, I don't think there is a way to get Google Talk invisibility without Shared Status support. I'd be happy to be proved wrong, though.

And I agree that Shared Status is not ideal for a lot of users. Perhaps it could be a configurable option?

[ayanc]

@deryni: I actually find both invisibility (which enables me to continue IMing with one contact, without being disturbed by others --- when the invisible option exists, people tend to treat the DND status as little more than a casual suggestion) as well as shared status (which allows me to leave clients running from multiple machines, but still allows me to set my global status from the one I am actually on) to be useful. Also, as moxfyre points out, there is no way to implement just invisibility on google talk without supporting shared status as well.

This is of course a personal opinion. And I understand that you dislike the shared status model. Having said that, given that pidgin has worked hard to support a wide variety of protocols many of which probably have far worse design choices, I'm surprised that the dev team would be unwilling to support all the features in google talk which is fairly popular client. The current patch already has no effect on non-google talk jabber users. And if you feel there are people who want to use google talk but not google's shared status system, making it a configurable option as moxfyre suggested may be best.

Of course, at the end of the day this is your decision.

[onlineapps]

@ayanc: +1 for the invisibility v. dnd. dnd is mostly ignored.

[js]

"The current patch already has no effect on non-google talk jabber users." Which is exactly why this is the wrong bug report...

[darkrain42]

onlineapps and ayanc: I suggest a LART (a.k.a. a clue-by-four). In all seriousness, though, the GMail Gtalk interface interprets 'DND' (as set by Pidgin) as "Away", and "Away" (as set by Pidgin) as Idle, which is perhaps where the confusion comes from (and I know people on my buddy list typically don't use "DND" to mean Do Not Disturb, so when I see DnD, my initial assumption is that they're using the Gmail client.

Anyway, move this discussion to the other ticket, please.

[moxfyre]

Okay, I'm moving the discussion to #11433... posted there.

[zx2c4]

Would the author of that patch rebase it against the current HEAD?