NTLM proxy authorization broken

[thecrux]

we have a proxy server Squid with ntlm authorization.

1. It seems that pidgin cant authorizate via proxy, because of wrong NTLM flags used in NTLMSSP header that pidgin sent to proxy.

From http://davenport.sourceforge.net/ntlm.html#theNtlmFlags : Negotiate Unicode (0x00000001) - The client sets this flag to indicate that it supports Unicode strings.

This flag is set to 0 in all NTLMSSP headers, that pidgin send. So proxy read all strings as a 8bit strings, but they are unicode.

2. Also, pidgin send wrong workstation name (insead of hostname it used proxy name).

3. On windows platform pidgin can use SSPI service to transparently authorizate user (dont need to ask username and password) http://davenport.sourceforge.net/ntlm.html#ntlmsspAndSspi

[thecrux]

(quick hack) fix ntlm auth and also add transparent ntlm auth for windows build

[datallah]

I'd like to get this into 2.2.0.

It seems to me that it should only be using the SSPI on windows when using "environmental" proxy settings; if there is a specific configuration specified, it should use that instead.

[thecrux]

Replying to datallah:

It seems to me that it should only be using the SSPI on windows when using "environmental" proxy settings; if there is a specific configuration specified, it should use that instead.

Ok. I modified patch.

But there is a issue with authorization: pidgin send both Basic and NTLM auth scheme header without knowing what type of authorization proxy required. This is not good. Firstly pidgin must send request without auth stuff, and after discover what type of authorization scheme supported by proxy choose a most stronger type. This is not simple to implement, because proxy close connection after sending 407 error and we must init new connection with correct proxy-authorization header. Design of code in libpurple/proxy.c does not provide several connections, all authorization negotiation must be within only one connection session.

For example, if proxy server squid configured to accept only ntlm auth, it will write error in logs about unknown basic auth and will not accept connection (even if NTLM auth header exist in pidgins request, but set after basic).

[thecrux]

environmental sspi patch

[datallah]

I'm not the most knowledgeable about proxy stuff, but RFC 2617 Section 1.2 states:

A client that wishes to authenticate itself with a proxy--usually, but not necessarily, after receiving a 407 (Proxy Authentication Required)--MAY do so by including a Proxy-Authorization header field with the request.

To me, this means that it is acceptable to include the Proxy-Authorization headers at the time libpurple does. It isn't clear whether or not it is acceptable to include multiple Proxy-Authorization headers.

[datallah]

I'm going to include parts 1 and 2 in 2.2.0, I haven't been able to adequately evaluate part 3 and I want to hear more about the issue discussed in comment 2.

[datallah]

If you'd like to be in the COPYRIGHT file, let me know what name to use.

[datallah@…]

(In [2510b3bdf51be7bd339aca4a39ec86bf73c6716c]) This is parts 1 and 2 of patch #2910 from Vladimir (thecrux) to improve NTLM proxy handling. Specifically, it specifies that we're using Unicode strings and uses the hostname of the local machine instead of incorrectly using the proxy server's hostname for authentication. References #2910.

[thecrux]

Replying to datallah:

I'm going to include parts 1 and 2 in 2.2.0, I haven't been able to adequately evaluate part 3 and I want to hear more about the issue discussed in comment 2.

Proxy server squid do not understand multiple Proxy-Authorization (p-a) headers: it's using only first header. libpurple's first p-a header - Basic. So, if proxy not accept basic auth it will reject access. Again, libpurple must discover what type of auth accepted by proxy and send only ONE p-a header with most stronger type of auth.

[igitur]

I'm experiencing exactly the same problem reported by thecrux.

Even though I setup my proxy host, port, username and password correctly, the proxy server returns:

HTTP/1.1 407 Proxy Authentication Required ( The ISA Server requires authorization to fulfill the request. Access to the Web Proxy filter is denied.  )
Proxy-Authenticate: Negotiate
Proxy-Authenticate: Kerberos
Proxy-Authenticate: NTLM

I verified that Pidgin sends:

CONNECT server:port HTTP/1.1
Proxy-Authorization: Basic RTkxMsometextXNzODM=
Proxy-Authorization: NTLM TlRMTVNTUAABAAsometextCAAIACAAAABYMDAxNzc4NQ==

As stated, it seems the proxy server recognises only the first authorisation header and ignores the NTLM authorisation.

How about an option in the GUI for explicitly selecting the authorisation method? Include an option for "All methods", which would emulate the current behaviour.

[thecrux]

last revision of patch

[oblio]

Bug confirmed for Pidgin 2.4.0, Windows XP, Squid with NTLM authentification. Pidgin still sends the wrong header and can't be authenticated by the proxy.

Maybe the workaround with the GUI option could be implemented, instead of the more complex option of detecting authentification type used by the proxy.

This basically makes Pidgin unusable in all sorts of corporate networks :(

[datallah]

There are two issues here:

1. (some?) proxy servers don't like multiple Proxy-Authorization headers
2. libpurple doesn't support using the current kerberos ticket to authenticate to the proxy server.

I'd like to get the current patch updated to deal with only the second issue to avoid complicating the situation. I'd like the win32-only code to be put in a new file in the win32 subdirectory and namespaced accordingly to avoid cluttering the core ntlm stuff.

IRT the first issue, we need to detect what the server supports and only send that, a GUI option isn't going to be an acceptable thing to do. Presumably most clients aren't sending multiple headers and we probably shouldn't either.

[jgardner]

I have identified this bug while using Microsoft ISA 2006 as the proxy server. When trying to authenticate to the proxy server, it is denied when provided valid credentials.

Is there any possibility of this being fixed in future releases? TIA

[darkrain42]

Ticket #10329 has been marked as a duplicate of this ticket.

[mr_jrt]

Just a "me too". My workplace has recently switched to ISA 2006 with NTLM authentication, and Pigin is no longer usable with MSN via HTTP.

[deryni]

Ticket #10615 has been marked as a duplicate of this ticket.

[deryni]

#10615 has packet captures of pidgin and Yahoo! if that is useful.

[dlevine278]

Any sense if this is being worked...The case has been open for 2yrs. This issue could prevent Pidgin from being rolled out to literally thousands of individuals within corporations using proxies supporting NTLM authentication.

[deryni]

The last patch that was submitted had issues that were requested to be fixed, the original author has not done that and no one else has stepped up to do it either. I'm not sure any of the pidgin developers currently have NTLM proxies with which to test this (which makes fixing the patch up and making sure it still works problematic). If someone who has such a proxy wishes to address the comments datallah made about the patch and submit a new version it will likely be accepted (assuming it looks reasonably correct).

[thecrux]

Replying to deryni:

The last patch that was submitted had issues that were requested to be fixed, the original author has not done that and no one else has stepped up to do it either.

Issue 1: proxy servers don't like multiple Proxy-Authorization headers

To fix this issue we must not send any Proxy-Authorization headers until recieve 407 Error from proxy.

Issue 2: libpurple doesn't support using the current kerberos ticket to authenticate to the proxy server.

patch was not about kerberos. I cant fix that. Just propose to use SSPI service on win platform to transparently authorizate user.

[thecrux]

do not send any Proxy-Authorization headers until recieve 407 error from proxy

[deryni]

Assuming I understand the proxy code or the current problem at all that looks like it would solve it, since we should respond to an auth failure with the right auth response and the removed code was just an attempt to remove a round-trip or two. Setting to Patches Needing Review so someone who feels more comfortable with this code can take a look.

[thecrux@…]

(In [32e63a51dbb65b8b63d134bde098d40167574148]):
Don't send Proxy-Authorization headers until we've received a 407 Proxy Authentication Required response from the server. Fixes #2910, which has an additional component about using existing kerberos tickets that should be moved to a new trac ticket.

[rekkanoryo@…]

(In [d6f80b7ba5ba2da4ae203eac6f07565721b10ad8]):
ChangeLog the Proxy-Authorization thing. Refs #2910. Need a proper name from thecrux to credit.

[darkrain42]

This change has caused at least two reported issues so far, where the debug log looks something like the following:

(00:41:18) proxy: Attempting connection to IP
(00:41:18) proxy: Connecting to talk.google.com:5223 via IP:3128 using HTTP
(00:41:18) proxy: Connection in progress
(00:41:18) proxy: Connected to talk.google.com:5223.
(00:41:18) proxy: Using CONNECT tunneling for talk.google.com:5223
(00:41:18) proxy: Proxy server replied with:
HTTP/1.0 407 Proxy Authentication Required

Server: squid

Mime-Version: 1.0

Date: Thu, 03 Dec 2009 04:34:35 GMT

Content-Type: text/html

Content-Length: 1674

Expires: Thu, 03 Dec 2009 04:34:35 GMT

X-Squid-Error: ERR_CACHE_ACCESS_DENIED 0

Proxy-Authenticate: Basic realm="Enter Your Username and password"

X-Cache: MISS from proxyserver

Via: 1.0 proxyserver (squid)

Proxy-Connection: close
(00:41:18) proxy: Connection attempt failed: Server closed the connection
(00:41:18) connection: Connection error on 034EB160 (reason: 0 description: SSL Connection Failed)

One of the users is going to file a ticket.

[darkrain42@…]

(In [d612071593881b9ae7e92a1820cf13d3e45b1f94]):
disapproval of revision 'd6f80b7ba5ba2da4ae203eac6f07565721b10ad8'

This change has produced regressions when using HTTP/1.0, as the proxy closes the connection, which is treated as a connection failure. Refs #10880, #10856 for examples. Refs #2910

[darkrain42@…]

(In [a35d515dd2c8f385ed4563358fccee9108573018]):
disapproval of revision '32e63a51dbb65b8b63d134bde098d40167574148'

This change has produced regressions when using HTTP/1.0, as the proxy closes the connection, which is treated as a connection failure. Closes #10880, #10856 for examples. Refs #2910.

[thecrux]

Replying to darkrain42@…:

(In [d612071593881b9ae7e92a1820cf13d3e45b1f94]):
disapproval of revision 'd6f80b7ba5ba2da4ae203eac6f07565721b10ad8' This change has produced regressions when using HTTP/1.0, as the proxy closes the connection, which is treated as a connection failure. Refs #10880, #10856 for examples. Refs #2910

Sorry for that. Patch was not tested carefully... I will attach another patch (against 2.6.4) to fix this issue.

Patch description:

• add field of type PurpleProxyAuth? in struct PurpleProxyInfo? to save type of authorization on proxy (unknown, basic, ntlm)
• when receive 407 code from proxy save type of authorization from "Proxy-Authenticate" header to this field
• disconnect from proxy, initialte new connection to the same proxy host and sent correct Proxy-Authorization header
• if specified wrong credentials - disconnect from proxy finnaly

Some details:

• do not use g_slist_remove() in function try_connect() and dont free addr (we need address of proxy for the next connection)
• move g_slist_remove() to function purple_proxy_connect_data_disconnect()
• dont free elements of GSlist in purple_proxy_connect_data_destroy() (pidgin crash on free, may be because g_slist_remove() free it before us)

Patched pidgin was tested with proxy server squid with NTLM auth, and on another proxy with Basic auth. All works fine (wireshark's dumps also show that schema works correctly).

[thecrux]

fix proxy auth

[duke87]

Replying to darkrain42: hi how to apply these patches ??? where to add

[ledi]

Replying to duke87:

Replying to darkrain42: hi how to apply these patches ??? where to add

Same here, I'm willing to test the patch, but I'm a n00b in patching Pdgin (sorry) A patched .exe is not available ?

[thecrux]

Replying to ledi:

Same here, I'm willing to test the patch, but I'm a n00b in patching Pdgin (sorry) A patched .exe is not available ?

If you want to test last patch i've submitted you can download patched pidgin here: http://xrt.no-ip.org/pidgin/pidgin-2.6.4-fix2910.exe (warning, this is not official build)

p.s. Simple instruction to build pidgin on windows http://developer.pidgin.im/wiki/BuildingWinPidgin

[datallah@…]

(In [583c1ae15e7cbb18ba5a65705fff249b5eda32c7]):
*** Plucked a35d515dd2c8f385ed4563358fccee9108573018 (f05c54b03e6bbfdbff38c01697fbd353a969e05e): disapproval of revision '32e63a51dbb65b8b63d134bde098d40167574148'

This change has produced regressions when using HTTP/1.0, as the proxy closes the connection, which is treated as a connection failure. Closes #10880, #10856 for examples. Refs #2910.

[dlevine278]

Want to confirm that NTLM proxy support has yet to be included in the official release? It's hard to tell for sure from the comments.

Thanks in advance,

Dave

[dlevine278]

Is there someone that could answer the previous question?

[darkrain42]

Replying to dlevine278:

Want to confirm that NTLM proxy support has yet to be included in the official release? It's hard to tell for sure from the comments. Thanks in advance, Dave

Correct. A broken version was included in one release, but immediately backed out for the next release because it was broken.