Occasionally over the past few years, I have received reports from users of both my xearth and threat monitor programs that they don't work from behind certain kinds of firewalls. Not having this particular problem myself, I had never been able to track down the actual problem.

It appeared that I was doing everything correctly. I was using the Windows WinInet library, which is supposed to be the same communications layer that Internet Explorer itself uses. If Internet Explorer was working, so should my programs. In most cases, this was true - even if the user had specific proxy settings configured in IE, xearth would follow suit and use those same proxy settings. Except in some cases.

When I started my new job here in New Zealand, I quickly noticed (hey, xearth is required desktop software!) that I was having the same connection problem from behind the corporate firewall. Now that I could see the problem firsthand, and the circumstances surrounding it, it didn't take long to realise what the problem was.

It turns out that proxy servers that require authentication can use various different kinds of authentication. Microsoft's own proxy server can be configured to require NTLM authentication, but because this is a challenge-response protocol that requires a bidirectional conversation over HTTP, simple one-at-a-time HTTP won't work. The key is that the INTERNET_FLAG_KEEP_CONNECTION flag must be passed to the InternetOpenUrl function. The flag description in the documentation does say it's required for "Microsoft Network (MSN), NTLM, and other types of authentication", but it fails to mention that the user may have also set up a proxy server that requires this type of authentication too. So, the INTERNET_FLAG_KEEP_CONNECTION flag should be passed to all calls to InternetOpenUrl.

Anyway, I'm glad to have sorted this out, after who knows how many years.