I am trying to use jakarta commons http client. Doing kerberos authentication to communicate with a server. Authentication always fails. On digging deeper I found out that the kerberos token header has carriage return line feed characters in it which is the root cause of the issue. Why does it have \r\n characters and why is that an issue?
\r\n characters in the kerberos token is an issue because \r is an HTTP control characters. That essentially means that the server only sees first line of the token and ignores rest hence failing authentication. To make it worse, the next likes of the token are treated as new headers and will not be formatted well. So http server will freak out.
The reason this happens is because RFC 1521 specification says that base64 encoding line length is limited at 76 characters. Hence \r\n characters. And this does not work well with HTTP protocol. This surfaces only if you use Kerberos tokens that are base64 encoded.
So solution to this is to strip out \r\n characters from the kerberos base64 encoded token header. Also, older versions of apache commons codec will not limit base64 encoding to 76 lines and hence not an issue. So if you have hook into encoding, strip out the problematic characters. If you don't then use older version of apache commons code with commons-httpclient.