The RFC2617 says to encode the username and password to base64 but don't say what character encoding to use when creating the octets for input into the base64 algorithm.

Should I assume US-ASCII or UTF8? Or has someone settled this question somewhere already?

Original spec - RFC 2617

RFC 2617 can be read as "ISO-8859-1" or "undefined". Your choice. It's known that many servers use ISO-8859-1 (like it or not) and will fail when you send something else. So probably the only safe choice is to stick to ASCII.

For more information and a proposal to fix the situation, see the draft "An Encoding Parameter for HTTP Basic Authentication" (which formed the basis for RFC 7617).

New - RFC 7617

Since 2015 there is RFC 7617, which obsoletes RFC 2617. In contrast to the old RFC, the new RFC explicitly defines the character encoding to be used for username and password.

• The default encoding is still undefined. Is is only required to be compatible with US-ASCII (meaning it maps ASCII bytes to ASCII bytes, like UTF-8 does).
• The server can optionally send an additional authentication parameter charset="UTF-8" in its challenge, like this:
  WWW-Authenticate: Basic realm="myChosenRealm", charset="UTF-8"
  This announces that the server will accept non-ASCII characters in username / password, and that it expects them to be encoded in UTF-8 (specifically Normalization Form C). Note that only UTF-8 is allowed.

Complete version:

Read the spec. It contains additional details, such as the exact encoding procedure, and the list of Unicode codepoints that should be supported.

Browser support

As of 2024, modern browsers will default to UTF-8 if a user enters non-ASCII characters for username or password (even if the server does not use the charset parameter).

• Chrome appears to use UTF-8
• Firefox is using UTF-8 (bug 1419658)
• Internet Explorer does not use UTF-8 (issue #11879588 )

Realm

The realm parameter still only supports ASCII characters even in RFC 7617.