How can I escape history expansion exclamation mark ! inside a double quoted string?

The command substitution is not necessary for the surprising behavior, although it is the most common use case. The same question applies to just echo "'!b'".

b=a

# Enable history substitution.
# This option is on by default on interactive shells.
set -H

echo '!b'
# Output: '!b'
# OK. Escaped by single quotes.

echo $(echo '!b')
# Output: '!b'
# OK. Escaped by single quotes.

echo "$(echo '$b')"
# Output: '$b'
# OK. Escaped by single quotes.

echo "$(echo '!b')"
# Output: history expands
# BAD!! WHY??

• In the last example, what is the best way to escape the !?
• Why was it not escaped even if I used single quotes, while echo "$(echo '$b')" was? What is the difference between ! and $?
• Why was does echo $(echo '!b') (no quotes) work? (pointed by @MBlanc).

I would prefer to do this without:

• set +H as I would need set -H afterwards to maintain shell state

• backslash escapes, because I need one for every ! and it has to be outside the quotes:

    echo "$(echo '\!a')"
    # '\!a'.
    # No good.
  
    echo "$(echo 'a '\!' b '\!' c')"
    # a ! b ! c
    # Good, but verbose.
  

• echo $(echo '!b') (no quotes), because the command could return spaces.

Version:

bash --version | head -n1
# GNU bash, version 4.2.25(1)-release (i686-pc-linux-gnu)

In your last example,

echo "$(echo '!b')"

the exclamation point is not single-quoted. Because history expansion occurs so early in the parsing process, the single quotes are just part of the double-quoted string; the parser hasn't recognized the command substitution yet to establish a new context where the single quotes would be quoting operators.

To fix, you'll have to temporarily turn off history expansion:

set +H
echo "$(echo '!b')"
set -H