Associative array not unset when nullglob is set

When I set nullglob in bash:

shopt -s nullglob

and then declare an associative array:

declare -A arr=( [x]=y )

I cannot unset specific keys within the array:

unset arr[x]
echo ${#arr[@]} # still 1

However, unsetting nullglob makes this operation work like I expect:

shopt -u nullglob
unset arr[x]
echo ${#arr[@]} # now it's 0; x has been removed

What's going on here? I don't see how shell globbing could be relevant to the situation. I've tested this on bash 4.4.19 and 5.0.0.

This can be explained by reference the the bash documentation (the man page), paraphrased here:

After word splitting, unless the -f option has been set, Bash scans each word for the characters '*', '?', and '['. If one of these characters appears, then the word is regarded as a pattern, and replaced with an alphabetically sorted list of filenames matching the pattern.

If no matching filenames are found, and the shell option nullglob is disabled, the word is left unchanged. If the nullglob option is set, and no matches are found, the word is removed.

In other words, nullglob affects what will happen to your arr[x] argument. It will either be left alone or removed.

You can see this effect by turning on echo-before-execute flag with set -x:

pax$ declare -A arr=( [x]=y )
pax$ shopt -s nullglob
pax$ set -x
pax$ unset arr[x]
+ unset

Note that this is the "word is removed" case. The "word is left unchanged" case is show thus:

pax$ shopt -u nullglob
+ shopt -u nullglob
pax$ unset arr[x]
+ unset 'arr[x]'

That final echoed command above also provides a clue as to how to delete an entry if you have enable nullglob. Just quote the argument to prevent expansion:

unset 'arr[x]'

This will work regardless of the nullglob setting, because of the section regarding quoting in the documentation:

Enclosing characters in single quotes preserves the literal value of each character within the quotes.