degree symbol incorrect in map axis labels

If I create maps using geom_sf, the axis labels have the wrong symbol for degrees. I get degree symbols that are vertically centred in the text, rather than raised like superscipts.

For example,

library(sf)                                            
library(ggplot2)                                       
nc = st_read(system.file("shape/nc.shp", package="sf"))

ggplot() +                                             
  geom_sf(data = nc) +
  theme(axis.text = element_text(size=16))

When I see examples online, they typically look correct (e.g. image below, copied from here), so I guess it is related to something in my local setup.

I have tried changing font, using library(extrafont) but this problem remains in every font I tried.

Update

I don't think this is a ggplot-specific issue, as I get the same thing with any graphics that uses the degree keyword in plotmath. For example

par(mar=c(0,0,0,0))
plot.new()
text(0.5,0.5, bquote(120*degree*N), cex=5)

I am on Linux (Kubuntu 19.04), R 3.5.2, ggplot2 v. 3.2.1, sf v. 0.7-7. Not sure what other information might be relevant, but I can update answer with anything else that is requested.

Finally managed to track down the answer:

From ?X11 it says:

Problems with incorrect rendering of symbols (e.g., of quote(pi) and expression(10^degree)) have been seen on Linux systems which have the Wine symbol font installed – fontconfig then prefers this and misinterprets its encoding. Adding the following lines to ‘~/.fonts.conf’ or ‘/etc/fonts/local.conf’ may circumvent this problem by preferring the URW Type 1 symbol font.

<fontconfig>
<match target="pattern">
  <test name="family"><string>Symbol</string></test>
  <edit name="family" mode="prepend" binding="same">
    <string>Standard Symbols L</string>
  </edit>
</match>
</fontconfig>

Adding these lines to /etc/fonts/local.conf solved the problem for me.