Overlay a polar matplotlib axis over a geopandas map

I would like to overlay a polar axis over a map of a GeoPandas GeoDataFrame centered at a specific point. The ultimate goal is for the plot on the polar axis to be exactly centered at the point. I have found a way to do this but it is hacky and I wonder if there is a better way.

The problem is that the centers of the geopandas plot and the polar axes are slightly offset. For example, this code:

import geopandas as gpd

lat, lon = 37, -122
gdf = gpd.GeoDataFrame({'id': ['my_point']}, geometry=gpd.points_from_xy([lon], [lat], crs=4326))

fig, ax = plt.subplots()
gdf.plot(ax=ax)
ax.set_axis_off()

polar_ax=fig.add_axes(rect=[0, 0, 1, 1], polar=True, frameon=False)

Yields this figure:

Notice how the point is slightly off the origin of the polar axes.

If I finagle the rect argument in fig.add_axes, I can get the two to line up:

import geopandas as gpd

lat, lon = 37, -122
gdf = gpd.GeoDataFrame({'id': ['my_point']}, geometry=gpd.points_from_xy([lon], [lat], crs=4326))

fig, ax = plt.subplots()
gdf.plot(ax=ax)

polar_ax=fig.add_axes(rect=[0, 0, 1.025, 0.99], polar=True, frameon=False)

My questions: Is there a better way to do this (i.e., one that will automatically adjust the polar axes so they are centered at the same location as the geopandas plot)? Will this "adjustment factor" for the rect argument work universally, or does it depend on the particulars of the geodataframe I'm plotting?

To complete @swatchai's answer, I got it working by first creating the polar plot, drawing the canvas once to set the extents and boundaries of the figure and then setting the position of polar axis :

import geopandas as gpd
import matplotlib.pyplot as plt

lat, lon = 37, -122
gdf = gpd.GeoDataFrame({'id': ['my_point']}, geometry=gpd.points_from_xy([lon], [lat], crs=4326))

fig, ax = plt.subplots()
gdf.plot(ax=ax)
ax.set_axis_off()

# Create the polar axis
polar_size = 0.8
polar_ax = fig.add_axes([0, 0, polar_size, polar_size], polar=True, frameon=False)

# Draw the figure once to set limits and extents
fig.canvas.draw()

# Transform data coordinates to display coordinates
display_x, display_y = ax.transData.transform((lon, lat))

# Transform display coordinates to figure coordinates
fig_x, fig_y = fig.transFigure.inverted().transform((display_x, display_y))

# Calculate new position for the polar axis
width = polar_size
height = polar_size
left = fig_x - width / 2
bottom = fig_y - height / 2

# Apply the new position
polar_ax.set_position([left, bottom, width, height])

plt.show()