python: heatmap with categorical color and continuous transparency

I want to make a heatmap in python (seaborn, matplotlib, etc) with two dimensions of information. I have a categorical value I want to assign to color, and a continuous variable (i.e. between 0-100 or 0-1) I want to assign to transparency, so each box has its own color and transparency (or intensity).

for example:

colors = pd.DataFrame([['b','g','r'],['black','orange','purple'],['r','yellow','white']])
transparency = pd.DataFrame([[0.1,0.2,0.3],[0.9,0.1,0.2],[0.1,0.6,0.3]])

how can I make a heatmap from this data such that the top left box is blue in color and 10% transparency (or 10% opaqueness, whichever), and so on?

The best idea I have so far is to turn the colors into integer values, add those to the transparency values, and then make a custom colormap where each integer has a different color, ranging from white to the color in between the integer values. That sounds complicated to make and I'm hoping there's a built-in way to do this.

Any ideas?

You could draw individual rectangles, giving each a specific color and transparency:

import matplotlib.pyplot as plt
from matplotlib.patches import Rectangle, Patch
import pandas as pd

colors = pd.DataFrame([['b', 'g', 'r'], ['black', 'orange', 'purple'], ['r', 'yellow', 'white']])
transparency = pd.DataFrame([[0.1, 0.2, 0.3], [0.9, 0.1, 0.2], [0.1, 0.6, 0.3]])

fig, ax = plt.subplots()

for i, (color_col, transp_col) in enumerate(zip(colors.columns, transparency.columns)):
    for j, (color, transp) in enumerate(zip(colors[color_col], transparency[transp_col])):
        ax.add_patch(Rectangle((i - 0.5, j - 0.5), 1, 1,
                               facecolor=color, alpha=transp, edgecolor='none', lw=0))

ax.invert_yaxis()  # start at the top
ax.autoscale(enable=True, tight=True)  # recalculate axis limits
ax.set_xticks(range(len(colors.columns)), colors.columns)
ax.set_yticks(range(len(colors.index)), colors.index)
plt.show()