Plotting Contour plot for a dataframe with x axis as date and y axis as time and values as a parameter

I have a dataframe with the indexes as datetime and columns as Radiation. I am trying to plot a contour plot of radiation data with respect to date as x-axis time as y-axis. Contourf won't take the date time as it is. I have extracted the date and time from the datetime column and then tried to plot using contourf.

data['DateTime'] = pd.to_datetime(data['DateTime'], format='%d-%m-%Y %H:%M')
data['Date'] = data['DateTime'].dt.date
data['Time'] = data['DateTime'].dt.time
fig, ax = plt.subplots()
ax.contourf(data['Date'], data['Time'],data['SW(W/m2)'])

It's showing an error as: TypeError: float() argument must be a string or a real number, not 'datetime.time'

Any ideas on how I should solve this error? Sample file

I created a sample dataframe for this, so replace with your own data:

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import matplotlib.dates as mdates
from datetime import datetime, timedelta

dates = pd.date_range(start='2024-01-01', end='2024-01-07', freq='H')
radiation = np.random.rand(len(dates)) * 1000  

data = pd.DataFrame({
    'DateTime': dates,
    'SW(W/m2)': radiation
})

data['Date'] = data['DateTime'].dt.date
data['Time'] = data['DateTime'].dt.time

data['DateOrdinal'] = data['DateTime'].apply(lambda x: x.toordinal())
data['TimeSeconds'] = data['DateTime'].apply(lambda x: x.hour * 3600 + x.minute * 60 + x.second)

date_values = np.unique(data['DateOrdinal'])
time_values = np.unique(data['TimeSeconds'])

date_grid, time_grid = np.meshgrid(date_values, time_values)
radiation_grid = np.zeros_like(date_grid, dtype=float)

for i, date in enumerate(date_values):
    for j, time in enumerate(time_values):
        radiation_value = data[(data['DateOrdinal'] == date) & (data['TimeSeconds'] == time)]['SW(W/m2)']
        if not radiation_value.empty:
            radiation_grid[j, i] = radiation_value.values[0]

fig, ax = plt.subplots()
contour = ax.contourf(date_grid, time_grid, radiation_grid, cmap='viridis')

date_ticks = pd.date_range(start='2024-01-01', end='2024-01-07', freq='D')
ax.set_xticks([dt.toordinal() for dt in date_ticks])
ax.set_xticklabels([dt.strftime('%Y-%m-%d') for dt in date_ticks])

time_ticks = [0, 6*3600, 12*3600, 18*3600, 24*3600]
time_labels = ['00:00', '06:00', '12:00', '18:00', '24:00']
ax.set_yticks(time_ticks)
ax.set_yticklabels(time_labels)

plt.colorbar(contour, ax=ax, label='Radiation (W/m²)')
ax.set_xlabel('Date')
ax.set_ylabel('Time')
ax.set_title('Radiation Contour Plot')
plt.xticks(rotation=45)
plt.show()

which gives