Fast plotly updating inside Jupyter

I am running a Jupyter notebook inside VSCode. I want to use plotly (or anything else, but I am trying this as a potential solution) to dynamically plot data as it is generated.

I tried to generate a minimum working example of what I want. In the following, I generate some fake data and then update the plotly figure to simulate that data being generated in real time.

In the code below, the first cell should be run and the figure allowed to draw before the second cell is run (I don't know why these steps have to be split, but the figure does not appear if all the code is in one cell). The second cell does the thing I am most interested in.

What I see is that the time taken to update is extremely slow - around 2-3 seconds for each 'tval' loop iteration. If I reduce 'ntraces' to 1, then it becomes much faster, suggesting that part of the bottleneck is having to iterate through each of the traces individually to update the data. It seems that if there was a way to update a 2D array, rather than a number of separate 1D arrays, it might be quicker.

If there are ideas of how to speed this up, or an alternative approach which is much faster, I would be very grateful.

import plotly.graph_objects as go
import numpy as np
import random

T = 20
tvals = np.arange(1, 101, 2)
noise_amp = 0.25
tau = 60
ntraces = 32
traces = np.arange(0, ntraces, 1)
tarray = np.arange(1, 51, 2)
pvalsArray = [np.array([np.sin(2*np.pi*tval/T)*np.exp(-tval/tau)+random.random()*noise_amp for tval in tarray]) for _ in traces]
fig = go.FigureWidget()

for i, ion in enumerate(traces):
    fig.add_trace(go.Scatter(x=tarray, y=pvalsArray[i][0:1], mode='lines+markers', name='lines'))
fig.show()

for j, tval in enumerate(tarray):
    for i in range(len(traces)):
        fig.data[i].x = tarray[:j]
        fig.data[i].y = pvalsArray[i][0:j]

Solution

Updating each trace individually is slow. A much faster way to update multiple traces in Plotly is to update the data for all of them at once. You can achieve this by passing a list of dictionaries to the update method of your figure and using fig.batch_update().

Pseudocode:


# populate the figure with all the traces, but with an empty initial data set.
for i, _ in enumerate(traces):
    fig.add_trace(go.Scatter(x=[], y=[], mode='lines+markers', name=f'Trace {i+1}'))

fig.show()


# time-step loop
for j in range(len(tvals)):
    # create a single list of dictionaries
    update_data = [ {'x': tvals[:j+1], 'y': pvalsArray[i][:j+1]} for i in range(ntraces) ]
    
    with fig.batch_update():
        for k in range(ntraces):
            fig.data[k].x = update_data[k]['x']
            fig.data[k].y = update_data[k]['y']

    # add a small delay if needed
    time.sleep(0.01)

The fig.batch_update() should improve the performance, we are telling Plotly.js to bundle all the changes together and send them to the front end in a single event.