Flask session variable not persisting between Ajax requests

In my Flask app, I'm trying to display the progress of a backend task to the client. The client sends an initial request which begins the task. The task contains a loop, and as it iterates, it updates a session variable which contains the completion percentage. The client then regularly sends additional requests to retrieve the value of the session variable so it can be displayed.

The problem is that whenever the session variable is retrieved, it only contains its initial value of 0%. If I replace the session variable with a global variable, it works fine, but the app needs to support multiple simultaneous users, so I can't use a global variable. My understanding is that session variables are meant for this exact purpose, to persist between requests within the same session, but I must be missing something, because it's not doing so here.

Here's a simplified version of my Flask code:

@app.route("/task")
def task():
    session["load_percent"] = 0
    percent = 0
    
    for i in range(10000000):
        #actual task performed here
            
        old_percent = percent
        percent = round(i / 10000000 * 100)
        if old_percent != percent:
            session["load_percent"] = percent
            print("Actual percentage: " + str(session["load_percent"]))
            #this prints the correct value of the session variable

    return "{}"

@app.route("/get_load_percent")
def get_load_percent():
    try:
        print("Retrieved percentage: " + str(session["load_percent"]))
        #this always prints 0
        return json.dumps({"load_percent": session["load_percent"]})
    except (KeyError):
        return "{}"

And my frontend JS:

function startTask() {
    $.ajax({
        type: "GET",
        url: $('body').data('task'),
        dataType: 'json',
        success: function(response) {
            //respond to task completion here
        }
    });

    let percentageHTML = "";
    let percentageInterval = setInterval(function() {
        $.ajax({
            type: "GET",
            url: $('body').data('getloadpercent'),
            dataType: 'json',
            success: function(response) {
                percentageHTML = response["load_percent"];
                console.log(percentageHTML);
                //always prints 0
            }
        })
    }, 100);
}

I've checked similar questions, and their solutions don't work. The session variable does not exceed the size limitations, and the key is being set every time rather than having its value changed in place, so it doesn't need to be marked as modified.

Update: I did some more testing, and occasionally, the very last request for the percentage value will return 100. Additionally, setting the session variable to a completely different number at the start of task(), before the loop has started, still has it return 0.

This tells me that the cookies for the session are being sent/decoded correctly, but something is causing Flask to not read the updated session variable until the function it's updated in has finished.

I figured out the issue. Short answer is that I need to use server-side sessions using Flask-Session instead.

Long answer: I was misunderstanding exactly what the session variable was. session["load_percent"] is created locally within the function called by the request, using the session data sent with the request. It won't get updated for other functions to use until the first function returns a response with the updated session data.

Update: Flask-Session also didn't work. It still uses the same locally-created variable and from what I can tell doesn't update the actual session data until the function (and by extension, the loop) has ended. Gonna look into implementing Celery instead.