How to implement ChatGPT like stream response in .NET Maui

I'm learning .NET maui by implement a local chat app using phi-4 onnx model. I have finished the basic chat flow, every thing works, but currently the bot message is appeared in one time, I want to output the response char by char like ChatGPT, need help on this.

Here are my demo code:

    // handle user input
    private async void OnSendMessage(object sender, EventArgs e)
    {
        var messageText = MessageEntry.Text?.Trim();
        if (string.IsNullOrEmpty(messageText))
            return;

        var userMessage = new ChatMessage
        {
            Id = Guid.NewGuid().ToString(),
            Text = messageText,
            IsUser = true,
            Timestamp = DateTime.Now
        };

        Messages.Add(userMessage);

        MessageEntry.Text = string.Empty;

        await HandlePhi4Message(messageText);
    }

// invoke phi4 model to generate output
private async Task HandlePhi4Message(string userQ)
{
    var systemPrompt = "You are an AI assistant that helps people find information. Answer questions using a direct style. Do not share more information that the requested by the users.";

    var fullPrompt = $"<|system|>{systemPrompt}<|end|><|user|>{userQ}<|end|><|assistant|>";
    var tokens = tokenizer.Encode(fullPrompt);

    var generatorParams = new GeneratorParams(model);

    generatorParams.SetSearchOption("max_length", 2048);
    generatorParams.SetSearchOption("past_present_share_buffer", false);
    using var tokenizerStream = tokenizer.CreateStream();
    var generator = new Generator(model, generatorParams);

    generator.AppendTokens(tokens[0].ToArray());

    var responseGuid = Guid.NewGuid().ToString();
    var botMessage = new ChatMessage
    {
        Id = responseGuid,
        Text = "",
        IsUser = false,
        Timestamp = DateTime.Now
    };
    Messages.Add(botMessage);

    while (!generator.IsDone())
    {
        generator.GenerateNextToken();
        var output = tokenizerStream.Decode(generator.GetSequence(0)[^1]);
        Console.Write(output);
        var message = Messages.FirstOrDefault(m => m.Id == responseGuid);
        if (message != null)
        {
            message.Text += output;
        }
    }

    await Task.CompletedTask;
}

I'm pretty sure in Messages.Add(botMessage); the UI didn't update, it updated after the while loop.

I found a blog post from syncfusion team: https://www.syncfusion.com/blogs/post/dotnet-maui-chatgpt-like-app-using-openai

But according to the screenshot at the end of the article, I think it's not stream response, because the AI response appeared in one time.

Below are my debug screenshots:

According to the above debug screenshots, even though Messages updated (Count is 2), but the UI didn't update, only after OnSendMessage returned, the UI get updated.

The xaml part is simple

<Button Grid.Column="1"
    Text="Send"
    Margin="5"
    CornerRadius="5"
    Clicked="OnSendMessage"
    BackgroundColor="{DynamicResource PrimaryColor}"
    TextColor="White" />

why the UI has to be updated after OnSendMessage?

---

update: Changing to await Task.Yield()

Sadly Task.Yield() not working, according the debug screenshot, the output is Phi which means the AI already generated some output, now Message count is 2, the UI didn't update, my input text introduce yourself is not cleared, even though I have MessageEntry.Text = string.Empty; before the AI operations.

You need to let the GUI framework actually work, you can't just hog the main thread until it's done generating the response. Luckily that's as simple as adding a yield operation:

    while (!generator.IsDone())
    {
        generator.GenerateNextToken();
        var output = tokenizerStream.Decode(generator.GetSequence(0)[^1]);
        Console.Write(output);
        var message = Messages.FirstOrDefault(m => m.Id == responseGuid);
        if (message != null)
        {
            message.Text += output;
        }

        await Task.Yield(); // <---
    }

Also get rid of that silly await Task.CompletedTask, it's doing nothing but hide the compiler's message that was trying to help you figure it out. Plus it's embarrassing.