mvvmvideo-streamingreactiveuiavaloniawriteablebitmap

How to modify an Avalonia WritableBitmap and update the changes as image control on UI?


I have a WritableBitmap object _myBitmap in my MainWindowViewModel class which I want to use to show the live footage from a webcam. But essentially, what I have to do is to write some new values into the WritableBitmap's frame buffer periodically, as shown in the method _changeMyBitmapColor.

Expected app behavior: the gray level of the bitmap changes from 0 to 50, 100, 150... every 500ms.

Actual app behavior: The change of the bitmap color is only reflected on the UI when the app window is resized or when the dummy combo box is pressed. The color changed from 0 to 50 and seems to stay at 50 forever. Also, the displayed bitmap only changed its color when the window is resized or the dummy ComboBox in MainWindow.axaml is pressed.

I have no idea why resizing the window or poking the dummy ComboBox on the UI updates the displayed bitmap. Nor do I understand why the displayed bitmap only seem to change its color once throughout the lifetime of the app. From the console logs, it is confirmed that the framebuffer of the WritableBitmap _myBitmap is successfully modified every 500ms.

Here is a reproducible example. I implemented the ReactiveUI approach as described in this tutorial. Since there are unsafe methods in the project, there is a <AllowUnsafeBlocks>true</AllowUnsafeBlocks> in the TryUpdateBitmap.csproj. The project is created using the default Avalonia MVVM project template by running:

dotnet new avalonia.mvvm -o TryUpdateBitmap

File: MainWindowViewModel

namespace TryUpdateBitmap.ViewModels;

using Avalonia.Media.Imaging;
using Avalonia;
using Avalonia.Platform;
using System.Threading;
using System;
using System.Threading.Tasks;
using ReactiveUI;

public class MainWindowViewModel : ReactiveObject
{
    public MainWindowViewModel()
    {
        // Assume the bitmap is of shape 640x480 with 4 channels
        _myBitmap = new WriteableBitmap(new PixelSize(640, 480), new Vector(96, 96), PixelFormat.Bgra8888, AlphaFormat.Opaque);
        Console.WriteLine("Start app");
        Task.Run(_updateMyBitmapColorPeriodicallyAsync);
    }
    public WriteableBitmap MyBitmap 
    { 
        get => _myBitmap; 
        set => this.RaiseAndSetIfChanged(ref _myBitmap, value); 
    }

    private WriteableBitmap _myBitmap;
    private  async Task _updateMyBitmapColorPeriodicallyAsync()
    {
        PeriodicTimer timer = new(TimeSpan.FromMilliseconds(500)); // Color of my bitmap pixels changes every 500ms
        int colorOption = 0;
        while (await timer.WaitForNextTickAsync())
        {
            _changeMyBitmapColor(colorOption);
            colorOption = (colorOption + 1) % 5; // there will be 3 different color options
            Console.WriteLine("Bitmap modified!");
        }
    }
    private unsafe void _changeMyBitmapColor(int colorOption)
    {
        byte[] pixelValueOptions = [50, 100, 150, 200, 225];
        using ILockedFramebuffer frameBuffer = MyBitmap.Lock();
        void* dataStart = (void*)frameBuffer.Address;
        Span<byte> buffer = new Span<byte>(dataStart, 640 * 480 * 4); // assume each pixel is 1 byte in size and my image has 4 channels
        buffer.Fill(pixelValueOptions[colorOption]); // fill the bitmap memory buffer with some values
    }
}

File: MainWindow.axaml

<Window xmlns="https://github.com/avaloniaui"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:vm="using:TryUpdateBitmap.ViewModels"
        xmlns:d="http://schemas.microsoft.com/expression/blend/2008"
        xmlns:mc="http://schemas.openxmlformats.org/markup-compatibility/2006"
        mc:Ignorable="d" d:DesignWidth="800" d:DesignHeight="450"
        x:Class="TryUpdateBitmap.Views.MainWindow"
        x:DataType="vm:MainWindowViewModel"
        Icon="/Assets/avalonia-logo.ico"
        Title="IHaveGivenUpIHaveLetMyselfDownIWillRunAroundAndDesertMyself">

    <Design.DataContext>
        <vm:MainWindowViewModel/>
    </Design.DataContext>
    <Grid ColumnDefinitions="640" RowDefinitions="500">
        <Image Source="{Binding MyBitmap, Mode=TwoWay}" Grid.Column="0" Grid.Row="0"/>
        <ComboBox Grid.Column="0" Grid.Row="1"/>
    </Grid>
</Window>

Any help is much appreciated! Happy Chinese New Year.


Solution

  • Calling InvalidateVisual() for the image allows you to use only one buffer. The call could not be made directly from the timer thread, so something like it is required:

    ViewModel:
        public event Action Refresh;
        ...
        while (await timer.WaitForNextTickAsync())
        {
            _changeMyBitmapColor(colorOption);
            ...
            await Dispatcher.UIThread.InvokeAsync(Refresh);
        }
    View:
        Loaded += (_, _) => {
            ((MainWindowViewModel)DataContext).Refresh += () => { 
                img.InvalidateVisual(); 
            };         
        };