WPF - Displaying Large Number of Log Messages Continously

I am developing an application using WPF. I used Serilog for logging and I have a log sink that displays the log events in a list box. Incoming log events are queued in the Emit method of the log sink and queued log events are processed in the CompositionTarget.Rendering event.

Queuing Log events:

public void EventsLogged(IEnumerable<Event> Events)
        {
            System.Windows.Application.Current?.Dispatcher.Invoke((Action)delegate
                {
                    foreach (var item in Events)
                    {
                        _logMessageQueue.Enqueue(new LogEvent(item));
                    }
                });
        }

Rendering event:

private void CompositionTargetOnRendering(object sender, EventArgs e)
        {
            while (_logMessageQueue.TryDequeue(out var logEvent))
                {
                    Messages.Add(logEvent);
                }          
        }

Messages is an ObservableCollection that is binded to the ItemSource of the list box.

Here is my problem, when there are huge amounts of enqueued log events in a short period of time, UI responsiveness becomes horrible. Is there any suggestions to improve my application? Or any learning material that I can follow would be really nice.

Thank you

Here are a couple of suggestions that should improve the performance:

• If you have to use the Dispatcher in order to collect the log events, you should use Dispatcher.InvokeAsync with an appropriate priority. Note that you should avoid flooding the Dispatcher with tasks. If you are expecting to invoke the Dispatcher in high frequencies consider to user a different approach (e.g. background thread):

public void EventsLogged(IEnumerable<Event> events)
{
  // Don't block
  Application.Current.Dispatcher.InvokeAsync(
    () =>
    {
      foreach (var item in events)
      {
        _logMessageQueue.Enqueue(new LogEvent(item));
      }
    }, DispatcherPriority.Background);
}

• Consider to avoid the Dispatcher at all. Unless LogEvent extends DispatcherObject you can perform the data structure conversion on a background thread or asynchronously (depending on your requirements):

// In case this member is an event handler, define the method of type 'void' (instead of 'Task')
public async Task EventsLoggedAsync(IEnumerable<Event> events)
{
  // Don't block
  await Task.Run(
    () =>
    {
      foreach (var item in events)
      {
        _logMessageQueue.Enqueue(new LogEvent(item));
      }
    });
}

• Improve your consumer part: the while loop is a blocking operation. You can prevent blocking by using the Dispatcher with an explicit priority to execute code when the Dispatcher is idle:

private void CompositionTargetOnRendering(object sender, EventArgs e)
{
  while (_logMessageQueue.TryDequeue(out var logEvent))
  {
    // Enqueue task to the Dispatcher queue 
    // and let the Dispatcher decide when to process the tasks by defining a priority
    Application.Current.Dispatcher.InvokeAsync(
      () =>
      {
        Messages.Add(logEvent);
      }, DispatcherPriority.Background);
  }
}

To further improve the solution I suggest to implement the Producer/Consumer pattern. If you use Channel you can make the complete implementation asynchronous (see Microsoft Docs: System.Threading.Channels library).

For efficiency, this example also avoids reading from the queue by handling the CompositionTarget.Rendering event. Instead the example will start a loop on a background thread to consume the queue.
Dispatcher.InvokeAsync is used to control the pressure on the Dispatcher: using DispatcherPriority.Background or DispatcherPriority.ContextIdle should relief the pressure so that the the main thread can continue to render the surface.

The final solution could look as follows:

class ChannelExample
{
  private Channel<LogEvent> LogEventChannel { get; }

  public ChannelExample()
  {
    var channelOptions = new UnboundedChannelOptions
    {
      SingleReader = true,
      SingleWriter = false,
    };

    this.LogEventChannel = Channel.CreateUnbounded<LogEvent>(channelOptions);

    // Start the consumer loop in the background
    Task.Run(DisplayQueuedLogEventsAsync);
  }

  // Optional: allows to stop the producer/consumer process 
  // and closes the queue for additional writes.
  // Because this is an application logger, this method is very likely redundant for your scenario
  private void CloseLogEventQueueForWrites()
    => this.LogEventChannel.Writer.Complete();

  // Thread-safe implementation allows for concurrency
  public async Task EventsLoggedAsync(IEnumerable<Event> events)
  {
    await Task.Run(
      async () =>
      {
        ChannelWriter<int> logEventChannelWriter = this.LogEventChannel.Writer;

        // Consider to use Parallel.ForEach (must be tested because it is not guaranteed that it will improve the performance)
        foreach (Event event in events)
        {
          var logMessage = new LogEvent(event);

          while (await logEventChannelWriter.WaitToWriteAsync())
          {
            if (valueChannelWriter.TryWrite(logMessage))
            {
              // TODO:: Optionally do something after an item 
              // was successfully written to the channel (e.g. increment counter)
            }
          }
        }
      });
  }

  private async Task DisplayQueuedLogEventsAsync()
  {
    ChannelReader<LogEvent> logEventChannelReader = this.LogEventChannel.Reader;

    // Asynchronous iteration will continue until the Channel is closed (by calling ChannelWriter.Complete()).
    await foreach (LogEvent logEvent in logEventChannelReader.ReadAllAsync())
    {
      // Use priority Background or ContextIdle 
      // to control and relief the pressure on the Dispatcher
      Application.Current.Dispatcher.InvokeAsync(
        () =>
        {
          Messages.Add(logEvent);
        }, DispatcherPriority.Background);
    }
  }
}