VirtualizingStackPanel is fast only when fresh. How can I fix it?

When I create ListBox with virtualization enabled and then update all its items appearance it works very fast. But when i slowly scroll down all items in ListBox and then update all items appearance it takes a lot of time. I think it because VirtualizingStackPanel does not destroy items when they are runs out of viewport. I wrote simple app to reproduce this behavior.

Code:

public partial class MainWindow : Window
{
    public MainWindow()
    {
        InitializeComponent();
        for(int i = 0; i < 5000; ++i) // creating 5k text boxes
            MyList.Items.Add(new TextBox() { Text = CurrText });
    }

    private void Button_Click(object sender, RoutedEventArgs e)
    {
        GC.Collect();
        n = (n + 1) % 2; // switch 0 to 1 or 1 to 0
        foreach (var item in MyList.Items)
            ((TextBox)item).Text = CurrText; // set new text
    }

    static int n = 0;
    string CurrText { get { return new string(n.ToString()[0], 50); } }
}

XAML:

<Window x:Class="VPanel.MainWindow"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="700" Width="525">
    <Grid>
        <Grid.RowDefinitions>
            <RowDefinition Height="5*"/>
            <RowDefinition Height="*"/>
        </Grid.RowDefinitions>
        <ListBox Name="MyList" VirtualizingStackPanel.IsVirtualizing="True"/>
        <Button Grid.Row="1" Content="UpdateText" Click="Button_Click"/>
    </Grid>
</Window>

Clicking a button "UpdateText" updates all textboxes text. If slowly scroll to end by dragging scroller, "UpdateText" button clicks with a huge lag.

Don't create TextBoxes manually.

The word "virtualize" refers to a technique by which a subset of user interface (UI) elements are generated from a larger number of data items based on which items are visible on-screen. Generating many UI elements when only a few elements might be on the screen can adversely affect the performance of your application.

You create your UI items manually so it's already too late for virtualization. Use bindings and it will create TextBox from ItemTemplate whenever it is required. It will also not refresh TextBox.Text value if it's not currently in the view. To do that change your MainWindow to create ObservableCollection instead of TextBoxes and operate on that:

public partial class MainWindow : Window
{
    private readonly ObservableCollection<string> _textBoxes = new ObservableCollection<string>();

    public ICollection<string> TextBoxes { get { return _textBoxes; } }

    private int n = 0;
    private string CurrText { get { return new string(n.ToString()[0], 50); } }

    public MainWindow()
    {
        for (int i = 0; i < 5000; ++i) _textBoxes.Add(CurrText);
        InitializeComponent();
        DataContext = this;
    }

    private void Button_Click(object sender, RoutedEventArgs e)
    {
        n = (n + 1) % 2; // switch 0 to 1 or 1 to 0
        for (int i = 0; i < _textBoxes.Count; i++) _textBoxes[i] = CurrText;
    }
}

and then change XAML to bind to TextBoxes list property

<Grid>
    <Grid.RowDefinitions>
        <RowDefinition Height="5*"/>
        <RowDefinition Height="*"/>
    </Grid.RowDefinitions>
    <ListBox ItemsSource="{Binding TextBoxes}">
        <ListBox.ItemTemplate>
            <DataTemplate>
                <TextBox Text="{Binding Path=., Mode=TwoWay}"/>
            </DataTemplate>
        </ListBox.ItemTemplate>
    </ListBox>
    <Button Grid.Row="1" Content="UpdateText" Click="Button_Click"/>
</Grid>