Listbox scrollbar thumb changes size when content is variable height

Question

I have a ListBox with many objects displayed, each of which can be a variable height, based on the number of values each object has. See my previous question that was answered here.

Many objects are 5-lines high, while others are 1. The scroll bar in the ListBox does not appear to like this, probably due to the virtualization. While you scroll through, the thumb on the scroll bar will change its size, based on how many items are actually fitting into the box at that current moment. This makes the thumb very big at times, and very small at other times.

Since this ListBox is also contained within a TabControl, when you switch from one tab to another, the ListBox will often scroll to a different section when you return to it.

Any ideas how to resolve an issue like this?

Additional Info: Disabling virtualization does fix the scrolling problem, although at the cost of a slower initial display. However, resizing the ListBox with the content inside causes some heavy lag when resizing horizontally (vertical is fine), which I'm assuming is due to my template's width changing and requiring a redraw on every element:

<DataTemplate DataType="{x:Type xmlset:Variable}">
    <Grid>
        <Grid.ColumnDefinitions>
            <ColumnDefinition Width="170"/>
            <ColumnDefinition Width="*"/>
        </Grid.ColumnDefinitions>
        <Border BorderThickness="1,0,0,1" BorderBrush="Black">
            <TextBlock Margin="2,2,0,2"  Text="{Binding Path=Identifier.Name, Mode=OneWay}"/>
        </Border>
        <ItemsControl IsTabStop="False" Grid.Column="1" ItemsSource="{Binding Path=Values, Mode=OneWay}">
            <ItemsControl.ItemTemplate>
                <DataTemplate>
                    <Grid>
                        <Grid.ColumnDefinitions>
                            <ColumnDefinition Width="120"/>
                            <ColumnDefinition Width="*"/>
                        </Grid.ColumnDefinitions>

                        <Border Grid.Column="0" BorderThickness="1,0,0,1" BorderBrush="Black">
                            <TextBlock Margin="2,2,0,2" Text="{Binding Path=Optimization, Mode=OneWay}"/>
                        </Border>
                        <Border Grid.Column="1" Width="Auto" BorderThickness="1,0,1,1" BorderBrush="Black">
                            <TextBox Margin="0,2,0,2" BorderThickness="0" Text="{Binding Path=Value}" TextChanged="TextBox_TextChanged"/>
                        </Border>
                    </Grid>
                </DataTemplate>
            </ItemsControl.ItemTemplate>
        </ItemsControl>
    </Grid>
</DataTemplate>

This is drawing borders around the edges of the fields to make a visual grouping, where val will stretch to the content size. The listbox also has HorizontalContentAlignmment = Stretch to ensure this looks correct.

-------------------
- var - opt - val -
-     -------------
-     - opt - val -
-     -------------
-     - opt - val -
-------------------

note: if this needs to be asked in a different question, tell me and i'll seperate the questions

Answer 1

Why not switch off any size restrictions on the ListBox itself, let it size to contents and wrap it into a ScrollViewer, setting a proper size for the latter?

The markup should look like the following:

    <ScrollViewer Width="640px" Height="480px">
        <ListBox>
            <ListBox.ItemTemplate>
                <DataTemplate>
                    <!--Visualization of a list item-->
                </DataTemplate>
            </ListBox.ItemTemplate>
        </ListBox>
    </ScrollViewer>

I saw no thumb size changings during scrolling if it was implemented this way.