Loading a large amount of images to be displayed in a WrapPanel

Question

I am using Entity Framework Code First

I have a Movie like so:

public class Movie
{
        public byte[] Thumbnail { get; set; }
        public int MovieId { get; set; }
}

And a Collection of Movies like so:

public class NorthwindContext : DbContext
{
    public DbSet<Movie> Movies { get; set; }
}

I have a MovieViewModel like so:

public class MovieViewModel
{
        private readonly Movie _movie;

        public MovieViewModel(Movie movie)
        {
            _movieModel = movieModel;
        }

        public byte[] Thumbnail { get { return _movie.Thumbnail; } }
}

When my App starts:

public ObservableCollection<MovieViewModel> MovieVms = 
                      new ObservableCollection<MovieViewModel>();

foreach (var movie in MyDbContext.Movies)
     MovieVms.Add(new MovieViewModel(movie));

I have 4000 movies. This process takes 25 seconds. Is there a better/faster way to do this?

My main page uses the thumbnails like so, but to be clear this loading time happens before anything UI related:

MyView = new ListCollectionView(MovieVms);

<ListBox ItemsSource="{Binding MyView}" />

Also my memory usage goes through the roof. How should I be loading these images? I need a full collection of view models off the bat to enable sorting, filtering, searching, but I only need the thumbnails of the items visible in my wrap panel.

EDIT---

Thanks Dave for a great answer. Can you elaborate on "make it an association (aka navigation property)"

var thumbnail = new Thumbnail();
thumbnail.Data = movie.GetThumbnail();
Globals.DbContext.Thumbnails.Add(thumbnail);
Globals.DbContext.SaveChanges();
movie.ThumbnailId = thumbnail.ThumbnailId;
Globals.DbContext.SaveChanges();

I can run that code with no errors, but my property in my MovieViewModel

public new byte[] Thumbnail { get { return _movie.Thumbnail.Data; } }

always has a null Thumbnail and errors as soon as my UI accesses it. A breakpoint on movie.ThumbnailId is never hit. Do I have to load the association manually?

Answer 1

I think you are essentially asking how to do several different things:

• Load the entire list of movies quickly, to allow for sorting and filtering in the UI
• Display the movie thumbnails in the UI but only when they are scrolled into view
• Keep memory usage to a minimum
• Display the UI as quickly as possible after the application starts

Load the movies quickly

First off, as @Dave M's answer states, you need to split the thumbnail into a separate entity so that you can ask Entity Framework to load the list of movies without also loading the thumbnails.

public class Movie
{
    public int Id { get; set; }
    public int ThumbnailId { get; set; }
    public virtual Thumbnail Thumbnail { get; set; }  // This property must be declared virtual
    public string Name { get; set; }

    // other properties
}

public class Thumbnail
{
    public int Id { get; set; }
    public byte[] Image { get; set; }
}

public class MoviesContext : DbContext
{
    public MoviesContext(string connectionString)
        : base(connectionString)
    {
    }

    public DbSet<Movie> Movies { get; set; }
    public DbSet<Thumbnail> Thumbnails { get; set; }
}

So, to load all of the movies:

public List<Movie> LoadMovies()
{
    // Need to get '_connectionString' from somewhere: probably best to pass it into the class constructor and store in a field member
    using (var db = new MoviesContext(_connectionString))
    {
        return db.Movies.AsNoTracking().ToList();
    }
}

At this point you will have a list of Movie entities where the ThumbnailId property is populated but the Thumbnail property will be null as you have not asked EF to load the related Thumbnail entities. Also, should you try to access the Thumbnail property later you will get an exception as the MoviesContext is no longer in scope.

Once you have a list of Movie entities, you need to convert them into ViewModels. I'm assuming here that your ViewModels are effectively read-only.

public sealed class MovieViewModel
{
    public MovieViewModel(Movie movie)
    {
        _thumbnailId = movie.ThumbnailId;
        Id = movie.Id;
        Name = movie.Name;
        // copy other property values across
    }

    readonly int _thumbnailId;

    public int Id { get; private set; }
    public string Name { get; private set; }
    // other movie properties, all with public getters and private setters

    public byte[] Thumbnail { get; private set; }  // Will flesh this out later!
}

Note that we're just storing the thumbnail ID here, and not populating the Thumbnail yet. I'll come to that in a bit.

Load thumbnails separately, and cache them

So, you've loaded the movies, but at the moment you haven't loaded any thumbnails. What you need is a method that will load a single Thumbnail entity from the database given its ID. I would suggest combining this with a cache of some sort, so that once you've loaded a thumbnail image you keep it in memory for a while.

public sealed class ThumbnailCache
{
    public ThumbnailCache(string connectionString)
    {
        _connectionString = connectionString;
    }

    readonly string _connectionString;
    readonly Dictionary<int, Thumbnail> _cache = new Dictionary<int, Thumbnail>();

    public Thumbnail GetThumbnail(int id)
    {
        Thumbnail thumbnail;

        if (!_cache.TryGetValue(id, out thumbnail))
        {
            // Not in the cache, so load entity from database
            using (var db = new MoviesContext(_connectionString))
            {
                thumbnail = db.Thumbnails.AsNoTracking().Find(id);
            }

            _cache.Add(id, thumbnail);
        }

        return thumbnail;
    }
}

This is obviously a very basic cache: the retrieval is blocking, there is no error handling, and the thumbnails should really be removed from the cache if they haven't been retrieved for a while in order to keep memory usage down.

Going back to the ViewModel, you need to modify the constructor to take a reference to a cache instance, and also modify the Thumbnail property getter to retrieve the thumbnail from the cache:

public sealed class MovieViewModel
{
    public MovieViewModel(Movie movie, ThumbnailCache thumbnailCache)
    {
        _thumbnailId = movie.ThumbnailId;
        _thumbnailCache = thumbnailCache;
        Id = movie.Id;
        Name = movie.Name;
        // copy other property values across
    }

    readonly int _thumbnailId;
    readonly ThumbnailCache _thumbnailCache;

    public int Id { get; private set; }
    public string Name { get; private set; }
    // other movie properties, all with public getters and private setters

    public BitmapSource Thumbnail
    {
        get
        {
            if (_thumbnail == null)
            {
                byte[] image = _thumbnailCache.GetThumbnail(_thumbnailId).Image;

                // Convert to BitmapImage for binding purposes
                var bitmapImage = new BitmapImage();
                bitmapImage.BeginInit();
                bitmapImage.StreamSource = new MemoryStream(image);
                bitmapImage.CreateOptions = BitmapCreateOptions.None;
                bitmapImage.CacheOption = BitmapCacheOption.Default;
                bitmapImage.EndInit();

                _thumbnail = bitmapImage;
            }

            return _thumbnail;
        }
    }
    BitmapSource _thumbnail;
}

Now the thumnail images will only be loaded when the Thumbnail property is accessed: if the image was already in the cache, it will be returned immediately, otherwise it will be loaded from the database first and then stored in the cache for future use.

Binding performance

The way that you bind your collection of MovieViewModels to the control in your view will have an impact on perceived loading time as well. What you want to do whenever possible is to delay the binding until your collection has been populated. This will be quicker than binding to an empty collection and then adding items to the collection one at a time. You may already know this but I thought I'd mention it just in case.

This MSDN page (Optimizing Performance: Data Binding) has some useful tips.

This awesome series of blog posts by Ian Griffiths (Too Much, Too Fast with WPF and Async) shows how various binding strategies can affect the load times of a bound list.

Only loading thumbnails when in view

Now for the most difficult bit! We've stopped the thumbnails from loading when the application starts, but we do need to load them at some point. The best time to load them is when they are visible in the UI. So the question becomes: how do I detect when the thumbnail is visible in the UI? This largely depends on the controls you are using in your view (the UI).

I'll assume that you are binding your collection of MovieViewModels to an ItemsControl of some type, such as a ListBox or ListView. Furthermore, I'll assume that you have some kind of DataTemplate configured (either as part of the ListBox/ListView markup, or in a ResourceDictionary somewhere) that is mapped to the MovieViewModel type. A very simple version of that DataTemplate might look like this:

<DataTemplate DataType="{x:Type ...}">
    <StackPanel>
        <Image Source="{Binding Thumbnail}" Stretch="Fill" Width="100" Height="100" />
        <TextBlock Text="{Binding Name}" />
    </StackPanel>
</DataTemplate>

If you are using a ListBox, even if you change the panel it uses to something like a WrapPanel, the ListBox's ControlTemplate contains a ScrollViewer, which provides the scroll bars and handles any scrolling. In this case then, we can say that a thumbnail is visible when it appears within the ScrollViewer's viewport. Therefore, we need a custom ScrollViewer element that, when scrolled, determines which of its "children" are visible in the viewport, and flags them accordingly. The best way of flagging them is to use an attached Boolean property: in this way, we can modify the DataTemplate to trigger on the attached property value changing and load the thumbnail at that point.

The following ScrollViewer descendant (sorry for the terrible name!) will do just that (note that this could probably be done with an attached behaviour instead of having to subclass, but this answer is long enough as it is).

public sealed class MyScrollViewer : ScrollViewer
{
    public static readonly DependencyProperty IsInViewportProperty =
        DependencyProperty.RegisterAttached("IsInViewport", typeof(bool), typeof(MyScrollViewer));

    public static bool GetIsInViewport(UIElement element)
    {
        return (bool) element.GetValue(IsInViewportProperty);
    }

    public static void SetIsInViewport(UIElement element, bool value)
    {
        element.SetValue(IsInViewportProperty, value);
    }

    protected override void OnScrollChanged(ScrollChangedEventArgs e)
    {
        base.OnScrollChanged(e);

        var panel = Content as Panel;
        if (panel == null)
        {
            return;
        }

        Rect viewport = new Rect(new Point(0, 0), RenderSize);

        foreach (UIElement child in panel.Children)
        {
            if (!child.IsVisible)
            {
                SetIsInViewport(child, false);
                continue;
            }

            GeneralTransform transform = child.TransformToAncestor(this);
            Rect childBounds = transform.TransformBounds(new Rect(new Point(0, 0), child.RenderSize));
            SetIsInViewport(child, viewport.IntersectsWith(childBounds));
        }
    }
}

Basically this ScrollViewer assumes that it's Content is a panel, and sets the attached IsInViewport property to true for those children of the panel that lie within the viewport, ie. are visible to the user. All that remains now is to modify the XAML for the view to include this custom ScrollViewer as part of the ListBox's template:

<Window x:Class="..."
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        xmlns:my="clr-namespace:...">

    <Window.Resources>
        <DataTemplate DataType="{x:Type my:MovieViewModel}">
            <StackPanel>
                <Image x:Name="Thumbnail" Stretch="Fill" Width="100" Height="100" />
                <TextBlock Text="{Binding Name}" />
            </StackPanel>
            <DataTemplate.Triggers>
                <DataTrigger Binding="{Binding Path=(my:MyScrollViewer.IsInViewport), RelativeSource={RelativeSource AncestorType={x:Type ListBoxItem}}}"
                             Value="True">
                    <Setter TargetName="Thumbnail" Property="Source" Value="{Binding Thumbnail}" />
                </DataTrigger>
            </DataTemplate.Triggers>
        </DataTemplate>
    </Window.Resources>

    <ListBox ItemsSource="{Binding Movies}">
        <ListBox.Template>
            <ControlTemplate>
                <my:MyScrollViewer HorizontalScrollBarVisibility="Disabled" VerticalScrollBarVisibility="Auto">
                    <WrapPanel IsItemsHost="True" />
                </my:MyScrollViewer>
            </ControlTemplate>
        </ListBox.Template>
    </ListBox>

</Window>

Here we have a Window containing a single ListBox. We've changed the ControlTemplate of the ListBox to include the custom ScrollViewer, and inside that is the WrapPanel that will layout the items. In the window's resources we have the DataTemplate that will be used to display each MovieViewModel. This is similar to the DataTemplate introduced earlier, but note that we are no longer binding the Image's Source property in the body of the template: instead, we use a trigger based on the IsInViewport property, and set the binding when the item becomes 'visible'. This binding will cause the MovieViewModel class's Thumbnail property getter to be called, which will load the thumbnail image either from the cache or the database. Note that the binding is to the property on the parent ListBoxItem, into which the markup for the DataTemplate is injected.

The only problem with this approach is that, as the thumbnail loading is done on the UI thread, scrolling will be affected. The easiest way to fix this would be to modify the MovieViewModel Thumbnail property getter to return a "dummy" thumbnail, schedule the call to the cache on a separate thread, then get that thread to set the Thumbnail property accordingly and raise a PropertyChanged event, thus ensuring the binding mechanism picks-up the change. There are other solutions but they would raise the complexity significantly: consider what is presented here as just a possible starting point.

Answer 2

Whenever you request an Entity from EF, it automatically loads all scalar properties (only associations are lazy loaded). Move the Thumbnail data to it's own Entity, make it an association (aka navigation property) and take advantage of the Lazy loading.

public class Movie
{
    public int Id { get; set; }
    public int ThumbnailId { get; set; }
    public virtual Thumbnail Thumbnail { get; set; }

    public string Name { get; set; }
    public double Length { get; set; }
    public DateTime ReleaseDate { get; set; }
    //etc...
}

public class Thumbnail
{
    public int Id { get; set; }
    public byte[] Data { get; set; }
}

public class MovieViewModel
{
    private readonly Movie _movie;

    public MovieViewModel(Movie movie)
    {
        _movieModel = movieModel;
    }

    public byte[] Thumbnail { get { return _movie.Thumbnail.Data; } }
}

Now the thumbnail data will only be loaded from the database when Thumbnail property of the ViewModel is accessed by the UI.