WPF "Lazy" VisualBrush

Question

I'm trying to implement somesting like an "Lazy" VisualBrush right now. Does anybody have an idea how to to that? Meaning: Something that behaves like an VisualBrush but does not update on every change in the Visual but at max once a second (or whatever).

I better should give some background why I'm doing this and what I alreay tried I guess :)

Problem: My job right now is to improve the performance of an rather big WPF application. I tracked down the main performance issue (at the UI level anyway) to some visual brushes used in the application. The application consists of an "Desktop" area with some rather complex UserControls and an Navigation area containing a scaled down version of the Desktop. The navigation area is using visual brushes to get the job done. Everything is fine as long as the Desktop items are more or less static. But if the elements are changing frequently (because they contain an animation for example) the VisualBrushes go wild. They will update along with the framerate of the animations. Lowering the framerate helps of course, but I'm looking for an more general solution to this problem. While the "source" control only renders the small area affected by the animation the visual brush container is rendered completly causing the application performance to go to hell. I already tried to use BitmapCacheBrush instead. Doesn't help unfortunately. The animation is inside the control. So the brush have to be refreshed anyway.

Possible solution: I created a Control behaving more or less like an VisualBrush. It takes some visual (as the VisualBrush) but is using a DiapatcherTimer and RenderTargetBitmap to do the job. Right now I'm subscribing to the LayoutUpdated event of the control and whenever it changes it will be scheduled for "rendering" (using RenderTargetBitmap). The actual rendering then is triggered by the DispatcherTimer. This way the control will repaint itself at maximum in the frequency of the DispatcherTimer.

Here is the code:

public sealed class VisualCopy : Border
{
    #region private fields

    private const int mc_mMaxRenderRate = 500;
    private static DispatcherTimer ms_mTimer;
    private static readonly Queue<VisualCopy> ms_renderingQueue = new Queue<VisualCopy>();
    private static readonly object ms_mQueueLock = new object();

    private VisualBrush m_brush;
    private DrawingVisual m_visual;
    private Rect m_rect;
    private bool m_isDirty;
    private readonly Image m_content = new Image();
    #endregion

    #region constructor
    public VisualCopy()
    {
        m_content.Stretch = Stretch.Fill;
        Child = m_content;
    }
    #endregion

    #region dependency properties

    public FrameworkElement Visual
    {
        get { return (FrameworkElement)GetValue(VisualProperty); }
        set { SetValue(VisualProperty, value); }
    }

    // Using a DependencyProperty as the backing store for Visual.  This enables animation, styling, binding, etc...
    public static readonly DependencyProperty VisualProperty =
        DependencyProperty.Register("Visual", typeof(FrameworkElement), typeof(VisualCopy), new UIPropertyMetadata(null, OnVisualChanged));

    #endregion

    #region callbacks

    private static void OnVisualChanged(DependencyObject obj, DependencyPropertyChangedEventArgs args)
    {
        var copy = obj as VisualCopy;
        if (copy != null)
        {
            var oldElement = args.OldValue as FrameworkElement;
            var newelement = args.NewValue as FrameworkElement;
            if (oldElement != null)
            {
                copy.UnhookVisual(oldElement);
            }
            if (newelement != null)
            {
                copy.HookupVisual(newelement);
            }
        }
    }

    private void OnVisualLayoutUpdated(object sender, EventArgs e)
    {
        if (!m_isDirty)
        {
            m_isDirty = true;
            EnqueuInPipeline(this);
        }
    }

    private void OnVisualSizeChanged(object sender, SizeChangedEventArgs e)
    {
        DeleteBuffer();
        PrepareBuffer();
    }

    private static void OnTimer(object sender, EventArgs e)
    {
        lock (ms_mQueueLock)
        {
            try
            {
                if (ms_renderingQueue.Count > 0)
                {
                    var toRender = ms_renderingQueue.Dequeue();
                    toRender.UpdateBuffer();
                    toRender.m_isDirty = false;
                }
                else
                {
                    DestroyTimer();
                }
            }
            catch (Exception ex)
            {
            }
        }
    }
    #endregion

    #region private methods
    private void HookupVisual(FrameworkElement visual)
    {
        visual.LayoutUpdated += OnVisualLayoutUpdated;
        visual.SizeChanged += OnVisualSizeChanged;
        PrepareBuffer();
    }

    private void UnhookVisual(FrameworkElement visual)
    {
        visual.LayoutUpdated -= OnVisualLayoutUpdated;
        visual.SizeChanged -= OnVisualSizeChanged;
        DeleteBuffer();
    }


    private static void EnqueuInPipeline(VisualCopy toRender)
    {
        lock (ms_mQueueLock)
        {
            ms_renderingQueue.Enqueue(toRender);
            if (ms_mTimer == null)
            {
                CreateTimer();
            }
        }
    }

    private static void CreateTimer()
    {
        if (ms_mTimer != null)
        {
            DestroyTimer();
        }
        ms_mTimer = new DispatcherTimer { Interval = TimeSpan.FromMilliseconds(mc_mMaxRenderRate) };
        ms_mTimer.Tick += OnTimer;
        ms_mTimer.Start();
    }

    private static void DestroyTimer()
    {
        if (ms_mTimer != null)
        {
            ms_mTimer.Tick -= OnTimer;
            ms_mTimer.Stop();
            ms_mTimer = null;
        }
    }

    private RenderTargetBitmap m_targetBitmap;
    private void PrepareBuffer()
    {
        if (Visual.ActualWidth > 0 && Visual.ActualHeight > 0)
        {
            const double topLeft = 0;
            const double topRight = 0;
            var width = (int)Visual.ActualWidth;
            var height = (int)Visual.ActualHeight;
            m_brush = new VisualBrush(Visual);
            m_visual = new DrawingVisual();
            m_rect = new Rect(topLeft, topRight, width, height);
            m_targetBitmap = new RenderTargetBitmap((int)m_rect.Width, (int)m_rect.Height, 96, 96, PixelFormats.Pbgra32);
            m_content.Source = m_targetBitmap;
        }
    }

    private void DeleteBuffer()
    {
        if (m_brush != null)
        {
            m_brush.Visual = null;
        }
        m_brush = null;
        m_visual = null;
        m_targetBitmap = null;
    }

    private void UpdateBuffer()
    {
        if (m_brush != null)
        {
            var dc = m_visual.RenderOpen();
            dc.DrawRectangle(m_brush, null, m_rect);
            dc.Close();
            m_targetBitmap.Render(m_visual);
        }
    }

    #endregion
}

This works pretty good so far. Only problem is the trigger. When I use LayoutUpdated then the Rendering is triggered constantly even if the Visual itself is not changed at all (propably because of animations in other parts of the application or whatever). LayoutUpdated is just fired way to often. As a matter of fact I could just skip the trigger and just update the control using the timer without any trigger. It doesn't matter. I also tried to override OnRender in the Visual and raise an custom event to trigger the update. Doesn't work either because OnRender is not called when something deep inside the VisualTree changes. This is my best shot right now. It's working much better then the original VisualBrush solution already (from the performance point of view at least). But I'm, still looking for an even better solution.

Does anyone have an idea how to a) trigger the update only when nessasarry or b) get the job done with an altogether differen approach?

Thanks!!!

Answer 1

I've monitored the visual status of controls using the internals of WPF via reflection. So the code I've written hooks into the CompositionTarget.Rendering event, walks the tree, and looks for any changes in the subtree. I was writing it to intercept data being pushed to MilCore and then use it for my own purposes, so take this code as a hack and nothing more. If it helps you, great. I was using this on .NET 4.

First, the code to walk the tree read the status flags:

using System;
using System.Collections.Generic;
using System.Linq;
using System.Text;
using System.Windows;
using System.Windows.Media;
using System.Reflection;

namespace MilSnatch.Utils
{
    public static class VisualTreeHelperPlus
    {
        public static IEnumerable<DependencyObject> WalkTree(DependencyObject root)
        {
            yield return root;
            int count = VisualTreeHelper.GetChildrenCount(root);
            for (int i = 0; i < count; i++)
            {
                foreach (var descendant in WalkTree(VisualTreeHelper.GetChild(root, i)))
                    yield return descendant;
            }
        }

        public static CoreFlags ReadFlags(UIElement element)
        {
            var fieldInfo = typeof(UIElement).GetField("_flags", BindingFlags.Instance | BindingFlags.NonPublic);
            return (CoreFlags)fieldInfo.GetValue(element);
        }

        public static bool FlagsIndicateUpdate(UIElement element)
        {
            return (ReadFlags(element) &
                (
                    CoreFlags.ArrangeDirty |
                    CoreFlags.MeasureDirty |
                    CoreFlags.RenderingInvalidated
                )) != CoreFlags.None;
        }
    }

    [Flags]
    public enum CoreFlags : uint
    {
        AreTransformsClean = 0x800000,
        ArrangeDirty = 8,
        ArrangeInProgress = 0x20,
        ClipToBoundsCache = 2,
        ExistsEventHandlersStore = 0x2000000,
        HasAutomationPeer = 0x100000,
        IsCollapsed = 0x200,
        IsKeyboardFocusWithinCache = 0x400,
        IsKeyboardFocusWithinChanged = 0x800,
        IsMouseCaptureWithinCache = 0x4000,
        IsMouseCaptureWithinChanged = 0x8000,
        IsMouseOverCache = 0x1000,
        IsMouseOverChanged = 0x2000,
        IsOpacitySuppressed = 0x1000000,
        IsStylusCaptureWithinCache = 0x40000,
        IsStylusCaptureWithinChanged = 0x80000,
        IsStylusOverCache = 0x10000,
        IsStylusOverChanged = 0x20000,
        IsVisibleCache = 0x400000,
        MeasureDirty = 4,
        MeasureDuringArrange = 0x100,
        MeasureInProgress = 0x10,
        NeverArranged = 0x80,
        NeverMeasured = 0x40,
        None = 0,
        RenderingInvalidated = 0x200000,
        SnapsToDevicePixelsCache = 1,
        TouchEnterCache = 0x80000000,
        TouchesCapturedWithinCache = 0x10000000,
        TouchesCapturedWithinChanged = 0x20000000,
        TouchesOverCache = 0x4000000,
        TouchesOverChanged = 0x8000000,
        TouchLeaveCache = 0x40000000
    }

}

Next, supporting code for Rendering event:

//don't worry about RenderDataWrapper. Just use some sort of WeakReference wrapper for each UIElement
    void CompositionTarget_Rendering(object sender, EventArgs e)
{
    //Thread.Sleep(250);
    Dictionary<int, RenderDataWrapper> newCache = new Dictionary<int, RenderDataWrapper>();
    foreach (var rawItem in VisualTreeHelperPlus.WalkTree(m_Root))
    {
        var item = rawItem as FrameworkElement;
        if (item == null)
        {
            Console.WriteLine("Encountered non-FrameworkElement: " + rawItem.GetType());
            continue;
        }
        int hash = item.GetHashCode();
        RenderDataWrapper cacheEntry;
        if (!m_Cache.TryGetValue(hash, out cacheEntry))
        {
            cacheEntry = new RenderDataWrapper();
            cacheEntry.SetControl(item);
            newCache.Add(hash, cacheEntry);
        }
        else
        {
            m_Cache.Remove(hash);
            newCache.Add(hash, cacheEntry);
        }
            //check the visual for updates - something like the following...
            if(VisualTreeHelperPlus.FlagsIndicateUpdate(item as UIElement))
            {
                //flag for new snapshot.
            }
        }
    m_Cache = newCache;
}

Anyways, in this way I monitored the visual tree for updates, and I think you can monitor them using something similar if you'd like. This is far from best practices, but sometimes pragmatic code has to be. Beware.

Answer 2

I think your solution is pretty good already. Instead of a timer you could try to do it with a Dispatcher callback with a ApplicationIdle priority, this would effectively make the updates lazy since it will only occur when the application isn't busy. Also, as you have already stated you might try to use the BitmapCacheBrush instead of the VisualBrush to draw your overview image and see if this makes any difference.

Regarding your question on WHEN to redraw the brush:

Basically you want to know when things changed in a way that would mark your existing thumbnail image as dirty.

I think you could either attack this problem in the backend/model and have a dirty flag there or try to get it from the front end.

Backend obviously depends on your application so I can't comment.

In the front end the LayoutUpdated event seems the right thing to do but as you say it could fire more often than necessary.

Here is a shot in the dark - I don't know how LayoutUpdated works internally so it might have the same problem as LayoutUpdated: You could override ArrangeOverride in the control you want to observe. Whenever ArrangeOverride is called you fire your own layout updated event using a dispatcher so that it is fired after the layout pass finishes. (maybe even wait for a couple of milliseconds longer and don't queue more events if a new ArrangeOverride should be called in the meanwhile). Since a layout pass will always call Measure and then Arrange and travel up the tree this should cover any changes anywhere inside the control.