Clip BitmapImage using Strokes from a InkCanvas

Question

I'm tasked to create a "Cinemagraph" feature, the user must select a desired area using an InkCanvas to draw the selected pixels that should remain untouched for the rest of the animation/video (or, to select the pixels that should be "alive").

Example:

I'm thinking about getting the Stroke collection from the InkCanvas and use that to clip the image and merge with the untouched one.

How can I do that? I can easily load the images from disk, but how can I clip an image based on a stroke?

More Details:

After drawing and selecting the pixels that should remain static, I have a Stroke collection. I can get the Geometry of each individual Stroke, but I probably need to merge all geometries.

Based on that merged Geometry, I need to invert (the Geometry) and use to clip my first frame, later with the clipped image ready, I need to merge with all other frames.

My code so far:

//Gets the BitmapSource from a String path:
var image = ListFrames[0].ImageLocation.SourceFrom();
var rectangle = new RectangleGeometry(new Rect(new System.Windows.Point(0, 0), new System.Windows.Size(image.Width, image.Height)));
Geometry geometry = Geometry.Empty;

foreach(Stroke stroke in CinemagraphInkCanvas.Strokes)
{
    geometry = Geometry.Combine(geometry, stroke.GetGeometry(), GeometryCombineMode.Union, null);
}

//Inverts the geometry, to clip the other unselect pixels of the BitmapImage.
geometry = Geometry.Combine(geometry, rectangle, GeometryCombineMode.Exclude, null);

//This here is UIElement, I can't use this control, I need a way to clip the image without using the UI.
var clippedImage = new System.Windows.Controls.Image();
clippedImage.Source = image;
clippedImage.Clip = geometry;

//I can't get the render of the clippedImage control because I'm not displaying that control.

Is there any way to clip a BitmapSource without using an UIElement?

Maybe, Maybe

I'm thinking about OpacityMask and a brush... but I can't use a UIElement, I need to apply the OpacityMask directly to the BitmapSource.

Answer 1

I made it! (You can see the result here, ScreenToGif > Editor > Image Tab > Cinemagraph)

---

Code

SourceFrom() and DpiOf() and ScaledSize():

/// <summary>
/// Gets the BitmapSource from the source and closes the file usage.
/// </summary>
/// <param name="fileSource">The file to open.</param>
/// <param name="size">The maximum height of the image.</param>
/// <returns>The open BitmapSource.</returns>
public static BitmapSource SourceFrom(this string fileSource, Int32? size = null)
{
    using (var stream = new FileStream(fileSource, FileMode.Open))
    {
        var bitmapImage = new BitmapImage();
        bitmapImage.BeginInit();
        bitmapImage.CacheOption = BitmapCacheOption.OnLoad;

        if (size.HasValue)
            bitmapImage.DecodePixelHeight = size.Value;

        //DpiOf() and ScaledSize() uses the same principles of this extension.

        bitmapImage.StreamSource = stream;
        bitmapImage.EndInit();

        //Just in case you want to load the image in another thread.
        bitmapImage.Freeze();             
        return bitmapImage;
    }
}

GetRender():

/// <summary>
/// Gets a render of the current UIElement
/// </summary>
/// <param name="source">UIElement to screenshot</param>
/// <param name="dpi">The DPI of the source.</param>
/// <returns>An ImageSource</returns>
public static RenderTargetBitmap GetRender(this UIElement source, double dpi)
{
    Rect bounds = VisualTreeHelper.GetDescendantBounds(source);

    var scale = dpi / 96.0;
    var width = (bounds.Width + bounds.X) * scale;
    var height = (bounds.Height + bounds.Y) * scale;

    #region If no bounds

    if (bounds.IsEmpty)
    {
        var control = source as Control;

        if (control != null)
        {
            width = control.ActualWidth * scale;
            height = control.ActualHeight * scale;
        }

        bounds = new Rect(new System.Windows.Point(0d, 0d), 
                          new System.Windows.Point(width, height));
    }

    #endregion

    var roundWidth = (int)Math.Round(width, MidpointRounding.AwayFromZero);
    var roundHeight = (int)Math.Round(height, MidpointRounding.AwayFromZero);

    var rtb = new RenderTargetBitmap(roundWidth, roundHeight, dpi, dpi, 
                                     PixelFormats.Pbgra32);

    DrawingVisual dv = new DrawingVisual();
    using (DrawingContext ctx = dv.RenderOpen())
    {
        VisualBrush vb = new VisualBrush(source);

        var locationRect = new System.Windows.Point(bounds.X, bounds.Y);
        var sizeRect = new System.Windows.Size(bounds.Width, bounds.Height);

        ctx.DrawRectangle(vb, null, new Rect(locationRect, sizeRect));
    }

    rtb.Render(dv);
    return (RenderTargetBitmap)rtb.GetAsFrozen();
}

---

Gets the ImageSource and Geometry:

//Custom extensions, that using the path of the image, will provide the
//DPI (of the image) and the scaled size (PixelWidth and PixelHeight).
var dpi = ListFrames[0].ImageLocation.DpiOf();
var scaledSize = ListFrames[0].ImageLocation.ScaledSize();

//Custom extension that loads the first frame.
var image = ListFrames[0].ImageLocation.SourceFrom();

//Rectangle with the same size of the image. Used within the Xor operation.
var rectangle = new RectangleGeometry(new Rect(
    new System.Windows.Point(0, 0), 
    new System.Windows.Size(image.PixelWidth, image.PixelHeight)));
Geometry geometry = Geometry.Empty;

//Each Stroke is transformed into a Geometry and combined with an Union operation.
foreach(Stroke stroke in CinemagraphInkCanvas.Strokes)
{
    geometry = Geometry.Combine(geometry, stroke.GetGeometry(), 
        GeometryCombineMode.Union, null);
}

//The rectangle with the same size of the image is combined with all of 
//the Strokes using the Xor operation, basically it inverts the Geometry.
geometry = Geometry.Combine(geometry, rectangle, GeometryCombineMode.Xor, null);

Applying the Geometry to the Image element:

//UIElement used to hold the BitmapSource to be clipped.
var clippedImage = new System.Windows.Controls.Image
{
    Height = image.PixelHeight,
    Width = image.PixelWidth,
    Source = image,
    Clip = geometry
};
clippedImage.Measure(scaledSize);
clippedImage.Arrange(new Rect(scaledSize));

//Gets the render of the Image element, already clipped.
var imageRender = clippedImage.GetRender(dpi, scaledSize);

//Merging with all frames:
Overlay(imageRender, dpi, true);   

---

Overlay(), Merges the frames:

private void Overlay(RenderTargetBitmap render, double dpi, bool forAll = false)
{
    //Gets the selected frames based on the selection of a ListView, 
    //In this case, every frame should be selected.
    var frameList = forAll ? ListFrames : SelectedFrames();

    int count = 0;
    foreach (FrameInfo frame in frameList)
    {
        var image = frame.ImageLocation.SourceFrom();

        var drawingVisual = new DrawingVisual();
        using (DrawingContext drawingContext = drawingVisual.RenderOpen())
        {
            drawingContext.DrawImage(image, new Rect(0, 0, image.Width, image.Height));
            drawingContext.DrawImage(render, new Rect(0, 0, render.Width, render.Height));
        }

        //Converts the Visual (DrawingVisual) into a BitmapSource
        var bmp = new RenderTargetBitmap(image.PixelWidth, image.PixelHeight, dpi, dpi, PixelFormats.Pbgra32);
        bmp.Render(drawingVisual);

        //Creates a BmpBitmapEncoder and adds the BitmapSource to the frames of the encoder
        var encoder = new BmpBitmapEncoder();
        encoder.Frames.Add(BitmapFrame.Create(bmp));

        //Saves the image into a file using the encoder
        using (Stream stream = File.Create(frame.ImageLocation))
            encoder.Save(stream);
    }
}

Example:

Clean, unedited animation.

Selected pixels that should be animated.

Image already clipped (black is transparent).

Cinemagraph done!

As you can see, only the selected pixels can change, the others remain static.