Heavy rendering task (in canvas) in JavaFX blocks GUI

Question

I want to create an application that performs many renderings in a canvas. The normal JavaFX way blocks the GUI: It is realy hard to press the button in the application code below (run with Java 8).

I searched the web, but JavaFX does not support background rendering: All rendering operation (like strokeLine) are stored in a buffer and are executed in the JavaFX application thread later. So I cannot even use two canvases and exchange then after rendering.

Also the javafx.scene.Node.snapshot(SnapshotParameters, WritableImage) cannot be used to create an image in a background thread, as it needs to run inside the JavaFX application thread and so it will block the GUI also.

Any ideas to have a non blocking GUI with many rendering operations? (I just want to press buttons etc. while the rendering is performed somehow in background or paused regularly)

package canvastest;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.ExecutorService;
import java.util.concurrent.Executors;
import java.util.concurrent.Future;

import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.application.Platform;
import javafx.event.ActionEvent;
import javafx.scene.Scene;
import javafx.scene.canvas.Canvas;
import javafx.scene.canvas.GraphicsContext;
import javafx.scene.control.Button;
import javafx.scene.layout.VBox;
import javafx.scene.paint.Color;
import javafx.scene.shape.StrokeLineCap;
import javafx.stage.Stage;

public class DrawLinieTest extends Application
{
    int             interations     = 2;

    double          lineSpacing     = 1;

    Random          rand            = new Random(666);

    List<Color>     colorList;

    final VBox      root            = new VBox();

    Canvas          canvas          = new Canvas(1200, 800);

    Canvas          canvas2         = new Canvas(1200, 800);

    ExecutorService executorService = Executors.newSingleThreadExecutor();

    Future<?>       drawShapesFuture;

    {
        colorList = new ArrayList<>(256);
        colorList.add(Color.ALICEBLUE);
        colorList.add(Color.ANTIQUEWHITE);
        colorList.add(Color.AQUA);
        colorList.add(Color.AQUAMARINE);
        colorList.add(Color.AZURE);
        colorList.add(Color.BEIGE);
        colorList.add(Color.BISQUE);
        colorList.add(Color.BLACK);
        colorList.add(Color.BLANCHEDALMOND);
        colorList.add(Color.BLUE);
        colorList.add(Color.BLUEVIOLET);
        colorList.add(Color.BROWN);
        colorList.add(Color.BURLYWOOD);

    }

    public static void main(String[] args)
    {
        launch(args);
    }

    @Override
    public void start(Stage primaryStage)
    {
        primaryStage.setTitle("Drawing Operations Test");

        System.out.println("Init...");

        // inital draw that creates a big internal operation buffer (GrowableDataBuffer)
        drawShapes(canvas.getGraphicsContext2D(), lineSpacing);
        drawShapes(canvas2.getGraphicsContext2D(), lineSpacing);

        System.out.println("Start testing...");
        new CanvasRedrawTask().start();

        Button btn = new Button("test " + System.nanoTime());
        btn.setOnAction((ActionEvent e) ->
        {
            btn.setText("test " + System.nanoTime());
        });

        root.getChildren().add(btn);
        root.getChildren().add(canvas);

        Scene scene = new Scene(root);

        primaryStage.setScene(scene);
        primaryStage.show();
    }

    private void drawShapes(GraphicsContext gc, double f)
    {
        System.out.println(">>> BEGIN: drawShapes ");

        gc.clearRect(0, 0, gc.getCanvas().getWidth(), gc.getCanvas().getHeight());

        gc.setLineWidth(10);

        gc.setLineCap(StrokeLineCap.ROUND);

        long time = System.nanoTime();

        double w = gc.getCanvas().getWidth() - 80;
        double h = gc.getCanvas().getHeight() - 80;
        int c = 0;

        for (int i = 0; i < interations; i++)
        {
            for (double x = 0; x < w; x += f)
            {
                for (double y = 0; y < h; y += f)
                {
                    gc.setStroke(colorList.get(rand.nextInt(colorList.size())));
                    gc.strokeLine(40 + x, 10 + y, 10 + x, 40 + y);
                    c++;
                }
            }
        }

        System.out.println("<<< END: drawShapes: " + ((System.nanoTime() - time) / 1000 / 1000) + "ms");
    }

    public synchronized void drawShapesAsyc(final double f)
    {
        if (drawShapesFuture != null && !drawShapesFuture.isDone())
            return;
        drawShapesFuture = executorService.submit(() ->
        {
            drawShapes(canvas2.getGraphicsContext2D(), lineSpacing);

            Platform.runLater(() ->
            {
                root.getChildren().remove(canvas);

                Canvas t = canvas;
                canvas = canvas2;
                canvas2 = t;

                root.getChildren().add(canvas);
            });

        });
    }

    class CanvasRedrawTask extends AnimationTimer
    {
        long time = System.nanoTime();

        @Override
        public void handle(long now)
        {
            drawShapesAsyc(lineSpacing);
            long f = (System.nanoTime() - time) / 1000 / 1000;
            System.out.println("Time since last redraw " + f + " ms");
            time = System.nanoTime();
        }
    }
}

EDIT Edited the code to show that a background thread that sends the draw operations and than exchange the canvas does not resolve the problem! Because All rendering operation (like strokeLine) are stored in a buffer and are executed in the JavaFX application thread later.

Answer 1

You are drawing 1.6 million lines per frame. It is simply a lot of lines and takes time to render using the JavaFX rendering pipeline. One possible workaround is not to issue all drawing commands in a single frame, but instead render incrementally, spacing out drawing commands, so that the application remains relatively responsive (e.g. you can close it down or interact with buttons and controls on the app while it is rendering). Obviously, there are some tradeoffs in extra complexity with this approach and the result is not as desirable as simply being able to render extremely large amounts of draw commands within the context of single 60fps frame. So the presented approach is only acceptable for some kinds of applications.

Some ways to perform an incremental render are:

1. Only issue a max number of calls each frame.
2. Place the rendering calls into a buffer such as a blocking queue and just drain a max number of calls each frame from the queue.

Here is a sample of the first option.

import javafx.animation.AnimationTimer;
import javafx.application.Application;
import javafx.concurrent.*;
import javafx.scene.Scene;
import javafx.scene.canvas.*;
import javafx.scene.control.Button;
import javafx.scene.image.*;
import javafx.scene.layout.VBox;
import javafx.scene.paint.Color;
import javafx.scene.shape.StrokeLineCap;
import javafx.stage.Stage;

import java.util.ArrayList;
import java.util.List;
import java.util.Random;
import java.util.concurrent.locks.*;

public class DrawLineIncrementalTest extends Application {
    private static final int FRAME_CALL_THRESHOLD = 25_000;

    private static final int ITERATIONS = 2;
    private static final double LINE_SPACING = 1;
    private final Random rand = new Random(666);
    private List<Color> colorList;
    private final WritableImage image = new WritableImage(ShapeService.W, ShapeService.H);

    private final Lock lock = new ReentrantLock();
    private final Condition rendered = lock.newCondition();
    private final ShapeService shapeService = new ShapeService();

    public DrawLineIncrementalTest() {
        colorList = new ArrayList<>(256);
        colorList.add(Color.ALICEBLUE);
        colorList.add(Color.ANTIQUEWHITE);
        colorList.add(Color.AQUA);
        colorList.add(Color.AQUAMARINE);
        colorList.add(Color.AZURE);
        colorList.add(Color.BEIGE);
        colorList.add(Color.BISQUE);
        colorList.add(Color.BLACK);
        colorList.add(Color.BLANCHEDALMOND);
        colorList.add(Color.BLUE);
        colorList.add(Color.BLUEVIOLET);
        colorList.add(Color.BROWN);
        colorList.add(Color.BURLYWOOD);
    }

    public static void main(String[] args) {
        launch(args);
    }

    @Override
    public void start(Stage primaryStage) {
        primaryStage.setTitle("Drawing Operations Test");

        System.out.println("Start testing...");
        new CanvasRedrawHandler().start();

        Button btn = new Button("test " + System.nanoTime());
        btn.setOnAction(e -> btn.setText("test " + System.nanoTime()));

        Scene scene = new Scene(new VBox(btn, new ImageView(image)));
        primaryStage.setScene(scene);
        primaryStage.show();
    }

    private class CanvasRedrawHandler extends AnimationTimer {
        long time = System.nanoTime();

        @Override
        public void handle(long now) {
            if (!shapeService.isRunning()) {
                shapeService.reset();
                shapeService.start();
            }

            if (lock.tryLock()) {
                try {
                    System.out.println("Rendering canvas");
                    shapeService.canvas.snapshot(null, image);
                    rendered.signal();
                } finally {
                    lock.unlock();
                }
            }

            long f = (System.nanoTime() - time) / 1000 / 1000;
            System.out.println("Time since last redraw " + f + " ms");
            time = System.nanoTime();
        }
    }

    private class ShapeService extends Service<Void> {
        private Canvas canvas;

        private static final int W = 1200, H = 800;

        public ShapeService() {
            canvas = new Canvas(W, H);
        }

        @Override
        protected Task<Void> createTask() {
            return new Task<Void>() {
                @Override
                protected Void call() throws Exception {
                    drawShapes(canvas.getGraphicsContext2D(), LINE_SPACING);

                    return null;
                }
            };
        }

        private void drawShapes(GraphicsContext gc, double f) throws InterruptedException {
            lock.lock();
            try {
                System.out.println(">>> BEGIN: drawShapes ");

                gc.clearRect(0, 0, gc.getCanvas().getWidth(), gc.getCanvas().getHeight());
                gc.setLineWidth(10);
                gc.setLineCap(StrokeLineCap.ROUND);

                long time = System.nanoTime();

                double w = gc.getCanvas().getWidth() - 80;
                double h = gc.getCanvas().getHeight() - 80;

                int nCalls = 0, nCallsPerFrame = 0;

                for (int i = 0; i < ITERATIONS; i++) {
                    for (double x = 0; x < w; x += f) {
                        for (double y = 0; y < h; y += f) {
                            gc.setStroke(colorList.get(rand.nextInt(colorList.size())));
                            gc.strokeLine(40 + x, 10 + y, 10 + x, 40 + y);
                            nCalls++;
                            nCallsPerFrame++;
                            if (nCallsPerFrame >= FRAME_CALL_THRESHOLD) {
                                System.out.println(">>> Pausing: drawShapes ");
                                rendered.await();
                                nCallsPerFrame = 0;
                                System.out.println(">>> Continuing: drawShapes ");
                            }
                        }
                    }
                }

                System.out.println("<<< END: drawShapes: " + ((System.nanoTime() - time) / 1000 / 1000) + "ms for " + nCalls + " ops");
            } finally {
                lock.unlock();
            }
        }
    }
}

Note that for the sample, it is possible to interact with the scene by clicking the test button while the incremental rendering is in progress. If desired, you could further enhance this to double buffer the snapshot images for the canvas so that the user doesn't see the incremental rendering. Also because the incremental rendering is in a Service, you can use the service facilities to track rendering progress and relay that to the UI via a progress bar or whatever mechanisms you wish.

For the above sample you can play around with the FRAME_CALL_THRESHOLD setting to vary the maximum number of calls which are issued each frame. The current setting of 25,000 calls per frame keeps the UI very responsive. A setting of 2,000,000 would be the same as fully rendering the canvas in a single frame (because you are issuing 1,600,000 calls in the frame) and no incremental rendering will be performed, however the UI will not be responsive while the rendering operations are being completed for that frame.

Side Note

There is something weird here. If you remove all of the concurrency stuff and the double canvases in the code in the original question and just use a single canvas with all logic on the JavaFX application thread, the initial invocation of drawShapes takes 27 seconds, and subsequent invocations take less that a second, but in all cases the application logic is asking the system to perform the same task. I don't know why the initial call is so slow, it seems like a performance issue in the JavaFX canvas implementation to me, perhaps related to inefficient buffer allocation. If that is the case, then perhaps the JavaFX canvas implementation could be tweaked so that a hint for a suggested initial buffer size could be provided, so that it more efficiently allocates space for its internal growable buffer implementation. It might be something worth filing a bug or discussing it on the JavaFX developer mailing list. Also note that the issue of a very slow initial rendering of the canvas is only visible when you issue a very large number (e.g. > 500,000) of rendering calls, so it won't effect all applications.