R microbenchmark warning - Could not measure a positive execution time for x evaluations

Question

I'm developing a function to plot a zoo object so I'm benchmarking the execution speeds of various options using microbenchmark.

However, each time I run the microbenchmark I get the following error:

Warning message:

In microbenchmark::microbenchmark(overlapping_base, overlapping_lattice, : Could not measure a positive execution time for 2524 evaluations.

The value of evaluations is changing each time I run the code.

My guess is that the execution of the benchmark is unable to finish. This means that each time I run the microbenchmark I get a very different response (which is drastically changing the execution speed). Because of that, I'm unable to ascertain which plotting method is the quickest.

The code I'm using:

library(lattice)
library(ggplot2)
library(zoo)

z <- structure(c(-0.02838132, -0.02642141, -0.02511601, -0.02443906, 
-0.02414024, -0.02417388, -0.02451562, -0.02393946, -0.02242496, 
-0.02104852, 0.01739089, 0.01981665, 0.02210309, 0.02376982, 
0.02496479, 0.02538433, 0.02558674, 0.02642987, 0.02848319, 0.0305888, 
-0.02174604, -0.01987625, -0.01885374, -0.01837707, -0.01820114, 
-0.01826306, -0.01862707, -0.01826593, -0.01710169, -0.01580846, 
-0.00849795, -0.00590564, -0.00353068, -0.00165086, -0.00038422, 
0.00032302, 0.00035668, 0.00161519, 0.00415983, 0.00657286, 0.00965842, 
0.01212343, 0.01442211, 0.01579477, 0.01667243, 0.01666898, 0.01611297, 
0.01692785, 0.01928281, 0.02143322, -0.0194464, -0.01757103, 
-0.0158406, -0.01457289, -0.01380771, -0.0135324, -0.0140401, 
-0.01340732, -0.01169943, -0.00988943, -0.02916019, -0.02673978, 
-0.02466732, -0.0237339, -0.02308799, -0.02317864, -0.02376145, 
-0.0227629, -0.02004088, -0.01753194, -0.05127982, -0.04929435, 
-0.04880177, -0.04996632, -0.05004232, -0.0493009, -0.05101994, 
-0.05113115, -0.04892411, -0.0475149, -0.07164312, -0.0686314, 
-0.0660294, -0.06348602, -0.06356825, -0.06302109, -0.06276327, 
-0.06209531, -0.05954085, -0.05830172, 0.01311737, 0.0148998, 
0.01550045, 0.01542315, 0.01480778, 0.01336875, 0.01106039, 0.01161289, 
0.01314323, 0.0142203, -0.07079136, -0.06946734, -0.06746012, 
-0.06623026, -0.06813571, -0.06920641, -0.07168502, -0.0710586, 
-0.06919208, -0.06819284, -0.04583223, -0.04687478, -0.04720935, 
-0.04925846, -0.04846798, -0.05010275, -0.04891061, -0.04988978, 
-0.04763314, -0.04486417, 0.01508285, 0.01771387, 0.01775548, 
0.0186012, 0.01964036, 0.0206486, 0.02302578, 0.02525439, 0.02973214, 
0.0317942, 0.00488448, 0.00831908, 0.01058987, 0.01139279, 0.01237221, 
0.01294144, 0.01263533, 0.01327397, 0.01563062, 0.01794102, -0.00091997, 
-0.00010884, 0.00179444, 0.00415676, 0.00597374, 0.00498312, 
0.00294308, 0.00433847, 0.00660325, 0.00808107, 0.01277616, 0.0147175, 
0.01651502, 0.01767393, 0.0183272, 0.01811757, 0.01746553, 0.01833143, 
0.02001129, 0.02195872, 0.03273455, 0.03516022, 0.03719699, 0.03895484, 
0.04029506, 0.04113609, 0.04134692, 0.04248674, 0.0445687, 0.04700408, 
-0.03783289, -0.03577733, -0.03476676, -0.03349859, -0.03263628, 
-0.03324435, -0.0336928, -0.03278793, -0.03165878, -0.03065102, 
0.02888535, 0.03179749, 0.03449393, 0.03605427, 0.037507, 0.03839276, 
0.03837656, 0.03956103, 0.04147407, 0.0435806, -0.0104104, -0.00884235, 
-0.00788977, -0.007072, -0.00677401, -0.00643356, -0.0062355, 
-0.00593316, -0.00524193, -0.0045362, -0.07363815, -0.0713515, 
-0.06969982, -0.06867965, -0.0681607, -0.06815293, -0.06900493, 
-0.068134, -0.06649919, -0.06501484, 0.02291233, 0.02638201, 
0.02907269, 0.03113551, 0.03215457, 0.03223916, 0.03193051, 0.03326608, 
0.03605787, 0.03868543, -0.14471737, -0.14382875, -0.1425845, 
-0.14208004, -0.14131485, -0.14088634, -0.14169012, -0.1417378, 
-0.14083509, -0.13993506, -0.04463136, -0.04119767, -0.03801833, 
-0.03594387, -0.03533695, -0.03557779, -0.03625033, -0.03449043, 
-0.03091173, -0.0278551), .Dim = c(10L, 24L), .Dimnames = list(
    NULL, c("CH1", "CH2", "CH3", "CH4", "CH5", "CH6", "CH7", 
    "CH8", "CH9", "CH10", "CH11", "CH12", "CH13", "CH14", "CH15", 
    "CH16", "CH17", "CH18", "CH19", "CH20", "CH21", "CH22", "CH23", 
    "CH24")), index = 1:10, class = "zoo")

# benchmark plotting operations
overlapping_base <- plot(z, screens = 1, col = 1:ncol(z))
overlapping_lattice <- xyplot(z, screens = 1, col = 1:ncol(z))
overlapping_ggplot2 <- autoplot(z, facets = NULL) + aes(linetype = NULL)

microbenchmark::microbenchmark(overlapping_base, overlapping_lattice, overlapping_ggplot2, times = 1000)

Any ideas why I get that warning or how to fix it?

Answer 1

This warning is typical if you benchmark something with extremely fast execution. If you look at the benchmarks you'll see timings of 0 ns. Note that no actual plotting occurs during the benchmark runs.

Execute overlapping_base to see what you are actually benchmarking. Note that grid plots are not plotted in non-interactive use if they are not wrapped in print.

Fixing your code:

# benchmark plotting operations
overlapping_base <- function() plot(z, screens = 1, col = 1:ncol(z))
overlapping_lattice <- function() print(xyplot(z, screens = 1, col = 1:ncol(z)))
overlapping_ggplot2 <- function() print(autoplot(z, facets = NULL) + aes(linetype = NULL))

microbenchmark::microbenchmark(overlapping_base(),
                               overlapping_lattice(), 
                               overlapping_ggplot2(), times = 10)

#Unit: milliseconds
#                  expr       min        lq     mean   median       uq      max neval cld
#    overlapping_base()  13.63389  19.36380 100.2684 152.1086 154.7808 160.4726    10  a 
# overlapping_lattice()  49.70754  51.79813 123.7162 125.9431 192.6951 204.0190    10  a 
# overlapping_ggplot2() 238.17771 249.62242 342.9059 388.5780 411.9674 420.9687    10   b