How do I create a leaflet map with thousands of marks that doesn't crash my browser?

Question

I'm using the leaflet package in R to generate a map with a large number of circles on it. The goal is a map I can publish to my website. The problem I'm having is that as I increase the number of circles, the resulting map loads very slowly, I get "unresponsive script" warnings and ultimately it completely freezes up my browser.

I know this sort of thing is possible, because I've found a leaflet map that works the way I want mine to work:

http://cartologic.com/geoapps/map_viewer/5/ny-crimes-2014-dot-density-map

I notice on the above map that the circles don't appear "clickable" like the circles on my map, and that they seem to load in square chunks. I have a hunch that these things are related to my problem. Unfortunately, I'm too much of a novice at leaflet/javascript stuff to figure this out on my own.

Here is a toy example illustrating my problem:

library("leaflet")
library("htmlwidgets")

dots <- data.frame(x=c(runif(10000, -93.701281, -93.533053)),
                   y=c(runif(10000,  41.515962,  41.644369)))

m <- leaflet(dots) %>%
  addTiles('http://{s}.basemaps.cartocdn.com/dark_all/{z}/{x}/{y}.png') %>% 
  setView(-93.617167, 41.580166, zoom = 12) %>% 
  addCircles(~x, ~y, weight = 1, radius = 5, 
             color = "#FFA500", stroke = TRUE, fillOpacity = 0.1) 

m

saveWidget(widget = m, file="example.html", selfcontained = TRUE)

Answer 1

mapview can help you here. It builds upon the leaflet library for smaller data sets, but uses special javascript functionality for larger data.

your example with 1 Mio. points:

library(mapview)
library(sp)

dots <- data.frame(x=c(runif(1000000, -93.701281, -93.533053)),
                   y=c(runif(1000000,  41.515962,  41.644369)))

coordinates(dots) <- ~ x + y
proj4string(dots) <- "+init=epsg:4326"

mapview(dots)

It may still take a while to render, but once rendered it should be quite responsive. Note that mapview is designed to work with spatial* objects, that is why we need the calls to set the coordinate slot and the projection.

For more information have a look here:

http://environmentalinformatics-marburg.github.io/web-presentations/20150723_mapView.html

Hope that helps.

Answer 2

If you want to add a large number of vector objects to a map, it is rare that it can be done easily.

Notice that the raster data is broken into tiles so that all the information does not have to be shown at one time. For your vector data (in this case, circles) you have to do the same thing.

Basically what I like to do is to break the large data set into smaller (vector) tiles, with the same boundaries as the raster tiles you are showing. Duplicate the data if you want it to appear at several zoom level. As you are showing circle, imagine that you partition the circles' center points on the tile boundary.

I have an application similar to this where I basically partition my vector data on tile boundaries and store the information in geojson files. When I get an event that the raster tile has been loaded I can then load the equivalent vector file as a geojson layer (same thing when the raster tile is unloaded). In this way, you can limit the amount of vector data that has to be displayed at any one time.

If you have a lot of points, they are not really going to be visible at low zoom levels anyway, so it might be better just to show them at an appropriate zoom level (perhaps with a different representation at low zooms - like a heat map). This will keep the amount of data being shown at any one time lower.

Answer 3

Since this question has a few upvotes, I'll generally describe both of the solutions I found. Maybe if I have time later I'll get all the files together on GitHub.

First, I found TileMill. Simply load a data file of coordinates into TileMill, style the way you want them to appear, and output tiles (png). Host those tiles on the web somewhere and load them with leaflet. This process was a bit too manual for my liking because TileMill kept crashing when I loaded in csv files that were too large for it to render on my machine.

I found the best solution was use Processing, adapting Robert Manduca's code here: https://github.com/rmanduca/jobmaps. I don't use Python so I rewrote those parts in R and modified the Processing code according to my specifications.