Stream lots of screenshots from server

Question

Let's say I have this tool that takes a screenshot of the user's desktop every 10 seconds and uploads the images to a server. Images are deleted from the server after 24 hours. I want to allow the user to view these screenshots in their browser. So let's say each image is 300K and there are 5000 of them uploaded in a 24 hour period -- that's 1.5 GB of data. These are png images, and often one screenshot isn't that different from the previous one, so I imagine I could compress them pretty well -- but I'm not sure how that would work. I want to allow the user to view these images in their browser. I want a slider under the images so the user can jump to any point in the 24 hour period. I guess something like the YouTube player would be ideal. Where the user could jump around and data is streamed, etc. Image quality is important because the user needs to be able to read text in the screenshots. I'd prefer a Python solution.

I've never done anything like this before, and am not sure how to approach the problem. What would you do?

Answer 1

The quick & easy solution is to

1. tile up the screen along a fixed grid
2. store the tiles separately, indexed by a strong hash (to eliminate duplicates, even between users)
3. store the entire screen as a 2-d array of tile hashes
4. reconstruct the original screen in the browser from the hashed tiles.

Going from one screen to another, very similar one, would be a matter of loading just a few tiles, so there's your compression.

Implement a simple reference counting scheme to remove the tiles again. You might even want to share screen parts between users to further reduce storage.

Experiment with various tile sizes to find out which works best; this may depend on screen resolution, user activity and the graphics format used to store the elements.

Image processing in Python can be done with PIL.

Answer 2

If the user is currently at image K and want to load image H, assuming the screenshots will be mostly the same, you can just send the delta diff between H and K.

You can calculate the diffs ahead of time and store them into a cache or database for faster access (but there will be a lot of data) or you can calculate it on the fly.