Enabling http compression with express

Question

I'm messing around with the Darksky API and under one of the query parameters it states:

extend=hourly optional

When present, return hour-by-hour data for the next 168 hours, instead of the next 48. When using this option, we strongly recommend enabling HTTP compression.

I'm using Express as a node proxy which hits the Darksky api (i.e. localhost:3000/api/forecast/LATITUDE, LONGITUDE).

What does "HTTP compression" mean and how would I go about enabling it?

Answer 1

Here compression means the gzip compression on the express server. You can use the compression middleware to add easy gzip compression to your server.

Read more about how you can install that middleware on here. https://github.com/expressjs/compression

An example implementation should be look like this.

var compression = require('compression')
var express = require('express')

var app = express()

// compress all responses
app.use(compression())

// add all routes

Answer 2

To quote from https://darksky.net/dev/docs

The Forecast Data API supports HTTP compression. We heartily recommend using it, as it will make responses much smaller over the wire. To enable it, simply add an Accept-Encoding: gzip header to your request. (Most HTTP client libraries wrap this functionality for you, please consult your library’s documentation for details.)

I'm not familiar with the Dark Sky API but I would imagine it returns a large amount of highly redundant data, which is ideal for compression. HTTP requests have a compression mechanism built in via Accept-Encoding, as mentioned above.

In your case that data will be travelling across the wire twice, once from Dark Sky to your server and then again from your server to your end user. You could compress just one of these two transmissions or both, it's up to you but it's likely you'd want both unless the end user is on the same local network as your server.

There are various SO questions about making compressed requests, such as:

node.js - easy http requests with gzip/deflate compression

The key decision for you is whether you want to decompress and recompress the data in your proxy or just stream it through. If you don't need a decompressed copy of the data in the server then it would be more efficient to skip the extra steps. You'd need to be careful to ensure all the headers are set correctly but if you just pass on the relevant headers that you receive (in both directions) it should be relatively simple to pipe through the response from Dark Sky.