I'm working on a MITM proxy on Node using typescript. I'm trying to decode responses that are encoded using gzip. I don't want to just remove accept-encoding header as I saw in some answers here on SO. I want to use zlib to decode the response body, but for some reason when I try to load a page that uses gzip (like github.com for example) the page doesn't load (images, colors, texts, etc.). My decompress is not working and I don't know why. The code that I'm using to decompress the response body is the following:
NOTE: serverResponse is the response from the server that I (as the proxy) am connecting to (github.com for example) and proxyResponse is the response from me (the proxy) to the client that started the request
protected async receiveResponse(serverResponse: http.IncomingMessage, proxyResponse: http.ServerResponse) {
const contentEncoding = serverResponse.headers["content-encoding"]
let responseContent: http.IncomingMessage | zlib.Gunzip = serverResponse
if (contentEncoding && contentEncoding.toLowerCase().includes("gzip")) {
responseContent = zlib.createGunzip()
serverResponse.pipe(responseContent)
delete serverResponse.headers["content-encoding"]
}
let responseBody: Buffer
try {
responseBody = await this.collectMessageBody(responseContent)
} catch (error) {
console.log(error)
return
}
proxyResponse.writeHead(serverResponse.statusCode!, serverResponse.headers)
proxyResponse.write(responseBody)
proxyResponse.end()
}
private collectMessageBody(stream: http.IncomingMessage | zlib.Gunzip): Promise<Buffer> {
return new Promise<Buffer>((resolve, reject) => {
let bodyBuffers: Buffer[] = []
stream.on('data', chunk => bodyBuffers.push(chunk))
stream.on('end', () => resolve(Buffer.concat(bodyBuffers)))
stream.on('error', error => reject(error))
})
}
It follows the same strategy that most answers here on the SO, but I don't know why mine is not working.
OBS: This is an open-source project, so the entire file can be found here: https://github.com/olmps/web-sniffer/blob/master/src/server.ts
I've simplified the logic removing unnecessary stuff to post here.
As an example, when loading github.com that uses gzip to compress its content, I get the following result when the proxy is on:

You could try to handle the content-length header as well.
If the content-encoding is gzip you can try to change the value of content-length header with the length of uncompressed body, not only to uncompress the body.
Hope this helps.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With