Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Decode GZIP response on Node.JS Man In the Middle Proxy

Tags:

node.js

zlib

I'm working on a MITM proxy on Node using typescript. I'm trying to decode responses that are encoded using gzip. I don't want to just remove accept-encoding header as I saw in some answers here on SO. I want to use zlib to decode the response body, but for some reason when I try to load a page that uses gzip (like github.com for example) the page doesn't load (images, colors, texts, etc.). My decompress is not working and I don't know why. The code that I'm using to decompress the response body is the following:

NOTE: serverResponse is the response from the server that I (as the proxy) am connecting to (github.com for example) and proxyResponse is the response from me (the proxy) to the client that started the request

  protected async receiveResponse(serverResponse: http.IncomingMessage, proxyResponse: http.ServerResponse) {

    const contentEncoding = serverResponse.headers["content-encoding"]
    let responseContent: http.IncomingMessage | zlib.Gunzip = serverResponse

    if (contentEncoding && contentEncoding.toLowerCase().includes("gzip")) {
      responseContent = zlib.createGunzip()
      serverResponse.pipe(responseContent)
      delete serverResponse.headers["content-encoding"]
    }

    let responseBody: Buffer

    try {
      responseBody = await this.collectMessageBody(responseContent)
    } catch (error) {
      console.log(error)
      return
    }
    
    proxyResponse.writeHead(serverResponse.statusCode!, serverResponse.headers)
    proxyResponse.write(responseBody)
    proxyResponse.end()
  }

  private collectMessageBody(stream: http.IncomingMessage | zlib.Gunzip): Promise<Buffer> {
    return new Promise<Buffer>((resolve, reject) => {
      let bodyBuffers: Buffer[] = []

      stream.on('data', chunk => bodyBuffers.push(chunk))
      stream.on('end', () => resolve(Buffer.concat(bodyBuffers)))
      stream.on('error', error => reject(error))
    })
  }

It follows the same strategy that most answers here on the SO, but I don't know why mine is not working.

OBS: This is an open-source project, so the entire file can be found here: https://github.com/olmps/web-sniffer/blob/master/src/server.ts

I've simplified the logic removing unnecessary stuff to post here.

As an example, when loading github.com that uses gzip to compress its content, I get the following result when the proxy is on:

enter image description here

like image 464
GGirotto Avatar asked May 19 '26 08:05

GGirotto


1 Answers

You could try to handle the content-length header as well.

If the content-encoding is gzip you can try to change the value of content-length header with the length of uncompressed body, not only to uncompress the body.

Hope this helps.

like image 116
Daniele Ricci Avatar answered May 20 '26 22:05

Daniele Ricci



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!