HTTP/2 requests and headers starting with colon

Question

Hello beloved SO community.

I have a problem that tortures me for months with no solution.

I am trying to make a request at an HTTP/2 endpoint that uses some headers that start with a colon. Example:

:method: "POST"

I have tried with python(hyper, requests), php(guzzle) and js(fetch). I have managed, presumably, with js to achieve the required result but the CORS policy returns me an "opaque" result.

Any help will be great!

JS result presumbaly correct "opaque"

fetch("https://www.example.com/users/sign_in", 
        {
          "credentials":"include",
          "headers":{
            "accept":"text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
            "accept-language":"en-US,en;q=0.9,el;q=0.8",
            "cache-control":"max-age=0",
            "content-type":"application/x-www-form-urlencoded",
            "accept-encoding": "gzip, deflate, br",
            "content-length": 614,
            "origin": "https://www.example.com",
            "sec-fetch-mode": "navigate",
            "sec-fetch-site": "same-origin",
            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.87 Safari/537.36",
          },
          "referrer":"https://www.example.com/users/sign_in",
          "referrerPolicy":"no-referrer-when-downgrade",
          "redirect": "follow",
          "body":"..."
          "method":"POST",
          "mode":"no-cors",
          ":authority": "www.example.com",
          ":method": "POST",
          ":path": "/users/sign_in",
          ":scheme": "https"
        })

Python result 500 or 404

 context = tls.init_context()
    context.check_hostname = False
    context.verify_mode = ssl.CERT_NONE
    with HTTP20Connection('www.example.com', port=443, ssl_context=context) as c:
        headers = {
            ":authority": "www.example.com",
            ":method": "GET",
            ":path": "/users/sign_in",
            ":scheme": "https",
            "accept": "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
            "accept-encoding": "gzip, deflate, br",
            "accept-language": "en-US,en;q=0.9,el;q=0.8",
            "sec-fetch-mode": "navigate",
            "sec-fetch-site": "none",
            "sec-fetch-user": "?1",
            "upgrade-insecure-requests": "1",
            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
        }

        c.request('GET', 'www.example.com/users/sign_in', headers=headers)

PHP result 500 or curl error 55

$jar = new CookieJar;
    $client = new Client([
        // Base URI is used with relative requests
        'cookies' => $jar,
        'version' => 2.0,
        'debug' => fopen('php://stderr', 'w'),
    ]);

    $client->request('GET', 'https://www.example.com/users/sign_in');

    $response = $client->request('POST', 'https://www.example.com/users/sign_in', [
        'headers' => [
            ":authority"=> "www.example.com",
            ":method"=> "POST",
            ":path"=> "/users/sign_in",
            ":scheme"=> "https",
            "accept"=> "text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3",
            "accept-encoding"=> "gzip, deflate, br",
            "accept-language"=> "en-US,en;q=0.9,el;q=0.8",
            "cache-control"=> "max-age=0",
            "content-length"=> "616",
            "content-type"=> "application/x-www-form-urlencoded",
            "origin"=> "https://www.example.com",
            "referer"=> "https://www.example.com/users/sign_in",
            "sec-fetch-mode"=> "navigate",
            "sec-fetch-site"=> "same-origin",
            "sec-fetch-user"=> "?1",
            "upgrade-insecure-requests"=> "1",
            "user-agent"=> "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.100 Safari/537.36"
        ],
        'form_params' => [ ... ],
        'version' => 2.0,
        'allow_redirects' => true
    ]);

Answer 1

These are HTTP/2 pseudo-headers that apply to requests and responses within an HTTP/2 stream. HTTP/2 creates a single persistent connection from each distinct origin end-point to a server. That connection transmits multiple requests and responses to and from the end-point; these are parsed into "frames" and transmitted as "streams". HTTP/2 can interleave frames from multiple request and response streams simultaneously to get huge performance benefits. intro to http/2

Pseudo-headers apply to streams; a different set of headers apply to the connection itself. Four pseudo-headers are defined for requests: :method, :scheme, :authority, and :path. No others are allowed. These 4 must be included in every request header block and they must precede any other headers:

"All pseudo-header fields MUST appear in the header block before regular header fields. Any request or response that contains a pseudo-header field that appears in a header block after a regular header field MUST be treated as malformed (Section 8.1.2.6)." http2 spec

I'm not familiar with how 'fetch' implements the headers, but in your code above you seem to have the pseudo-headers outside the header block, and they are placed at the end. Might be screwing up the fetch.