Node.js request directly to URL with options (http or https)

Question

I think I'm missing something about http and https requests

I have a variable that contains a URL, for example:

http(s)://website.com/a/b/file.html

I would like to know if there's a easy way to make a request to that URI to get the data

To make a http(s)Request, here's what I have to do now:

1. Test if the URL is http or https to make the appropriate request
2. Remove the http(s):// part and put the result in a variable (If I specify http or https in the hostname, I get an error)
3. Separate the hostname from the path: website.com and `/a/b/file.html
4. Put this variables in the options objects

Is this a must or are they easier solutions that don't involve getting out the hostname and path, and testing if the site is in http or https ?

Edit: I can't use http.get as I need to put some specific options

Answer 1

In order to get all components out of URL you need to parse it. Node v0.10.13 has stable module for it: url.parse

This is simple example how to do so:

var q = url.parse(urlStr, true);
var protocol = (q.protocol == "http") ? require('http') : require('https');
let options = {
    path:  q.pathname,
    host: q.hostname,
    port: q.port,
};
protocol.get(options, (res) => {...

Answer 2

For those ending up here, protocol includes :, and pathname does not include search so it must be added manually. Parameters shouldn't be parsed as they are not needed (so you can save computing time :)

Also it's not really a best practice to require inside a function and probably this code will end up inside a function so having all this improvements, so I would rewrite the answer to something like this:

import * as url from 'url';
import * as https from 'https';
import * as http from 'http';

const uri = url.parse(urlStr);
const { request } = uri.protocol === 'https:' ? https : http;
const opts = {
    headers, // Should be defined somewhere...
    method: 'GET',
    hostname: uri.hostname,
    port: uri.port,
    path: `${uri.pathname}${uri.search}`,
    protocol: uri.protocol,
};
const req = request(opts, (resp) => { ...