Consul service discovery with DNS on Nodejs

Question

TL;DR

Hi all, I'm trying to call nodejs backend microservices from a nodejs frontend written in Express, through Consul DNS interface but I am having errors.

I am using the nodejs dns api to set the dns for the sole node application in order to make subsequent dns.resolve() calls to the local Consul DNS interface.

Target

I would like to be able to make an http request to my backend service without the need to wire its IPs and ports in my client code. Neither I want to write a custom code to query the Consul HTTP API to get the ip:port couple for my service any time I need to call it.

Problem

Problem is that when I use axios (similar to request) to make an HTTP call to the backend service I always get an error because it cannot resolve the address. It seems Axios is not using the dns I previously setted with:

dns.setServers(['127.0.0.1:8600']);

Update_1

Setting the dns of the virtual machine (/etc/resolv.conf) to localhost e using the -recursor command line option for consul with the default dns everything works! I still would like to understand what I am doing wrong setting the dns only on my nodejs app.

Setup

• 1 FE node with a nodejs process running a simple webserver with Expressjs. In the app.get('/') route it make a REST POST call to a backend service called be01 through consul and axios (like request).
• 2 BE node with a nodejs process running a simple webserver with Expressjs exposing REST api.
• 1 Consul node with Consul running as a server.
• Every node has it own consul agent running and joined to the cluster.
• TCP ports are correctly open

This are the servers:

This is from the Consul UI:

Running consul members on the FE node I get:

agent-server  10.0.1.7:8301  alive   server  1.0.1  2         dc1  <all>
agent-be-01   10.0.1.5:8301  alive   client  1.0.1  2         dc1  <default>
agent-be-02   10.0.1.6:8301  alive   client  1.0.1  2         dc1  <default>
agent-fe      10.0.1.4:8301  alive   client  1.0.1  2         dc1  <default>

If I run dig @localhost -p 8600 be01.service.consul SRV on the FE node I correctly get this result (as in the Consul docs):

root@NGINXLB:~/node8080$ dig @localhost -p 8600 be01.service.consul SRV

; <<>> DiG 9.9.5-3ubuntu0.16-Ubuntu <<>> @localhost -p 8600 be01.service.consul SRV
; (1 server found)
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 43367
;; flags: qr aa rd; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 5
;; WARNING: recursion requested but not available

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 4096
;; QUESTION SECTION:
;be01.service.consul.       IN  SRV

;; ANSWER SECTION:
be01.service.consul.    0   IN  SRV 1 1 8081 agent-be-02.node.dc1.consul.
be01.service.consul.    0   IN  SRV 1 1 8080 agent-be-01.node.dc1.consul.

;; ADDITIONAL SECTION:
agent-be-02.node.dc1.consul. 0  IN  A   10.0.1.6
agent-be-02.node.dc1.consul. 0  IN  TXT "consul-network-segment="
agent-be-01.node.dc1.consul. 0  IN  A   10.0.1.5
agent-be-01.node.dc1.consul. 0  IN  TXT "consul-network-segment="

;; Query time: 2 msec
;; SERVER: 127.0.0.1#8600(127.0.0.1)
;; WHEN: Fri Dec 01 10:09:00 UTC 2017
;; MSG SIZE  rcvd: 246

This is the BE service code:

var express = require('express');
var bodyParser = require('body-parser');
var app = express();
var jsonParser = bodyParser.json()
var urlencodedParser = bodyParser.urlencoded({ extended: false })

app.post('/api/getUserTiles', jsonParser, function (req, res) {
  console.log("> API request for '/api/getUserTiles'");
  if (!req.body) { return res.sendStatus(400) }
  else {
    var user = req.body.user;
    console.log("User: " + user);
    res.json({
      "authorizedTiles": [
        {"tileID": "profile"}
        ,{"tileID": "search"}
        ,{"tileID": "test"}
      ],
      "email": "[email protected]",
      "userName":"Max",
      "has_error": false,
      "error_message": ""
    });
  }
});

var server = app.listen(8080, function () {
  var host = server.address().address
  var port = server.address().port
  console.log("Example app listening at http://%s:%s", host, port)
})

Calling it from the FE using its ip:port with curl works without problems:

root@NGINXLB:~/node8080$ curl -d="user=Max" -X POST http://10.0.1.5:8080/api/getUserTiles
{"authorizedTiles":[{"tileID":"profile"},{"tileID":"search"},{"tileID":"test"}],"email":"[email protected]","userName":"Max","has_error":false,"error_message":""}

The web service on the FE node, simplifying, is sort of this:

var axios = require('axios');
var dns = require('dns');
var consul = require('consul')();

dns.setServers(['127.0.0.1:8600']);
//console.log(dns.getServers());

dns.resolveSrv("be01.service.consul", function(err, records){
        console.log("\nDNS SRV query");
        if(err){
                console.log(err);
        }else{
                console.log(records);
        }
});

consul.health.service({
        "service": "be01",
        "dc": "dc1",
        "passing": true
        }, function(err, result){
                console.log("\nConsul.health.service query:");
                if(err){console.log(err); throw err;}
                else if(result.length > 0){
                        for(var i=0; i<result.length; i++){
                                console.log(result[i].Node.Address + ":" + result[i].Service.Port);
                        }
                }
});

axios.post("http://be01.service.consul/api/getUserTiles", {'user':'Max'})
  .then(function(response){
      if (response.data.has_error) {
       console.log("\nRESPONSE HAS ERROR!");
      }else {
        console.log("\nSUCCESS!");
      }
  })
  .catch(function(err) {
        console.log("\nERROR CALLING THE BE SERVICE");
  });

Running it I get the following result:

root@NGINXLB:~/node8080$ node app.js 

DNS SRV query
[ { name: 'agent-be-01.node.dc1.consul',
    port: 8080,
    priority: 1,
    weight: 1 },
  { name: 'agent-be-02.node.dc1.consul',
    port: 8081,
    priority: 1,
    weight: 1 } ]

Consul.health.service query:
10.0.1.5:8080
10.0.1.6:8081

ERROR CALLING THE BE SERVICE
{ Error: getaddrinfo ENOTFOUND be01.service.consul be01.service.consul:80
    at errnoException (dns.js:50:10)
    at GetAddrInfoReqWrap.onlookup [as oncomplete] (dns.js:92:26)
  code: 'ENOTFOUND',
  errno: 'ENOTFOUND',
  syscall: 'getaddrinfo',
  hostname: 'be01.service.consul',
  host: 'be01.service.consul',
  port: 80,
  ...
  ...

Conclusion

As you can see the nodejs client is calling the be service but it fails trying to resolve 'be01.service.consul'. Moreover it is using the port 80 instead of 8080 or 8081 as provided by the Consul DNS interface. What am I missing?

Answer 1

The problem is that axios is using dns.lookup which does not use the servers set by dns.setServers. dns.lookup and dns.resolve do not use the same mechanism for resolving.

In pure node the possible options are to either resolve the domain name to an IP using dns.resolve* before calling axios or something like this interceptor example (I haven't tried it).

Probably better solution is not to do this in node but use the bind option of the consul agent running locally to do the dns resolving.