DNS propagation - why is a subdomain only accessible minutes after it's creation (own nameservers)

Question

we have a question about the behavior of DNS propagation for subdomains.

Here's the scenario we are trying to achieve:

User1 registers at our site "company.com". A subdomain "user1.company.com" is automatically created, not as add-on domain of "company.com", but as standalone account in WHM.

So, an own zone is being created & an A record is being set (same IP as company.com) NS records are also set to "ns1.domain2.com + ns2.domain2.com", our own nameservers (no clustering, 2 different IP's, BIND method, they are provided for the moment by the same WHM installation as for company.com & subdomains) Domain2.com is handled by our registrar GoDaddy, nameservers ns1 + ns2 are also defined there.

Our problem is that right after the creation the domain "user1.company.com" is not immediately accessible (unlike an add-on domain).

When we nslookup for example the new domain "user1.company.com" with the nameservers from our ISP, right after it's creation, we get "Non-existent domain". Then, after 1-15 minutes, and depending on the DNS we try, the nslookup answers with the correct IP address. The DNS from Google (8.8.8.8) for example immediately! answers with the correct IP address.

What exactly happens when the user tries to access his subdomain "user1.company.com"? Are our nameservers contacted to resolve the subdomain or are the subdomains somehow propagated across the DNS servers worldwide? But why does Google's DNS answer immediately, propagation can't be so fast!

Any ideas where may be the problem and how to make a subdomain account accessible immediately after its creation, regardless of the DNS being used by the user?

Many thanks

Marc

Answer 1

My understanding of DNS is that to resolve a url, the process is as follows:

Usually when we contact a DNS server it is not the first time it has received a request for a specific website, they keep the records from previous requests until they expire. The amount of time till expiry (TTL - a value in seconds) varies depending on your settings. If you change you records, the chances are there are plenty of DNS servers out there with these cached records. Once these records expire or the first time the server receives a request for a url it does not know, the DNS server does a 'recursive lookup' in order to get fresh data.

The following is a recursive lookup of a.contoso.com. (Notice the dot at the end which is normally hidden)

The process starts off working backwards, starting with the hidden dot at the end of a URL:

1 - Contact the root name servers (dot servers), their IP addresses are pre-loaded onto DNS servers, these IP's are the same for every server and don't change, they give an IP address for the .COM DNS servers (or whichever TLD you use such as .net)

2 - You then query the .COM DNS for 'contoso' in contoso.com. (this may be where your problem lie if you've changed nameservers)

3 - You then query contoso.com's DNS (your nameservers) for the 'a' in a.contoso.com

ad infinitum ( b.a.contoso.com, 2.ww.c.b.a.contoso.com...)

The process of these recursive lookups replacing stale records is known as propagation.

I would presume your not getting a request sent to your name server because of propagation during a switch-over, your server is not replicating the A record to the nameserver correctly

Domain propagation is only an issue when transferring a domain, not when a new one is created as your not having to deal with out of date records since those records never existed. The request will go straight to the source.

Answer 2

This is most likely due to negative caching. That is to say that a DNS server remembers that the subdomain doesn't exist, and replies with NXDOMAIN without checking if it's still true. You can find the TTL for negative caching in the SOA record of the apex domain. In your case: dig company.com.

Another, unlikely, cause could be that not all the authoritative DNS servers are in sync yet. Since they operate independently, it can take some time before all authoritative DNS servers have the same records. This is called zone transfer, and happens through the AXFR and IXFR pseudo record types.

To debug this issue, visit a DNS lookup tool, and check the following:

• Do the authoritative servers reply with the correct records?
• What is the negative cache TTL in the SOA record?

As for why some recursive DNS servers immediately reply with the correct response, they might have made different trade-offs in how aggressively they cache records. TTLs are not always followed by DNS resolvers. Or the caches of some resolvers might happen not to have this record yet, causing them to ask the authoritative DNS server immediately.