Check if website exists

Question

I'm trying to make a is-website-down utility with java, but I have some problems.

Is there a way to check if a website exists? I tried this to see if a website is down:

URL url = new URL("http://localhost");
HttpURLConnection httpConnection = (HttpURLConnection) (url.openConnection());
int code = httpConnection.getResponseCode();
System.out.println("code: " + code);

It goes through IOException for Connection refused: connect when i.e. i try to connect to localhost while there is no active http server listening (the site is down).

I thought it would happen the same thing with some site that actually doesn't exist i.e.

URL url = new URL("http://www.sdfasfjkhaslfjkhaslkdjfhasldkjf.it");

But i receive a HTTP Status code 200 because my ISP automatically redirects me to a random advertisement page if the site i'm searching for doesn't exist.

So, if a site is down, my program says "Well, your website is down", but if the site doesn't exist my program says "Oh, your website is up and running!", and that's not really good.

Is there a way to check if a website exists?

Answer 1

Reading your comments about the ISP DNS poisoning. Your ISP is giving/leasing you a DNS server they are in control over that catches all NXDOMAIN responses and return a poisoned result to a server they control.

I would do a DNS check for an A record with something like Google Public DNS or some other publically known valid DNS server before you do the HTTP check. Google Public DNS does return NXDOMAIN when the domain is invalid. In Java, you may need to use something like dnsjava to get direct access to the DNS records you need.

In the end, do two checks, one for a valid DNS record against a known good source, then another one to see if HTTP is alive.