Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Url fetch to third party server keeps timing out from cron job?

I have a cron job that runs every 15 minutes. It starts a local servlet which tries to open a url from a server I have running (not part of app engine):

<cron>
  <url>/utils/doUpCheck</url>
  <description>Check that our third party server is up.</description>
  <schedule>every 5 minutes</schedule>
</cron>

in the servlet at /utils/doUpCheck:

String url = "https://example.com:9443/echo";
HttpURLConnection conn = (HttpURLConnection)new URL(url).openConnection();
conn.setConnectTimeout(10000);
conn.setReadTimeout(10000);
...

but I'm logging a timeout exception every time it runs:

java.net.SocketTimeoutException: 
  Timeout while fetching URL: https://example.com:9443/echo
  at com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:142)
  ...

I'm sure the server at example.com is running though, it's easy to check from a browser whenever I see this error occur. Any idea why this could be happening? Of course as a sanity check, I've replaced the url with "https://google.com", and it works ok. So seems like something specific with my server, but again it works fine from other apps / browsers,

Thanks

https://developers.google.com/appengine/docs/java/urlfetch/

------- Update ----------

One other item, I recently changed the IP address of my third-party server. So maybe app engine has some stale dns entry for that domain name? Like I said earlier, other apps / browsers all resolve the domain name ok.

------- Update Again ----------

I opened up an additional port on the same machine to serve http requests (using the same webserver app instance, too). So app engine can resolve this just fine:

http://example.com:9000/echo

but I continue to get the timeouts to:

https://example.com:9443/echo

I'm sure "http://example.com:9443" is serving though, as I can hit it with any browser ok. It's also using a cert from a trusted authority, I haven't had to make any exceptions for the browsers when visiting the url.

I'm stuck here, any ideas?

------- Update Again ----------

I've added some code to try the same with the low-level URL fetch service, same problem, socket timeout exception.

https://developers.google.com/appengine/docs/java/javadoc/com/google/appengine/api/urlfetch/package-summary

like image 651
user1219278 Avatar asked Jan 27 '14 05:01

user1219278


2 Answers

The certificate used for https encryption on your third party server must be signed by an authority that the AppEngine software trusts. A self signed certificate should result in a connection fail although timeout is not the expected result. Try http instead of https as another sanity check (test that with a browser first).

like image 97
Martin Berends Avatar answered Nov 12 '22 21:11

Martin Berends


Check how long this request normally takes. the handler invoked by cron must respond inside 60 sec. Also you by default urlfetch service has a timeout of 5 secs. So you need to use setTimeout to give you a longer potential response time -

See https://developers.google.com/appengine/docs/java/urlfetch/

like image 2
Tim Hoffman Avatar answered Nov 12 '22 19:11

Tim Hoffman