Softlayer ISP Blocking Network dropping out when connecting to wp_remote_post Wordpress

Question

We are really looking for help from the community we are getting issues where our clients website can no longer reach my api and timeout with.

Connection timed out after 10001 milliseconds. 

We have contacted our hosts media temple and they cant give me any resolution and we really need some help.

Here is a video of my issue we are experiencing.

http://bit.ly/1CEIYOH

Here is a trace route to our api, it works fine for many of our users but some are now getting issues like below.

1  50.23.120.193-static.reverse.softlayer.com (50.23.120.193)  0.513 ms  0.491 ms  0.481 ms
2  ae0.dar02.sr01.sjc01.networklayer.com (50.23.118.132)  0.366 ms ae0.dar01.sr01.sjc01.networklayer.com (50.23.118.130)  0.406 ms ae0.dar02.sr01.sjc01.networklayer.com (50.23.118.132)  0.302 ms
3  50.97.19.164-static.reverse.softlayer.com (50.97.19.164)  0.775 ms  0.701 ms ae8.bbr01.eq01.sjc02.networklayer.com (173.192.18.248)  0.803 ms
4  equinix-ix.sjc1.us.voxel.net (206.223.116.4)  0.741 ms  0.794 ms  0.868 ms
5  173.231.161.193 (173.231.161.193)  48.109 ms  44.392 ms  44.397 ms
6  0.te1-2.tsr1.dal2.us.voxel.net (107.6.99.249)  79.040 ms  79.065 ms  80.685 ms
7  0.te2-2.tsr3.iad1.us.voxel.net (173.231.161.189)  78.472 ms  78.438 ms  78.414 ms
8  173.231.160.90 (173.231.160.90)  81.183 ms  81.158 ms 173.231.160.82 (173.231.160.82)  78.348 ms
9  internap.iad1.us.voxel.net (173.231.161.74)  78.815 ms  78.794 ms  78.751 ms
10  border2.te4-1-bbnet1.wdc012.pnap.net (216.52.127.39)  78.455 ms border2.pc2-bbnet2.wdc012.pnap.net (216.52.127.76)  78.282 ms  78.261 ms
11  net2ez-3.border2.wdc012.pnap.net (64.94.31.142)  79.218 ms  79.159 ms  79.094 ms
12  * * *
13  * * *
14  * * *
15  * * *
16  * * *
17  * * *
18  * * *
19  * * *
20  * * *
21  * * *
22  * * *
23  * * *
24  * * *
25  * * *
26  * * *
27  * * *
28  * * *
29  * * *

Network just drops out here is Media Temples response.

We have received feedback from our provider's network engineer and he asked if your clients or the customers of softlayer open a ticket with softlayer. From what testing they have done it appears that there are some routing peering issues between various ISPs and softlayer.

New update

From your traceroutes the commonality appears to be either Softlayer, Internap or NetDC ISPs. I've spoken with NetDC and they claim they are not filtering anything. We have also reached out to softlayer but have not heard back from them. What would help narrow this down if we can run another script like you did earlier:

Another update

Ok, I've reached out to Softlayer (your customer's ISP), Internap (Intermediary ISP), and NetDC (Intermediary ISP). One of them is filtering things, and I'm hoping I get a quick response from them. It is out of our control at this moment, but hopefully these ISPs will be responsive and allow us to track down whatever is blocking your customers from reaching you.

Latest update

Good news, I got a response from Softlayer just as I sent you that last correspondence.  They are seeing the same issue from their end

and they believe the issue is within Internap. They have looped in Internap as well since they are direct customers of Internap so hopefully something will be discovered.

I was able to determine that there are some ip subnets of ours that are accessible by the Softlayer router that was having issues connecting to your IP. I need to verify that we can actually migrate you to one of those IPs.

I'm having an internet routing change made so Internap will hopefully not have the preferred route to your server and thus be bypassed, I believe the issue is not limited to just your IP which is why I'm taking this more extreme step.

Can someone please help

FINAL UPDATE.

The final solution was to completely move away from Media Temple they have REALLY let us down with their hosting after 3 weeks of our customers having issues they could not provide a solution and when asking them this clear question.

how come i can post to any other server but media temple from the ip thats having the issue?

Even the people on support where just literally just transferring me to another support operative because they could not answer the question.

What worries me is how come a hosting provider cant fix something like this? i lost quite a few paying customers due to this issue.

Solution was to move everything over to Amazon and install it onto a EC2 instance which i managed to do in a day which i wish i had done from the start i also install a ssl cert for $3 whereas the one on Media Temple are $75, i also paid $100 for their Cloudtech service as well.

Great thing about Amazon is even if a error like this did happen again i can just create a AMI and then just fire up a new EC2 and job done.

Everyone be careful about issues like this with third party hosting if you service relies on a API and you are given ISP issue then your business can go under very quickly if your hosting cannot find you a solution after 3 weeks of try belief me your users don't want to hear (oh i am just awaiting a response from Media Temple).

So solution use Amazon Web Service flexible powerful and fully in your control, rant over

Media Temple let us down massively.

Answer 1

What help are you expecting from the community, I am Sure that is just the tuning with the ISPs, it has nothing related to either deployment or hosting.

It will be a good , if you first check the stuff on local , and then a Dummy to check with different configurations over your ISPs so as to confirm what is the exact thing that is being blocked,.

then you will be able to solve your issue with ease..

as per your Question and my knowledge this is the Best answer i can provide