Pinterest rich pin not working & validator fails?

Question

We've been struggling with this problem for several months yet no avail. Rich pins on Pinterest used to work for our site but suddenly they stopped working and one after another all the rich pins (clickable title and price) disappeared for the pins of our site.

For example, with this product, we clearly have the open graph information up front in the head section of the HTML code:

<meta property="og:type" content="product" />
<meta property="og:title" content="Backless Gray Lace Tulle Flower Girl Dress with Big Bow" />
<meta property="og:description" content="This dress is made of high quality lace and tulle fabric; Unique Cross back with straight neckline in the front; Big ... Shop now use NYC2018 for 9% off TODAY!" />
<meta property="og:url" content="http://www.princessly.com/backless-gray-lace-tulle-flower-girl-dress-with-big-bow.html" />
<meta property="og:site_name" content="Princessly.com" />
<meta property="og:price:amount" content="49.62" />
<meta property="og:price:currency" content="USD" />
<meta property="og:availability" content="instock" />

As per their rich pin requirements here. It's all working good when we first implemented it on our site around half a year ago. Unfortunately for some reason it stopped working 1 month ago. See the pin page. It shows none of these rich pin information no more.

When we try the product page with the rich pin validator (https://developers.pinterest.com/tools/url-debugger/), it's still NOT working and keeps giving this error for every URL we test:

We were unable to retrieve any data from your URL.

Like this: http://www.princessly.com/pinterest_error.png

It's not a very useful error message. We are not sure if they can't access the page (http code? blank page or whatever?) or just don't find any open graph information there.

We tried everything asking around webmaster forums and seeking Pinterest official support which basically was not helpful other than giving us the docs we already knew and blamed our server for blocking access of their bot but after confirming with our hosting team several times, there were no errors at all regarding Pinterest.com accessing our site:

54.85.173.173 - - [11/Oct/2016:21:22:00 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)"
54.227.228.14 - - [11/Oct/2016:21:27:20 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)"
52.87.185.68 - - [11/Oct/2016:21:28:06 -0500] "GET /u-back-ivory-cotton-ruffle-neckline-flower-girl-dress.html HTTP/1.1" 200 22477 "-" "Pinterest/0.2 (+http://www.pinterest.com/)" 

Since we are a women products merchant, Pinterest means quite a sum of the sales. We have seen considerable drop of revenue since rich pin disappearance. We believe this issue is costing real money as we speak.

We are desperate and think we should try our luck here on SO. Not sure if this fits in programming questions but it sure involves code of HTML and Open Graph and one of the major social network who offers developer API.

As we can't get our hands on Pinterest bot and how it sees our page, there seems very limited ways for us to debug this.

Any idea how this happens and how to debug & fix it? Any Pinterest staff developer here could offer some insight? Thank you!

UPDATE 1

Here are the HTTP USER AGENTS I recorded via index.php of Princessly.com for the last 24 hours that contain 'Pinterest': http://www.princessly.com/Pinterest.txt

My host WiredTree was kind enough to check all this again for me. Seems Pinterest bots were indeed able to access my site no problem.

---

On checking the access logs for the site "princessly.com", I can see successful accesses from "Pinterest" to the site "Princessly.com". Please see the snippet from logs.

======

194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/bkg_btn_sprite.png HTTP/1.1" 200 2503 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/" 194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/tabs_border.gif HTTP/1.1" 200 43 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/" 194.103.189.40 - - [15/Nov/2016:03:32:31 -0600] "GET /skin/frontend/default/forest_fashion/images/pager_bg.gif HTTP/1.1" 200 102 "http://www.princessly.com/skin/frontend/default/forest_fashion/css/styles.css" "Pinterest/0.2 +http://pinterest.com/"

======

Is there any way I can find the name of the Pinterest bot That I can detect as HTTP USER AGENT?

From the logs I can see that the the Pinterest bot tag is "Pinterest/0.2 +http://pinterest.com".

1. Can we know if the requests made by Pinterest bot timed out?

I have checked the Apache logs for the IP addresses from which the Pinterest access were noted and I could not find any time out errors.

2. How do we know if we block AWS requests?

On checking I could not see any block rules setup for the site "Princessly.com". I have checked the .htaccess file and robots.txt file for the site. I have now whitelisted the AWS IP address range as per the link you have provided.

Can you please check the details now and let me know if there is any user panel for Pinterest or other details so that we can dig deeper.

---

UPDATE 2

Following advice from @Mackan, I created a few identical bare minimal pages exactly the same with each other and with correct open graph information, and uploaded them to several different sites. I'm recording all the PHP USER AGENT that accesses the page:

1. Sites on the same server as Princessly.com

Site IP: 173.199.151.128

http://www.princessly.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.princessly.com/Pinterest_agents.txt

-- FAIL

Site IP: 173.199.151.128

http://www.usabledatabases.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.usabledatabases.com/Pinterest_agents.txt

-- FAIL

Site IP: 173.199.151.130

http://www.bintley.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.bintley.com/Pinterest_agents.txt

-- FAIL

2. Different server than Princessly.com

https://www.thedataplanet.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

https://www.thedataplanet.com/Pinterest_agents.txt

-- SUCCESS

3. Another different server than Princessly.com

http://www.formkid.com/ivory-lace-tulle-wedding-flower-girl-dress-with-sheer-neck-TEST123.php

http://www.formkid.com/Pinterest_agents.txt

-- SUCCESS

After testing with the rich pin validator, it seems it's a server issue. The request never reaches the PHP file for all sites on the Princessly.com server.

WiredTree support was very kind to help me in finding the culprit but after shifting through over 5 support staff, nothing promising thus far. We have gone through .htaccess, Apache conf, modSecurity, firewall, whitelisting Pinterest IPs, AWS IPs, etc. Nothing worked.

This is really really weird.

UPDATE 3

Even weirder is, if you look at http://www.princessly.com/Pinterest_agents.txt you can see there are INDEED entries of "Pinterest/0.2 (+http://www.pinterest.com/)" there.

However if you give it a manual test, it FAILS and DOESN'T record the access at all.

It seems to be breaking intermittently, maybe.

This is pretty much in the quantum physics arena and I don't know what to do now, except the last resort that is to order and provision a new server and migrate Princessly.com there see how it goes.

Your idea?

Answer 1

We have the same issue. In our case this is being caused because the server has multiple SSL certificates and uses SNI so the client can get the certificate. However Pinterest is using curl on the back end which is NOT SNI aware. We have logged this with Pinterest

Answer 2

After debugging the issue, using https://developers.pinterest.com/tools/url-debugger/, the problem seems to stem from the server-side.

If you're seeing We weren't able to retrieve any data from your URL, then Pinterest may be timing out before we can create a Pin on your site. Unfortunately, this isn't something we can fix on our end, and usually happens when a site is slow. This error also appears if your site blocks Amazon Web Services (AWS) requests. You'll have to remove this block before applying for Rich Pins.

Creating some "minimal but working" pages and running them through the validator made the issue clearer, but unfortunately didn't resolve it. Either this is a problem with the local server (Apache) or possibly an issue with the pinterest bot.

The Pinterest bot seems to hit the server OK, but the validator can't parse or reach the endpoint. Since the minimal examples returns in ~less than a second, and contains a bare minimum to satisfy Pinterests requirements, it's clear that the issue is not within your code.

Migrating the solution to another host seems to have fixed the problem.

Edit

The new host seems to be working better (using one of the minimal examples):