After successful login I'm trying to navigate to https://plus.google.com/u/0/?tab=wX but casperjs hangs.
The last output is:
[debug] [phantom] Navigation requested: url=https://clients6.google.com/static/proxy.html?jsh=m;/_/scs/apps-static/_/js/k=oz.gapi.en.Z6gj5B0lzyA.O/m=__features__/am=IQ/rt=j/d=1/t=zcms/rs=AItRSTPU0_gqMrtQ831rDdqYv8Z1ZnxcbA#parent=https://tal kgadget.google.com&rpctoken=640385943, type=Other, willNavigate=true, isMainFrame=false [debug] [phantom] Navigation requested: url=https://plus.google.com/hangouts/_/pre?hl=en&authuser=0, type=Other, willNav igate=true, isMainFrame=false [debug] [phantom] Navigation requested: url=https://plus.google.com/u/0/_/blank, type=Other, willNavigate=true, isMainFr ame=false
It seems like it hangs on hangouts iframes. How to block loading of certain internal URLs?
I tried:
// try to hide hangouts
casper.on('page.resource.requested', function (requestData, request) {
if (requestData.url.indexOf('plus.google.com/hangouts') != -1) {
this.echo('RESOURCE ABOTRED ' + requestData.url);
request.abort();
return;
}
});
casper.on('navigation.requested', function (url, navigationType, navigationLocked, isMainFrame) {
if (url.indexOf('plus.google.com/hangouts') != -1 || url.indexOf('talkgadget') != -1) {
// this.echo('ALARM!!!' + url);
willNavigate = false;
// request.abort();
return false;
}
});
but its does not help at all as navigation requested is information event I cant block it!
Please advise.
It seem like google+ main page is completely protected from scraping with phantomjs.
It appears this approach may work:
casper.on('navigation.requested', function (url, navigationType, navigationLocked, isMainFrame) {
if (url.indexOf('plus.google.com/hangouts') != -1 || url.indexOf('facebook.com') != -1) {
this.page.navigationLocked = true;
}else{
this.page.navigationLocked = false;
}
});
Update: Appears this approach may prevent navigation of other concurrent steps, back to the drawing board...
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With