"undefined" and "cache/xxx" appened to URLs

Question

First of all I think this is more or less the same problem as "undefined" randomly appended in 1% of requested urls on my website since 12 june 2012 but since I'm a new user and cannot comment on this post and it has no solution yet, I can only ask a new question.

Since 12 June 2012 14:22 EET (the moment when the first error happened) we are experiencing very weird problems: Less than 1% of the requests to our site have the "undefined" string appended to the end or replacing a valid part of the url and the referrer is a completely valid URL to the site. For example we get a request for http://example.com/foo/undefined with referer http://example.com/foo/bar or request to http://example.com/undefined with referer http://example.com/ (the homepage). These URLs come from diverse client IP addresses, diverse ISPs and the browser is most often Chrome, but happens also with IE and Firefox 3.5. It seems like something is rewriting the URL to something invalid, keeping the original URL in the referrer tag. We can't reproduce this problem.

We are also exeperiencing another problem mentioned in the comments of the source post: we are receiving URL requests of the form http://example.com/cache/xxx where xxx looks like 32 character MD5 string (exmple: 3d453e96e68cc01ced7920ae77356078 or bbc80a4244caf556fdcaa9fb60231af7). We don't have the "cache" string in any of our valid URLs. One and the same xxx string may come from diverse IPs for several days, even weeks. And all these weird requests come from a Chrome browser. This problem didn't started on 2012-06-12. It happens at least since the beginning of the year but is much more rare than the first one. We can't reproduce this problem too.

Our web site is on IIS, the client side is heavily Javascript-based and we are using the Prototype framework (not jquery).

Answer 1

Answer's Summary

If there is an error and no one knows where it is, you will have to debug your code looking for statements that could be the error's starter. Nowadays we have some good tools that help us in this task.

Assumptions

1. If JavaScript is used to set or change image locations, it sometimes happens that an undefined makes its way into the URI.

2. Would be some javascript making an http request and, due to a bug, instead of requesting an absolute url it is requesting "undefined".

Conclusion: The issue, I believe is caused by JS.

Debugging on Firefox and Chrome

Installing and Running

• You should download FIREBUG ADDON, if you will attempt in Firefox. Then press F12 and you will see a window with some tabs. You will need to activate some tabs: Like 'Console', 'Script', 'Net'. The activation is provided by opening them up, reading the panel's information and clicking in the activate link.

• For Chrome, just check if CHROME'S DEVELOPERS TOOL is already installed. Then press F12.

Debugging

• In Firefox, navigate to one bugged page with Firebug opened and the Net panel activated. In the Net panel there will be a few options: 'Clear', 'Persist', 'All', 'Html', etc. Make sure the option ALL is selected. Don't do anything on the page and try not to mouse over anything on it. Look through the GET and POST requests. The request for the invalid URL will be probably displayed red and have a status of 404 Not Found or alike. If nothing appeared in the 'All' tab, make sure that you opened Firebug before entering in the page. For troubleshooting refresh the page and redo the section.

• In Chrome, navigate to one bugged page with CDT, short for Chrome Developer Tools, opened and the Network panel selected. As Firebug, Make sure the option ALL is selected it will be on the in the lower left corner after icons. Don't do anything on the page and try not to mouse over anything on it. The request for the invalid URL will be probably displayed red and have a status of 404 Not Found or alike. If nothing appeared in the 'Network' tab, make sure that you opened CDT before entering in the page. For troubleshooting refresh the page and redo the section.

Still can't see it

• For both, start to trigger some Javascript events, such as hover, click e etc, and keep your eyes on the panel. It will trace all your requests, the good and the bad ones. Stay focused on the bad requests and this will need creativity if bad requests don't come up. The events will trigger requests and you will be able to see them. Seeing them you will be able to know or discover a problem and probably the reason for that problem. For more Resources CDT Network and Firebug Net.

PS.: If the panel get dirty, click on clear button or icon to clean the requests from panel

• If you submit the page and see a failed request go out really quick but then lose it because the next page loads, enable persistence by clicking 'Persist' on Firebug or 'Preserve log' icon in the top left of the Net panel or in the lower left corner of the Network panel. Once it does, and it should, consider what you did to make that happen. See if you can make it happen again. After you figure out what user interaction is making it happen, dive into that code and start looking for things that are making invalid requests and never forget about seeing the request header. You can use the Script tab to setup breakpoints in your JavaScript and step through them. Investigate event handlers done via $(elemment).bind/click/focus/etc or from old school event attributes like onclick=""/onfocus="" etc.

• If you can't trigger bad request, then whatever is causing it isn't being triggered by your tests. Try using more things. The point is you should be able to make the request happen somehow. You just don't know yet. It has to show up in the Net panel. The only time it won't is when you aren't doing whatever triggers it.

What we are looking for in your code

var url = workingUrl + someObject['someProperty'];

var url = workingUrl + someObject.someProperty;

var url = workingUrl + $.md5("message"); //For the /cache/ problem. JQuery Plugin

var url = workingUrl + hex_md5("message"); //For the /cache/ problem. MD5 Implementation

<a href= "javascript:;" class="small_img" >
  //Code here
</a>

window.open('/image.?url_image=' + currentImage,winName,features);

PS.: someObject might be an object {}, an array [], or any of the internal browser types. The point is that a property will be accessed that doesn't exist.

Conclusion: There is no free lunch here to know what exactly is going on. However using debug's methods you should be able to discover, get closer or understand the issue.

Answer 2

What I can assume from the given situation is that since the website is Heavily Javascript Based... there are lots of ajax call being made. And I am more likely assuming that these are ajax requests with undefined parameter generated in Javascript code.

For debugging purpose (in development mode) You should be validating the ajax urls before making the ajax requests and whenever you find undefined divert it to the dummy page and try to grab as much details as you can.

This thing occurs mostly when one asynchronous requests depends on the other asynchronous requests. Without looking at the code or the example its very hard to suggest anything, but I hope putting some more debugging capability in your application and grabbing more and more details will able to identify the issue.

Somewhere I get the feeling its not the big deal, just some point in your heavy javascript app which you have misses or went unnoticed.

Post some more details so that we here at stackoverflow can help you out.

Answer 3

What if it's just some browser/platform bug?

I'm seeing the /undefined 404s on a Drupal 7 website AND also on a plain vanilla HTML+CSS (no JS whatsoever) mini-site connected to the Drupal one to serve some specific content.

It looks like it's almost always a Chrome/Safari problem, with a few IEx here and there, like this one: Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; InfoPath.2; .NET4.0C; .NET4.0E)

From a quick log search, it NEVER happens with Firefox.

The /undefined request comes at the end of page rendering, after CSS and images have been requested. And it ** does not ** happen again for a second page that is almost identical and re-uses layout, CSS and images from the first one.

My two cents...

Answer 4

Based on this post, I reverse-engineered the "Complitly" Chrome Plugin/malware, and found that this extension is injecting an "improved autocomplete" feature that was throwing "undefined" requests at every site that has a input text field with NAME or ID of "search", "q" and many others.

I found also that the enable.js file (one of complitly files) were checking a global variable called "suggestmeyes_loaded" to see if it's already loaded (like a Singleton). So, setting this variable to true disables the plugin.

TL:DR;

To disable the malware and stop "undefined" requests, apply this to every page with a search field on your site:

<script type="text/javascript">
    window.suggestmeyes_loaded = true;
</script>

This malware also redirects your users to a "searchcompletion.com" site, sometimes showing competitors ADS. So, it should be taken seriously.