Duplicate GET requests being generated several seconds later in Chrome and Firefox by unknown user agent when a CGI script is invoked

Question

When I invoke a CGI script (GET request) using either Chrome or Firefox, I noticed in the Apache access log that quite a few seconds later a HEAD request and a GET request with the same URI were being generated. To make sure it was not a peculiarity of my Apache setup, I wrote a simple Perl script and installed it on my ISP's website. This script simply does it's own logging to a disk file every time it is invoked of the time, the URI, the user agent, the remote address and port:

#!/usr/bin/perl -wT
use strict;
use CGI;

my $cgi = new CGI;
print $cgi->header(-type=>'text/plain', -expires=>'-1d');
print "hello";

open (LOG, ">>printenv2.txt");
my ($sec, $min, $hr, $day, $mon, $year) = localtime;
my $timestamp = sprintf("%02d/%02d/%04d %02d:%02d:%02d", $mon + 1, $day, 1900 + $year, $hr, $min, $sec);
print LOG $timestamp, "\n";
my @keys = qw(REQUEST_METHOD REQUEST_URI HTTP_USER_AGENT REMOTE_ADDR REMOTE_PORT);
foreach (@keys) {
    print LOG "$_ = $ENV{$_}\n";
}
print LOG "\n";
close LOG;

I invoked the script once with http://localhost/friends/forms/cgi/printenv2.cgi?arg=1

The script was run three times, however, the last two runs occurred more than 14 seconds after my initial invocation. Here is the output of the log:

09/11/2015 19:25:26
REQUEST_METHOD = GET
REQUEST_URI = /friends/forms/cgi/printenv2.cgi?arg=1
HTTP_USER_AGENT = Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/45.0.2454.85 Safari/537.36
REMOTE_ADDR = 127.0.0.1
REMOTE_PORT = 58421

09/11/2015 19:25:40
REQUEST_METHOD = HEAD
REQUEST_URI = /friends/forms/cgi/printenv2.cgi?arg=1
HTTP_USER_AGENT = Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)
REMOTE_ADDR = 127.0.0.1
REMOTE_PORT = 58428

09/11/2015 19:25:41
REQUEST_METHOD = GET
REQUEST_URI = /friends/forms/cgi/printenv2.cgi?arg=1
HTTP_USER_AGENT = Mozilla/5.0 (X11; Linux x86_64; rv:10.0) Gecko/20150101 Firefox/20.0 (Chrome)
REMOTE_ADDR = 127.0.0.1
REMOTE_PORT = 58440

As you can see the user agent associated with the two spurious invocations is different. I ran SpyHunter to see if I had a virus. I tried disabling all extensions. I tried un-installng and re-installing Chrome. Nothing helps. This only occurs with Chrome and Firefox - the problem does not appear with Internet Explorer.

HELP!

Answer 1

I had tried doing a search on the complete user agent string and came up with nothing. But I now decided to try again just searching on "Gecko/20150101" and came up with an article at: https://www.quppa.net/blog/2015/07/26/realplayerrealdownloader-poses-as-firefox-running-on-64-bit-linux-and-sends-head-and-get-requests/

I had installed the latest version of RealPlayer (now called RealTimes) and it had installed a Video Downloader. This is not installed as an extension in Chrome and Firefox, but rather runs as a separate process. I disabled this "addon" and now the spurious HEAD and GET requests seem to have disappeared. What the hell were they thinking?