IdHTTP.get returning HTTP1.1/ 403 Forbidden

Question

I'm trying to access an update.txt file on my website, using a DelphiXE compiled program and the IdHTTP component.

The code I'm using is as follows:

procedure TFormAbout.SpeedButtonUpdateClick(Sender: TObject);

function CheckUpdates: String;
var lIdHttp: TIdHTTP;
begin
  lIdHttp := TIdHTTP.Create(nil);
  result := lIdHttp.Get('http://www.test.com/test_down_load/update.txt');
end;

var
sWebVersion: String;
sVersionList: TStringList;

begin
try
  sWebVersion := Checkupdates;
except
  on E: Exception do
  begin 
    ShowMEssage(E.ErrorMessage);
    MessageDlg('An Error occured checking for an update.',mtError,[mbOK],0);
  end;
end;
if sWebVersion <> '' then
  begin
    sVersionList.CommaText := sWebVersion;
    ShowMessage('Version: ' + sVersionList[0] + ' - ' + 'Date: ' + sVersionList[1]);
  end;
end;

This however results in error: HTTP1.1/ 403 Forbidden

The IdHTTP component has been set up with the following properties.

HandleRedirects := true;
HTTPOptions [hoForceEncodeParams];
ProtocolVersion := pv1_1;
Request.UserAgent := Mozilla/5.0 (compatible; Test)

If I enter the URL in an IE browser, it returns the file without errors, but when accessing from my program, I get the error. Any pointers will be appreciated. .htaccess is correct for the site. Permissions for the file are correct on the website: 0644.

Do I have to set any of the other properties for the IdHTTP component. I only have this component on the about form. Do I need anything else.

The updateinfo.txt file simply contains the text in quotes: "18.3.5,2011/12/17"

I have simply used "test" here in place of my actual program name and URL.

Regards Adrian

Answer 1

I run into the exact same problem when using Indy's Get() function.

It more than likely that you are getting this error because you haven't set the UserAgent property and the website knowing that you are not accessing the file as a Browser is kicking up a fuss.

function CheckUpdates: String;
var lIdHttp: TIdHTTP;
begin
  lIdHttp := TIdHTTP.Create(nil);
  //avoid getting '403 Forbidden' by setting UserAgent
  lIdHttp.Request.UserAgent :=
      'Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20100101 Firefox/12.0';
  result := lIdHttp.Get('http://www.test.com/test_down_load/update.txt');
end;

Similar question, but correct answer was logged here: https://stackoverflow.com/questions/10870730/why-do-i-get-403-forbidden-when-i-connect-to-whatismyip-com