Accessing JSON data after Indy POST

Question

I am struggling to know how to access the response to an Indy POST request. I post the data either as JSON or paramstring. My code when using JSON is as follows.

params := TStringList.Create;
try
  params.Text :=
    '{'
    + format ('"client_secret":"%s",', [FilesFrm.ClientSecret])
    + format ('"client_id":"%s",', [FilesFrm.ClientId])
    + '"grant_type":"authorization_code",'
    + '"redirect_uri":"http://localhost:8080",'
    + format ('"code":"%s"', [fCode])
    + '}';
  idLogFile1.Active := true;
  // Make sure it uses HTTP 1.1, not 1.0
  IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoKeepOrigProtocol];
  IdHTTP1.Request.ContentType := 'application/json';
  IdHttp1.Request.Accept := 'application/vnd.hmrc.1.0+json';

  try
    result := IdHTTP1.Post (
      'https://test-api.service.hmrc.gov.uk/oauth/token',
      params);
  except
    on E: Exception do
    memo1.lines.add (E.ClassName + ': ' + E.message);
    end;

  memo1.Lines.add (result);
  memo1.Lines.add (idHTTP1.ResponseText);
finally
  params.free;
  end;

The result of printing out result and RepsonseText is just

EIdHTTPProtocolException: HTTP/1.1 400 Bad Request

HTTP/1.1 400 Bad Request

However, because I have a TidLogFile component attached to the TidHTTP, I can see what actually arrives, which is the following.

Recv 2/1/2019 7:56:07 AM: HTTP/1.1 400 Bad Request<EOL>
Content-Type: application/json<EOL>
Content-Length: 72<EOL>
Cache-Control: no-cache,no-store,etc, etc...
; Secure; HTTPOnly<EOL><EOL>
{"error":"invalid_request","error_description":"grant_type is required"}

Aside from the fact that grant_type appears to be in the original request data, I would like to be able to access the JSON response at the end, since "grant_type_is_required" is much more helpful than "Bad request", but I cannot find where it is.

I have subsequently found Response.ContentLength, which contains the value 72, and Response.ContentStream, which should in theory contain the 72 bytes of data, but produces access violations when I try to extract the data.

len := idHTTP1.Response.ContentLength;
memo1.Lines.Add(format ('Length = %d', [len]));
if assigned (idHTTP1.Response.ContentStream) then
  begin
  //idHTTP1.Response.ContentStream.Position := 0;
  result := ReadStringFromStream (idHTTP1.Response.ContentStream, len);
  end;
memo1.lines.add (result);

Answer 1

In addition to mjn42's answer, which is technically correct, TIdHTTP also has optional hoNoProtocolErrorException and hoWantProtocolErrorContent flags in its HTTPOptions property, which you can enable to avoid the EIdHTTPProtocolException being raised and to populate your result variable with error data, respectively:

params := TStringStream.Create(
  '{'
  + format ('"client_secret":"%s",', [FilesFrm.ClientSecret])
  + format ('"client_id":"%s",', [FilesFrm.ClientId])
  + '"grant_type":"authorization_code",'
  + '"redirect_uri":"http://localhost:8080",'
  + format ('"code":"%s"', [fCode])
  + '}',
  TEncoding.UTF8);
try    
  IdLogFile1.Active := true;

  // Make sure it uses HTTP 1.1, not 1.0, 
  // and disable EIdHTTPProtocolException on errors
  IdHTTP1.ProtocolVersion := pv1_1;
  IdHTTP1.HTTPOptions := IdHTTP1.HTTPOptions + [hoKeepOrigProtocol, hoNoProtocolErrorException, hoWantProtocolErrorContent];

  IdHTTP1.Request.ContentType := 'application/json';
  IdHTTP1.Request.Accept := 'application/vnd.hmrc.1.0+json';

  try
    result := IdHTTP1.Post('https://test-api.service.hmrc.gov.uk/oauth/token', params);
  except
    on E: Exception do begin
      Memo1.Lines.Add(E.ClassName + ': ' + E.message);
      raise;
    end;
  end;

  Memo1.Lines.Add(result);
finally
  params.Free;
end;

Answer 2

Here is an example which shows how the HTTP body can be accessed.

The body can be accessed if you catch the EIdHTTPProtocolException exception.

  on E: EIdHTTPProtocolException do
  begin
    WriteLn(E.Message);
    WriteLn(E.ErrorMessage);
  end;

Full example code:

program JSONPostExample;

{$APPTYPE CONSOLE}

uses
  IdHTTP, IdGlobal, SysUtils, Classes;

var
  HTTP: TIdHTTP;
  RequestBody: TStream;
  ResponseBody: string;
begin
  HTTP := TIdHTTP.Create;
  try
    try
      RequestBody := TStringStream.Create('{"日本語":42}',
        TEncoding.UTF8);
      try
        HTTP.Request.Accept := 'application/json';
        HTTP.Request.ContentType := 'application/json';
        ResponseBody := HTTP.Post('https://httpbin.org/post',
          RequestBody);
        WriteLn(ResponseBody);
        WriteLn(HTTP.ResponseText);
      finally
        RequestBody.Free;
      end;
    except
      on E: EIdHTTPProtocolException do
      begin
        WriteLn(E.Message);
        WriteLn(E.ErrorMessage);
      end;
      on E: Exception do
      begin
        WriteLn(E.Message);
      end;
    end;
  finally
    HTTP.Free;
  end;
  ReadLn;
  ReportMemoryLeaksOnShutdown := True;
end.

Note that you must not use a TStringList for the POST body. That version of TIdHTTP.Post() formats the data according to the application/x-www-form-urlencoded media type, which is not appropriate for JSON and will corrupt it.