delphihttp-status-code-403indy10idhttp

Error 403 using TIdHTTP for a GET request in Delphi RIO - Same url works in Postman

I am having a problem.

I try various possible solutions, but something is still wrong in my code, and I can't figure out what is it.

Function:

function HTTPRestApiGet(var HTTP: TIdHTTP; necesitaSSL: Boolean; var SSL: TIdSSLIOHandlerSocketOpenSSL; Usuario, Clave, URLBase, Recurso, CamposSolicitados: string; TiempoEspera: Integer = 5000;
  LogActivado: Boolean = True): THTTPRestApiResponse;
var
  requerimiento, respuesta: string;
  LogFile                 : TIdLogFile;
begin
  LogFile := TIdLogFile.Create(nil);
  //
  HTTP.HandleRedirects := True;
  HTTP.RedirectMaximum := 10;
  HTTP.ReadTimeout := TiempoEspera;
  HTTP.MaxAuthRetries := 0;
  HTTP.HTTPOptions := [hoInProcessAuth];
  HTTP.Request.BasicAuthentication := True;
  HTTP.Request.Username := Usuario;
  HTTP.Request.Password := Clave;
  HTTP.Request.Accept := 'http';
  HTTP.Request.ContentType := 'application/json';
  //
  HTTP.IOHandler := nil;
  if necesitaSSL then
    begin
      HTTP.IOHandler := SSL;
      SSL.SSLOptions.Mode := sslmClient;
      SSL.SSLOptions.Method := sslvSSLv23;
    end;
  // Configurar logging
  LogFile.Filename := 'http_log.txt';
  LogFile.Active := True;
  HTTP.Intercept := LogFile;
  //
  try
    requerimiento := URLBase + IfThen(Recurso <> EmptyStr, '/' + Recurso + IfThen(CamposSolicitados <> EmptyStr, '?_fields=' + CamposSolicitados, ''), EmptyStr);
    try
      respuesta := HTTP.Get(requerimiento);
    except
    end;
  finally
    HTTPRestApiLogRespuesta('GET', requerimiento, respuesta, HTTP.URL.uri, HTTP.ResponseText, HTTP.ResponseCode, LogActivado);
    with Result do
      begin
        requerimiento_exitoso := (HTTP.ResponseCode >= 200) and (HTTP.ResponseCode < 300);
        respuesta_codigo := HTTP.ResponseCode;
        respuesta_codigotexto := HTTP.ResponseText;
        respuesta_texto := respuesta;
        respuesta_id := -1;
      end;
    //
    LogFile.Free;
  end;
end;

Call:

HTTP := TIdHTTP.Create(nil);
SSL := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
respuesta := HTTPRestApiGet(HTTP, True, SSL, 'brgroup', '1234', 'https://apis.datos.gob.ar/series/api/series/?ids=148.3_INIVELNAL_DICI_M_26&start_date=' + fecha_desde + '&end_date=' + fecha_hasta,
  EmptyStr, EmptyStr, 5000, False);
SSL.Free;
HTTP.Free;

Response (log file):

HTTP/1.1 403 Forbidden

Stat Connected.
Sent 03/08/2024 14:31:22: GET /series/api/series/?ids=148.3_INIVELNAL_DICI_M_26&start_date=2024-04-01&end_date=2024-07-01 HTTP/1.1<EOL>Content-Type: application/json<EOL>Host: apis.datos.gob.ar<EOL>Accept: http<EOL>User-Agent: Mozilla/3.0 (compatible; Indy Library)<EOL>Authorization: Basic Og==<EOL><EOL>
Recv 03/08/2024 14:31:22: HTTP/1.1 403 Forbidden<EOL>Date: Sat, 03 Aug 2024 17:31:18 GMT<EOL>Content-Type: text/plain; charset=UTF-8<EOL>Content-Length: 16<EOL>Connection: keep-alive<EOL>X-Frame-Options: SAMEORIGIN<EOL>Referrer-Policy: same-origin<EOL>Cache-Control: private, max-age=0, no-store, no-cache, must-revalidate, post-check=0, pre-check=0<EOL>Expires: Thu, 01 Jan 1970 00:00:01 GMT<EOL>Report-To: {"endpoints":[{"url":"https:\/\/a.nel.cloudflare.com\/report\/v4?s=R4%2FV8fU73LVHwpLIjJ6e%2Bq91jVM76aHSX9KolkAQ2jd6gUdAGWjUl5yOZd728jdQb%2BoIx%2BzuQyFnqhINrrA76CWjQv%2Fw17ciaSSFRAmKOMSEBszwUe3s6K0en7dhPVXEC3wH5A%3D%3D"}],"group":"cf-nel","max_age":604800}<EOL>NEL: {"success_fraction":0,"report_to":"cf-nel","max_age":604800}<EOL>Server: cloudflare<EOL>CF-RAY: 8ad80a629ecaa78f-EZE<EOL>alt-svc: h3=":443"; ma=86400<EOL><EOL>error code: 1010

With Postman, same request, works fine:

curl --location 'https://apis.datos.gob.ar/series/api/series/?ids=148.3_INIVELNAL_DICI_M_26&start_date=2024-04-01&end_date=2024-07-01' \
--header 'Accept: http' \
--header 'Content-Type: application/json' \
--header 'Authorization: Basic Og=='

Can anyone help me with this?

I am trying various TIdHTTP options but it doesn't work. The function is working fine for other purposes.

Solution

  • You did not show the raw request data that curl is sending, but the only difference of consequence that I see between your TIdHTTP code and your curl command would be the User-Agent request header.

    Many servers are sensitive to the requesting agent, where they send different data to different agents. It is not uncommon for such servers to reject/not recognize Indy's default User-Agent value.

    When I tested your code, it indeed failed with Indy's default User-Agent value of Mozilla/3.0 (compatible; Indy Library), and when I instead set the TIdHTTP.Request.UserAgent property (or the global GIdDefaultUserAgent variable) to a value that mimicked a real web browser (in my case, Firefox), the request worked as expected and a JSON response was returned:

    HTTP.Request.UserAgent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:128.0) Gecko/20100101 Firefox/128.0';

    Given curl's popularity, it would not surprise me if agent-aware servers do recognize curl. FYI, curl's default User-Agent value is curl/[version] (eg: curl/8.9.1, which also worked in my TIdHTTP test).

    When you are comparing curl requests to other HTTP libraries, you should always strive to make the requests be as close to identical as possible to eliminate any possible differences that can confuse servers. And when there are behavior differences between similar requests, it is typically the User-Agent at fault.

    On a side note: