[SOLVED] Why might cURL return '(56) Failure when receiving data from the peer' from API when python requests and javascript fetch() work

Why might cURL return '(56) Failure when receiving data from the peer' from API when python requests and javascript fetch() work

I am migrating some control software written in python, running on a Raspberry Pi, to Home Assistant. I am having trouble getting a REST sensor to work. The sensor is supposed to post and receive info from a web based API.

The web API seems to work fine. I can communicate with it fine from python and from javascript running on the interface web pages. While trying to debug things I found that I couldn't use cURL to download the info from my API. cURL still seems to work fine when I use it for other purposes (using a laptop running ubuntu).

The minimal setup is an "API" running on https://eldwick.org.uk/immersion/test.php (obviously the real application does more than this)

<?php
header('Content-Type: application/json; charset=utf-8');
$result = [["a" => 2, "b" => 1], ["a" => 5, "b" => 3], ["a" => 8, "b" => 10], ["a" => 12, "b" => 4]];
echo json_encode($result, JSON_NUMERIC_CHECK);
?>

It seems to work fine with this running on my laptop

import requests

headers = { 'User-Agent' : 'Homeassistant', 'Content-Type': 'application/json' }
req = requests.post("https://eldwick.org.uk/immersion/test.php", headers=headers)
print(req.json())

or from a web page https://eldwick.org.uk/immersion/test.html with

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta http-equiv="Content-Type" content="text/html; charset=utf-8"/>
    </head>
    <body>
        <script>
            fetch("test.php",
                {method: 'POST',
                headers: {Accept: 'application/json', 'Content-Type': 'application/json', 'Access-Control-Allow-Origin': '*'}}
            )
            .then((response) => {
                return(response.json())
            })
            .then((responseJson) => {
                document.getElementById("test").innerHTML = JSON.stringify(responseJson);
            });
        </script>
        <div id="test"></div>
    </body>
</html>

Is the cURL behavior caused by something that might make the homeassistant sensor not work?

Is there something obvious I'm missing?

PS as @Xavier noted I don't say what cURL I did. I actually tried almost every permutation I could think of but the simplest was

$ curl -X POST http://eldwick.org.uk/immersion/test.php
curl: (52) Empty reply from server

The real API has a data body sent with passw, solar and temp value posted. For that I was trying something like

$ curl -d '{"passw":"******", "solar":666, "temp":99}' -H "Content-Type: application/json" http://eldwick.org.uk/immersion/get_schedule.php

sometimes with -v or --json

Solution

See @robertklep comment on OP

The problem was caused because I had set up the API on a shared hosting setup that had probably included a rough and ready filter for curl in the User-Agent (maybe set or fixable in the site .htaccess file) this can be easily circumvented by adding -H "User-Agent: Homeassistant" in the curl command line arguments