shellcurl

POST 4GB file from shell using cURL


I try to post a file with a filesize of 4GB to a REST API.

Instead of uploading a file with this size, cURL POSTs a file with Content-Length: 0.

curl -v -i -d @"/work/large.png" -H "Transfer-Encoding: chunked" http://localhost:8080/files
* Adding handle: conn: 0x7fcafc00aa00
* Adding handle: send: 0
* Adding handle: recv: 0
* Curl_addHandleToPipeline: length: 1
* - Conn 0 (0x7fcafc00aa00) send_pipe: 1, recv_pipe: 0
* About to connect() to localhost port 8080 (#0)
*   Trying localhost...
* Connected to localhost (localhost) port 8080 (#0)
> POST /files HTTP/1.1
> User-Agent: curl/7.30.0
> Host: localhost:8080
> Accept: */*
> Transfer-Encoding: chunked
> Authorization: bearer XXX.XXX.XXX
> x-user-token: bearer XXX.XXX.XXX
* upload completely sent off: 5 out of 0 bytes
< HTTP/1.1 201 Created
HTTP/1.1 201 Created
< Date: Thu, 02 Jan 2014 14:55:46 GMT
Date: Thu, 02 Jan 2014 14:55:46 GMT
< ETag: "d41d8cd98f00b204e9800998ecf8427e"
ETag: "d41d8cd98f00b204e9800998ecf8427e"
< Location: http://localhost:8080/files/66032e34-9490-4556-8495-fb485ca12811
Location: http://localhost:8080/files/66032e34-9490-4556-8495-fb485ca12811
* Server nginx/1.4.1 is not blacklisted
< Server: nginx/1.4.1
Server: nginx/1.4.1
< Content-Length: 0
Content-Length: 0
< Connection: keep-alive
Connection: keep-alive

Using files with a smaller size will work as expected.

-rw-r--r--  1 user1  wheel  4403200000  2 Jan 15:02 /work/large.png

Why does the upload fail? And, how to correctly upload such a file?

Cheers.


Solution

  • To upload large binary files using CURL you'll need to use --data-binary flag.

    In my case it was:

     curl -X PUT --data-binary @big-file.iso https://example.com
    

    Note: this is really an extended version of @KarlC comment, which actually is the proper answer.