google-chromecurlrequestuser-agent

CURL does not work with URLs with curly braces in parameters

Some URLs with brackets don't work with CURL but work on Chrome and Firefox.

For example this URL: https://rdtrkr.com/mg.php?voluum_id=d51b17bc-c537-4f3e-9879-2e373341ae5a&widget_id={widget_id}&campaign_id={campaign_id}&teaser_id={teaser_id}&geo={geo}&img=guy18.jpg&txt=german&lp=de&click_price={click_price}&click_id={click_id}&{click_id} does work in Chrome and firefox but when called with CURL, gives a 404 error.

curl  \
-H "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.80 Safari/537.36" \
-v "https://rdtrkr.com/mg.php?voluum_id=d51b17bc-c537-4f3e-9879-2e373341ae5a&widget_id={widget_id}&campaign_id={campaign_id}&teaser_id={teaser_id}&geo={geo}&img=guy18.jpg&txt=german&lp=de&click_price={click_price}&click_id={click_id}&{click_id}"

Produces the result:

< HTTP/2 404 
< server: nginx
< date: Thu, 13 Dec 2018 16:53:45 GMT
< content-type: text/html; charset=UTF-8
< content-length: 0

But with chrome developper tools in "Preserve log" mode I have :

Chrome dev tool results

CURL receives 404 instead of a 302 redirect. Is it related to the fact that CURL might be URL encoding brackets? I don't know what is going wrong here.

ps: I am not the owner of the website I'm using in the example.

Solution

  • Curly brackets are unsafe in URLs. cURL (unlike Google Chrome) tries to do you a favor and automatically encodes the URL.

    In other words, it transforms { to %7B and } to %7D.

    To prevent that behavior, you can pass the query string parameters using -d instead. Since -d changes the request to a POST, you'll also need to use -G to force the request to be a GET.

    So instead of doing

    curl "http://example.com?param1=xxx&param2=yyy"

    you could do

    curl "http://example.com" -G -d "param1=xxx&param2=yyy"

    In your particular case, for some reason the webserver you're targeting will still return 404 unless you supply an Accept-Language header:

    curl -v "http://rdtrkr.com/mg.php" \
     -G -d "voluum_id=d51b17bc-c537-4f3e-9879-2e373341ae5a&widget_id={widget_id}&campaign_id={campaign_id}&teaser_id={teaser_id}&geo={geo}&img=guy18.jpg&txt=german&lp=de&click_price={click_price}&click_id={click_id}&{click_id}" \
     -H "Accept-Language: en-US,en;q=0.9,fr;q=0.8,ru;q=0.7,es;q=0.6"

    gives

    *   Trying 34.192.193.118...
* Connected to rdtrkr.com (34.192.193.118) port 80 (#0)
> GET /mg.php?voluum_id=d51b17bc-c537-4f3e-9879-2e373341ae5a&widget_id={widget_id}&campaign_id={campaign_id}&teaser_id={teaser_id}&geo={geo}&img=guy18.jpg&txt=german&lp=de&click_price={click_price}&click_id={click_id}&{click_id} HTTP/1.1
> Host: rdtrkr.com
> User-Agent: curl/7.47.0
> Accept: */*
> Accept-Language: en-US,en;q=0.9,fr;q=0.8,ru;q=0.7,es;q=0.6
>
< HTTP/1.1 302 Found
< Server: nginx
< Date: Thu, 13 Dec 2018 17:39:18 GMT
< Content-Type: text/html; charset=UTF-8
< Content-Length: 0
< Connection: keep-alive
< Location: https://rotronica-premarity.com/d51b17bc-c537-4f3e-9879-2e373341ae5a?widget_id={widget_id}&campaign_id={campaign_id}&teaser_id={teaser_id}&geo={geo}&img=guy18.jpg&txt=german&lp=de&click_price={click_price}&click_id={click_id}
<
* Connection #0 to host rdtrkr.com left intact