javahttpclojuredownloadslurp

Slurping http://foobar.mp3 that redirects to http://fizzbar.mp3 in Clojure


I am trying to programatically download mp3 files from this rss feed. When I open an url such as: http://menlochurch.podbean.com/mf/feed/5gv2gb/170219_jortberg.mp3 it redirects to an url like: http://s62.podbean.com/pb/67f34563539acbe87b9566ecc5738d57/58aeff8e/data4/fs145/948579/uploads/170219_jortberg.mp3

If I curl the first url it downloads an empty file.

If I curl -L the first url it correctly downloads the file.

If I curl the second url it correctly downloads the file.

If I slurp the first url in Clojure it downloads something that looks like an mp3 (is big and has no legible text) but is unplayable and not the same as the output from curl -L.

If I slurp the second url in Clojure it also downloads something that looks like an mp3 (is big and has no legible text) but is unplayable and not the same as the output from curl -L and is identical to the output from the previous slurp.

āžœ ~ git:(master) āœ— ls -l *.mp3 -rw-r--r-- 1 adam adam 38038533 Feb 23 08:32 curl-url1.mp3 -rw-r--r-- 1 adam adam 38038533 Feb 23 08:37 curl-url2.mp3 -rw-r--r-- 1 adam adam 0 Feb 23 08:45 curl-without-L.mp3 -rw-r--r-- 1 adam adam 67144297 Feb 23 08:31 slurp-url1.mp3 -rw-r--r-- 1 adam adam 67144297 Feb 23 08:31 slurp-url2.mp3

As you can see whatever I'm slurping, is almost twice as big as what I'm curling.

I don't know what to make of this, am I using slurp in an inappropriate context? Any advice would be appreciated!


Solution

  • slurp is for strings so won't work with binary data.

    user=> (doc slurp)
    -------------------------
    clojure.core/slurp
    ([f & opts])
      Opens a reader on f and reads all its contents, returning a string.
      See clojure.java.io/reader for a complete list of supported arguments.