I have in input a list of jsons file (let say in.jsons):
{"a": "foo", "b": "bar"}
{"a": "bar", "d": "baz"}
...
I would like to generate a file with the same format, applying some transformation (my-awesome-fn). I would like to use babashka. My first feeling was a simple bash pipeline:
cat in.jsons | bb -i --stream script.clj > out.jsons
where script.clj looks like:
(require '[cheshire.core :as json])
(let [in (json/parse-string *input* true)
out (my-awesome-fn in)]
(json/generate-string out))
What I expected to see in my out.jsons file
{"aa": "faa", "bb": "bor"}
{"aa": "bii", "dd": "bla"}
...
What I get in my out.jsons file
"{\"aa\": \"faa\", \"bb\": \"bor\"}"
"{\"aa\": \"bii\", \"dd\": \"bla\"}"
...
I can understand: I convert my data to a string converted again when come to standard output.
If I use generate-stream with proper io writer instead of generate-string, I get something like:
{"aa": "faa", "bb": "bor"}#object[java.io.BufferedWriter 0x13e21dd4 "java.io.BufferedWriter@13e21dd4"
{"aa": "bii", "dd": "bla"}#object[java.io.BufferedWriter 0x13e21df5 "java.io.BufferedWriter@13e21df5"
...
I can also understand this result. I've never converted my stream object to string when coming back to bash.
But I don't how I can solve my issue. Can somebody help me please?
If I understand correctly, without -o Babashka prints the result of your function, which is a string. If, however, you add -o, it prints the result interpreted as "lines of text", which is what you want here.
Hence, to fix your issue you just have to add -o to the options you pass to bb:
cat in.jsons | bb -i -o --stream script.clj > out.jsons