Is there a library which allows determining whether a given content type is binary or text-based?
Obviously text/* is always textual, but for things like application/json, image/svg+xml or even application/x-latex it's rather tricky without inspecting the actual data.
There's a wrapper for libmagic for python -- pymagic. Thats the easiest method to accomplish what you want. Keep in mind that magic is only as good as the fingerprint. You can have false-positives if something 'looks' like another file format, but most cases pymagic will give you what you need.
One thing to watch out for would be the 'simple solution' of checking to see if any of the characters are 'outside' the printable ASCII range, as you will likely encounter unicode which will look like binary (and in fact, be binary) even though it's just textual content.