I'm new to python, and am a huge fan of static type checkers. I have some code that handles file uploads with the Bottle framework. See below.
def transcribe_upload(upload: FileUpload) -> Alternative:
audio:AudioSource = upload_source(upload)
...
def upload_source(upload:FileUpload) -> AudioSource:
...
I made a really simple mistake and passed upload.file
(the file-like object) to upload_source
instead of the entire FileUpload
object.
def transcribe_upload(upload: FileUpload) -> Alternative:
audio:AudioSource = upload_source(upload.file) # This is incorrect!
The typechecker didn't catch it. In fact, it doesn't catch ANY incorrect parameter passing to upload_source
:
def transcribe_upload(upload: FileUpload) -> Alternative:
audio:AudioSource = upload_source(4) # Why isn't mypy giving me an error?
audio:AudioSource = upload_source(upload.asdf) # Why isn't mypy giving me an error?
What's going on? I tested some basic functions separately and the typechecker caught when I tried to pass a number to a function that wanted a string, and it worked. What am I missing here?
EDIT
@kojiro suggsted that FileUpload
is equivalent to Any
. I think that may be correct. Here is the source for FileUpload
. It's imported like this: from bottle import FileUpload
.
If that's the case, why is it letting me use FileUpload
as if it were a type? (If I mess up the name, like FileUpld
it does give me an error).
More importantly, how do I get real types? I suppose the bottle authors have to add them?
bottle
is not a statically-typed library (it offers no type annotations in bottle.py
). Its organisation as a single bottle.py
(instead of a package) also prevents you from making a py.typed
yourself, which would otherwise allow mypy to pick up at least the class attributes, function/method signatures, and global variables (even if they don't have any typing information).
The only way to get "real types" is for the bottle
maintainers to add them. Assuming this isn't going to happen, you have several options:
Generate a skeletal outline (Python stub file, extension .pyi
) which will aid in basic code completion. This will catch variable, function, and class existence, but not variable and function signature typing information (simply because bottle.py
doesn't have any in the first place).
The skeletal outline can be generated in about a second using mypy's shipped stubgen
tool, which you should already have if you've installed mypy.
Collect runtime types by running pyannotate or MonkeyType on bottle's test suite, and use these tools to generate a .pyi
. This is more accurate than just running stubgen
, but the final quality is at the mercy of the test suite, and is very likely to result in a lot of false positive warnings by mypy, especially with descriptor classes like bottle's DictProperty
and lazy_attribute
.
Use pytype's type inference to generate a .pyi
. This generally results in more usable stubs, at the cost of an extremely long inference procedure that may end up building a dependency graph and scanning most of your site-packages
, and will fail quite often if bottle
has a lot of complex third-party dependencies. (EDIT: It looks like bottle
only depends on the Python standard library, in which case you should get a fairly fast inference by pytype)
You should end up getting a single bottle.pyi
that you must move to the same directory as bottle.py
. This should be in the site-packages
of your virtual environment. Once you've done 2. or 3., mypy
should be able to pick up typing errors properly.