pythonnullmysql-connectorpython-polars

Polars read_database_uri() failing on `NULL as col` column in sql query

I am able to read data from SQL with nulls, unless a column is created like NULL as col. This causes a Rust panic:

thread '<unnamed>' panicked at 'not implemented: MYSQL_TYPE_NULL', /__w/connector-x/connector-x/connectorx/src/sources/mysql/typesystem.rs:119:18
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
pyo3_runtime.PanicException: not implemented: MYSQL_TYPE_NULL

This is the offending query:

SELECT
    col1 as col1_modified,
    col2 as col2_modified,
    col3 as col3_modified,
    col4,
    col5, col6, col7, col8,
    col9, col10,
    col11, col12, col13, col14, col15, col16,
    col17,
    null as bad_col,
    col19 as col19_modified
FROM my_schema.my_table 
WHERE cond = 1

bad_col exists, but sometimes its values are not relevant, so I set it to null to use as an indicator. The query runs when replace null as bad_col with bad_col. I can then just manually set it to all null.

Is there a way to fix this null as column_name pattern breaking polars' read_database_uri()?

Solution

  • Python doesn't know what data type to use (how to structure this value in memory). You can resolve this in the SQL by adding a CAST() around the null literal for whatever type would be most appropriate for the result. Something like this:

        ...
    col17,
    CAST(null as varchar) as bad_col,
    col19 as col19_modified
...

    Best results will come from matching the data type for the real bad_col column.