databricksspark-structured-streamingdatabricks-unity-catalog

Databricks structured streaming write to managed table

In Databricks structured streaming is it possible to write a stream using the writeStream API to a unity catalog managed table?

I'm able to write the stream to an external table in ADLS Gen2 fine, but when it try to write it to a managed table using .start(catalog.schema.table) the stream writes my checkpoints correctly but nothing is written to the managed table. I'm unsure if I'm doing something wrong, or if this is in fact not possible.

Solution

  • I found that instead of .start("catalog"."schema"."table") I needed to use .table("catalog"."schema"."table").