databricksazure-databricks

Does Table ACL & Row and Column Level Security With Unity Catalog only apply when accessing tables in Databricks Unity Calalog


I will be implementing Table ACL & Row and Column Level Security With Unity Catalog.

While it is possible to achieve Row and Column Level Security With Unity Catalog will the Row and Column Level Security pass through to Azure SQL DB? By that I mean if I were apply Row and Column Level Security in a Databricks Unity Catalog table and then export the table to Azure SQL DB will the Row and Column Level Security be automatically applied in Azure SQL DB.

So for example, if a user e.g. userA@contoso.com cannot see a particular column while accessing the data from a Unity Catalog table, will userA@contoso.com also prevented from seeing the column while access the data from Azure SQL DB after it has been moved to Azure SQL DB


Solution

  • The Row and Column Level Security and Table ACLs defined in Databricks Unity Catalog do not carry over when exporting data to Azure SQL Database, regardless of whether the export is done via JDBC, pipelines, or notebooks.

    The reason behind this is Unity Catalog’s security model is enforced only at query time within Databricks. The access rules are not stored as metadata within the data itself, so once the data is exported, it becomes plain data in Azure SQL DB, with no security context.

    To maintain similar security in Azure SQL Database, you need to define access controls again, using native Azure SQL DB features.

    Below I've shown an example how I manually added RLS in SQL database:

    Firstly, I created RLS predicate function to ensure users only see rows matching their region:

    enter image description here

    Then, created the Security Policy:

    enter image description here

    Lastly, Simulated access for a specific region:

    enter image description here

    This ensures users only see the rows for their assigned region.