I am wondering if it is possible to set permissions for external locations satisfying the following requirements:
1: A group of people should be able to access data in a storage container using a view as defined:
CREATE VIEW <catalog>.<schema>.testview AS
SELECT *
FROM delta.abfss://<container>@<storage>.dfs.core.windows.net/<path>
We have this working. People can access the storage location like this.
2: Prevent said people from accessing the storage container directly outside of these pre-defined views. They should not be able to enter this SELECT statement directly OR be able to create Views that use such a SELECT statement.
3: However still allow them to create Views/Tables based on the above mentioned as in:
CREATE VIEW new_view AS
SELECT * FROM <catalog>.<schema>.testview
As you mention you want to prevent users from accessing the external storage container directly while allowing access only through pre-defined views in azure databricks
You can control access to containers, directories, and blobs in Azure Data Lake Storage Gen2 using the Access Control Lists (ACLs) feature.
You can assign a security principal to a specific access level for files and directories, with each assignment recorded as an entry in an access control list (ACL). Every file and directory within your storage account has an ACL. When a security principal (such as a user, group, service principal, or managed identity) attempts to perform an operation on a file or directory, the ACL verifies whether they have the required permissions.
To manage the ACL on a container, follow these steps:
Navigate to the container within the storage account.
The Access Permissions tab on the Manage ACL page will appear. Use the options in this tab to control access to the object.
To add a security principal to the ACL, click the Add Principal button. Use the search box to find the security principal, then click Select.
Create a security group in Azure AD for each team and manage permissions at the group level instead of assigning them to individual users.
Reference: Access Control Lists (ACLs) in Azure Data Lake Storage Gen2.