we are trying to create an online feature store using cosmosdb following this documentation: https://learn.microsoft.com/en-us/azure/databricks/machine-learning/feature-store/publish-features .
But I get an error when I publish the table to cosmosdb: AnalysisException: Catalog 'cosmoscatalog' not found. The issue only happens when using unity-enabled workspaces. I can publish using a non-unity enabled workspace.
P.S. If I create the table using the non-unity enabled workspace, then the unity-enabled workspace can update the cosmosdb. But the unity-enabled worskpace cannot create the cosmos container/database using the fs.publish_table.
I tried the following code:
from databricks.feature_store.online_store_spec import AzureCosmosDBSpec
from databricks.feature_store.client import FeatureStoreClient
fs = FeatureStoreClient()
account_uri = "https://online-feature-store.documents.azure.com:443/"
# Specify the online store.
online_store_spec = AzureCosmosDBSpec(
account_uri=account_uri,
write_secret_prefix="secret/write-cosmos",
read_secret_prefix="secret/read-cosmos",
database_name="online_feature_store_example",
container_name="feature_store_online_wine_features"
)
# Push the feature table to online store.
fs.publish_table("online_feature_store_example.wine_static_features", online_store_spec, mode='merge')
The following code works on workspaces without unity catalog enabled. However, on a unity-catalog enabled workspace, it trhows an error: AnalysisException: Catalog 'cosmoscatalog' not found
You need to create the database and container in CosmosDB with the name you are specifying in AzureCosmosDBSpec.