azure-data-explorerkusto-explorer

Can I use materialized views in Kusto emulator?


I was trying to use Kusto emulator for my local development, but cannot create materialized view, although from the docs it should be supported. Am I doing something wrong?

I am using latest docker image for Linux and cluster created at container (not backed with mounted disc from host).

Here is the error:

{
    "error": {
        "code": "Internal service error",
        "message": "Request aborted due to an internal service error.",
        "@type": "Kusto.Common.Svc.Exceptions.DistributedIngestionException",
        "@message": "An admin command cannot be executed due to a failure in the distributed ingestion: Details='[UtilsException] [ShardEngine] kusto_shard_locator_create failed (-2147467259: FATAL_FAILURE): Io(Custom { kind: Other, error: \"Failed to find VFS provider for 'c:\\\\kustodata\\\\dbs\\\\AutoTests32\\\\data'\" })'",
        "@context": {
            "timestamp": "2023-12-01T21:46:45.2741660Z",
            "serviceAlias": "",
            "machineName": "076cfa2eedb4",
            "processName": "Kusto.Personal",
            "processId": 8,
            "threadId": 1787,
            "clientRequestId": "KE.RunCommand;84f439e0-4424-4041-a039-4068cb803ced",
            "activityId": "8fbfb92b-1ade-4c81-af52-4b84556d3d1b",
            "subActivityId": "a7fe1828-4762-4e57-915d-19b06d1b36ff",
            "activityType": "DN.AdminCommand.TableAppendCommand",
            "parentActivityId": "2125b8ac-8861-497d-ade4-16329e3902ab",
            "activityStack": "(Activity stack: CRID=KE.RunCommand;84f439e0-4424-4041-a039-4068cb803ced ARID=8fbfb92b-1ade-4c81-af52-4b84556d3d1b > ExecuteMaterializedViewUnmonitoredQuery/8fbfb92b-1ade-4c81-af52-4b84556d3d1b > DN.AdminCommand.TableSetOrAppendCommand/2125b8ac-8861-497d-ade4-16329e3902ab > DN.AdminCommand.TableAppendCommand/a7fe1828-4762-4e57-915d-19b06d1b36ff)"
        },
        "@permanent": false
    }
}

Solution

  • I've tried the following script without issue using the Linux Emulator. Could you provide a script that reproduces your issue?

    // Create a persistant database
    .create database MyDb persist (
      @"/kustodata/dbs/my_db/md",
      @"/kustodata/dbs/my_db/data"
      )
    
    // Then select the database in Kusto Explorer
    
    // Create table
    .create table Log(Id:string, Timestamp:datetime)
    
    // Create MV on top of table
    .create materialized-view LatestLog on table Log
    {
        Log
        | summarize arg_max(Timestamp, *) by Id
    }
    
    //  Add some data, I ran this line twice to have the same id and different timestamp
    .append Log <|
    print Id="Id42", Timestamp=now()
    
    // Another id...
    .append Log <|
    print Id="Id50", Timestamp=now()
    
    // Then query the materialized view
    // It gave me the expected result
    LatestLog