azureiotazure-stream-analyticsstream-analyticsbigdata

Syntax issue in Stream Analytics Query running in Azure: Invalid column name: 'payload'


I am having a syntax issue with my stream analytics query. Following is my Stream Analytics query, where i am trying to get following fields from the events:

NON-WORKING QUERY

SELECT input.vehicleId,
FUEL_DIFF = LAG(input.Payload.FuelLevel) OVER (PARTITION BY vehicleId LIMIT DURATION(minute, 1)) - input.Payload.FuelLevel,
ODO_DIFF = input.Payload.OdometerValue - LAG(input.Payload.OdometerValue) OVER (PARTITION BY input.vehicleId LIMIT DURATION(minute, 1)) 
from input

Following is one sample input event on which the above query/job is ran on the series of events:

   {
      "IoTDeviceId":"DeviceId_1",
      "MessageId":"03494607-3aaa-4a82-8e2e-149f1261ebbb",
      "Payload":{
         "TimeStamp":"2017-01-23T11:16:02.2019077-08:00",
         "FuelLevel":19.9,
         "OdometerValue":10002
      },
      "Priority":1,
      "Time":"2017-01-23T11:16:02.2019077-08:00",
      "VehicleId":"MyCar_1"
   }

Following syntax error is thrown when the Stream Analytics job is ran:

Invalid column name: 'payload'. Column with such name does not exist.

Ironically, the following query works just fine:

WORKING QUERY

SELECT input.vehicleId,
FUEL_DIFF = LAG(input.Payload.FuelLevel) OVER (PARTITION BY vehicleId LIMIT DURATION(second, 1)) - input.Payload.FuelLevel
from input

The only diffrence between WORKING QUERY and NON-WORKING QUERY is number of LAG constructs used. The NON-WORKING QUERY has two LAG constructs, while WORKING QUERY has just one LAG construct.

I have referred Stream Analytics Query Language, they only have basic examples. Also tried looking into multiple blogs. In addition, I have tried using GetRecordPropertyValue() function, but no luck. Kindly suggest.

Thank you in advance!


Solution

  • This looks like a syntax bug indeed. Thank you for reporting - we will fix it in the upcoming updates.

    Please consider using this query as a workaround:

    WITH Step1 AS
    (
    SELECT vehicleId, Payload.FuelLevel, Payload.OdometerValue
    FROM input
    )
    
    SELECT vehicleId, 
    FUEL_DIFF = LAG(FuelLevel) OVER (PARTITION BY vehicleId LIMIT DURATION(minute, 1)) - FuelLevel,
    ODO_DIFF = OdometerValue - LAG(OdometerValue) OVER (PARTITION BY vehicleId LIMIT DURATION(minute, 1)) 
    from Step1