azure-functionsazure-table-storage

Data conversion in Azure Table storage


I have a simple Azure function that takes previous data from the table storage, uses it for producing a new row which is then recorded back to the same table. These rows have many different fields, some of them are integer, some of them are floating-point. But as I can see all they are stored as int64 numbers into the table. Sure all the fractions are lost. And second, it creates problems when I get this data into my web app as it doesn't work with 64-bit integers. If briefly, what happens inside the Azure function

export async function eventHubTrigger1(message: Message, context: InvocationContext): Promise<void> {
    context.log('Event hub function processed message:', message);
    const prevData: ProcessedRecord[] = context.extraInputs.get(tableInput) as ProcessedRecord[];
    const currData: DeviceData[] = context.extraInputs.get(blobInput) as DeviceData[];
    const row: ProcessedRecord = StatusEvaluatingFunction(prevData, message, trapData, context);
    context.extraOutputs.set(tableOutput, [row]);
}

this is what this "row" looks like

type ProcessedRecord = {
  PartitionKey: string;
  RowKey: string;
  Leak: number; // integer
  CycleCounts: number; // integer
  Status: number; // integer
  TotLossesKg: number; // double
  ...
};

and what it becomes when gets into the tableenter image description here

Is there any way to state explicitly the types of the values (like here) in a record that is stored into a table by means of an Azure function (particularly written in JS/TS)?

Googling didn't helped. Maybe, the most useful suggesttion I managed to find was just to keep all the data as strings in the table and convert them to numbers when necessary. But I hope there is a better way of resolving this problem.


Solution

  • Data conversion in Azure Table storage

    TypeScript for a processed record that corresponds to Azure Table storage property types from this reference

    Edm.Int32:

    Edm.Double:

    Edm.String:

    type ProcessedRecord = {
        PartitionKey: string;
        RowKey: string;
        Timestamp: Date;
        BooleanProperty: boolean;
        DateTimeProperty: Date;
        DoubleProperty: number;
        GuidProperty: string;
        Int32Property: number;
        Int64Property: number;
        StringProperty: string;
    };
    

    enter image description here

    const { app } = require('@azure/functions');
    const { TableClient, AzureNamedKeyCredential } = require("@azure/data-tables");
    
    // Define the ProcessedRecord type
    const ProcessedRecord = {
        PartitionKey: 'string',
        RowKey: 'string',
        Leak: 'number',
        CycleCounts: 'number',
        Status: 'number',
        TotLossesKg: 'number',
        // Add other properties as needed
    };
    
    app.eventHub('eventHubTrigger1', {
        connection: 'sampathRootManageSharedAccessKey_EVENTHUB',
        eventHubName: 'sampath',
        cardinality: 'many',
        handler: async (messages, context) => {
            const account = Azure Storage Account Name";
            const accountKey = "Azure Storage Account AccountKey ";
            const tableName = "Azure Storage Account TableName ";
    
            const credential = new AzureNamedKeyCredential(account, accountKey);
            const client = new TableClient(`https://${account}.table.core.windows.net`, tableName, credential);
    
            if (Array.isArray(messages)) {
                context.log(`Event hub function processed ${messages.length} messages`);
                for (const message of messages) {
                    context.log('Event hub message:', message);
    
                    // Assuming your message contains data to be inserted into the table
                    const entity = {
                        PartitionKey: 'p2',
                        RowKey: 'r2',
                        Leak: 1,
                        CycleCounts: 2,
                        Status: 4,
                        TotLossesKg: 2.5,
                        // Add other properties as needed
                    };
    
                    try {
                        // Insert the entity into the table
                        await client.createEntity(entity);
                        context.log('Entity inserted into the table:', entity);
                    } catch (error) {
                        context.log('Error inserting entity into the table:', error);
                    }
                }
            } else {
                context.log('Event hub function processed message:', messages);
            }
        }
    });
    
    
    

    Output:

    enter image description here

    enter image description here

    Azure Storage Table:

    enter image description here