azureazure-application-insightsazure-api-management

How to log Azure APIM user in Application Insights

Goal

Determine who is accessing which API endpoints within Azure APIM from Application Insights (AI).

Assumptions

To exaggerate, consider having 10k users and they share the same subscription key (perhaps that is a wrong approach, but please continue reading).

Setup

APIM enabled sending logs to AI and a Log Analytics workspace Application Insights

Attempt #1

I hit the API endpoint from Postman on my machine using a subscription key. I then did the same thing but from a VM. This resulted in two log entries in AI with 2 different IPs and the same subscription.

ApiManagementGatewayLogs
| project TimeGenerated, CallerIpAddress, ApiId, OperationId, ApimSubscriptionId

AI Logs

Attempt #2

Within the APIM instance, I went to Monitoring --> Analytics and clicked on the Users tab. This did not show any helpful information about the "User".

Monitoring Analytics

Discussion

It appears there is an unlimited number of subscription keys that can be made for a Standardv2 APIM instance. In that case, perhaps the correct approach is to solve this by creating 10k keys (treating them like API keys), but I haven't found guidance on that specifically.

I wouldn't expect APIM to know "who" is accessing it without it having being given the information. This is why I assume I'd need to employ JWT validation where the user would obtain a token and call the gateway. Given a JWT, the user name or system could be extracted from the JWT claims. Once I have the user claim, would I be able to enrich the AI logs with that information?

Questions

I am new to using Azure APIM and am looking forward to adopting it on our team. Thanks for any helpful advice in advance!

Solution
    • Should subscriptions be treated like API keys?

    Technically you can treat subscription keys as API Keys, but this will not be applicable in all the scenarios.

    By default, when you create an API, a subscription key is required for API access.

    You can use subscription keys as API keys in few scenarios where you can have a less data and it is detailed more clearly in the MSDoc here.

    • Should subscription keys be shared by multiple users or backends?

    Technically the sharing a subscription key with a group of users or backends can be done, but it is not well recommended if you are trying to retrieve a granular user level data identity.

    For example, if you have a single subscription key which is used by many users or backend applications, it becomes difficult to identify who accessed what and which application.

    So, assigning a unique subscription key to each individual user or backend is more flexible for tracking the required data.

    If still you want to assign multiple users or backend applications a same subscription key, make sure you are implementing a unique identifier within a JWT token that each user’s request has or as a query parameter.

    • How do I enrich Application Insight logs to include additional information?

    To enrich Application Insights for adding the additional details, you can first configure API management service to validate JWTs, and extract user claims. Now, pass this information to Application Insights further.

    Note: You can use logging or trace policies in the <outbound> policy to send and retrieve the data.

    To validate JWT, Go to APIM and add an inbound validate-JWT policy by including authentication header, claims and also an open Id URL as shown below.

    <validate-jwt header-name="Authorization" failed-validation-httpcode="401" failed-validation-error-message="Not accessible">
    <openid-config url="https://login.microsoftonline.com/tenantID/v2.0/.well-known/openid-configuration" />
    <audiences>
        <audience>api://d172xxxxadcab</audience>
    </audiences>
    <required-claims>
        <claim name="aud" match="all">
            <value>d17229xxx71eadcab</value>
        </claim>
    </required-claims>
</validate-jwt>

    And also, you need to use set-variable to extract the required user claims such as userid and add it in the logs for retrieval.

    Once sending logs is done, you can query the data from Application insights accordingly with the specific query related language. (Eg: KQL)