How to validate HMAC digest in python

I need to validate the digest of data send with HMAC.

I have been stuck on this for a couple of days now, I cannot match the digest in python, here is my code:

import hmac
import hashlib
import json

# i will receive, a webhook that looks like this
payload = {
        "applicantId" : "64899a9e52e16960a43a2d8c",
        "inspectionId" : "64899a9e52e16960a43a2d8d",
        "applicantType" : "individual",
        "correlationId" : "9e732c5f814e27b766d20a85d23d169c",
        "levelName" : "basic-kyc-level",
        "sandboxMode" : true,
        "externalUserId" : "uOev8DYqbH",
        "type" : "applicantCreated",
        "reviewStatus" : "init",
        "createdAt" : "2024-04-17 06:57:33+0000",
        "createdAtMs" : "2024-04-17 06:57:33.165",
        "clientId" : "uab_first_digital_trade_europe"
        }

plain_text_payload = ','.join(f"{k}:{str(v).lower()}" if isinstance(v, bool) else f"{k}:{v}" for k, v in payload.items())

encoded_payload = plain_text_payload.encode('utf-8')

secret_key = 'TZUQlLdW-E5VM7nbcByTbyQx9G_'
encoded_secret = secret_key.encode('utf-8')
    
computed_signature = hmac.new(encoded_secret, encoded_payload, hashlib.sha256).hexdigest()

print("Computed HMAC signature:", computed_signature)

# This is the siguature Im expecting
signature = "84ea2fd80f8972554a2f7c1d7e97823abff09a32bd9af3865251f696fe0f31af"


if computed_signature == signature:
    print("Signatures match.")
else:
    print("Signatures do not match.")

Now, the above doesnt work, because "true" is lowercase. I tried making it True as bool, and then converting it as text back to lowercase but nothing, if i put quotes around it, still not matching.

However, i managed to the the signature expected with the following tools:

https://www.devglan.com/online-tools/hmac-sha256-online

For the following, update the secret adding "_" at the end of it, for some reason Stack Overflow sanitises it

https://emn178.github.io/online-tools/sha256.html?input_type=utf-8&input=%7B%0A%20%20%22applicantId%22%20%3A%20%2264899a9e52e16960a43a2d8c%22%2C%0A%20%20%22inspectionId%22%20%3A%20%2264899a9e52e16960a43a2d8d%22%2C%0A%20%20%22applicantType%22%20%3A%20%22individual%22%2C%0A%20%20%22correlationId%22%20%3A%20%229e732c5f814e27b766d20a85d23d169c%22%2C%0A%20%20%22levelName%22%20%3A%20%22basic-kyc-level%22%2C%0A%20%20%22sandboxMode%22%20%3A%20true%2C%0A%20%20%22externalUserId%22%20%3A%20%22uOev8DYqbH%22%2C%0A%20%20%22type%22%20%3A%20%22applicantCreated%22%2C%0A%20%20%22reviewStatus%22%20%3A%20%22init%22%2C%0A%20%20%22createdAt%22%20%3A%20%222024-04-17%2006%3A57%3A33%2B0000%22%2C%0A%20%20%22createdAtMs%22%20%3A%20%222024-04-17%2006%3A57%3A33.165%22%2C%0A%20%20%22clientId%22%20%3A%20%22uab_first_digital_trade_europe%22%0A%7D&hmac_enabled=1&hmac_input_type=utf-8&hmac_key=TZUQlLdW-E5VM7nbcByTbyQx9G_

Can anyone help me understand how to get this to work?

Thank you

EDIT

I have tried doing this on the json payload directly with:

def validate_webhook():
    # Map the header to the appropriate hash function name
    algo_map = {
        'HMAC_SHA1_HEX': hashlib.sha1,
        'HMAC_SHA256_HEX': hashlib.sha256,
        'HMAC_SHA512_HEX': hashlib.sha512,
    }

    # Get the algorithm from the request header
    algo_name = request.headers.get('X-Payload-Digest-Alg')
    if algo_name not in algo_map:
        log_message(f"Unsupported algorithm: {algo_name}", "ALERT", run_id=g.run_id)
        raise RuntimeError('Unsupported algorithm')

    # Get the hash function based on the algorithm
    hash_func = algo_map[algo_name]

    secret_key = 'TZUQlLdW-E5VM7nbcByTbyQx9G_'

    # Calculate the HMAC for the content
    content = request.get_data()  # Gets the raw data as bytes
    computed_signature = hmac.new(secret_key.encode(), content, hash_func).hexdigest()

    # Get the signature sent in the request headers
    signature = request.headers.get('X-Signature')

    # Check if the computed signature matches the one in the header
    if computed_signature != signature:
        log_message(f"Webhook sumsub sign {content}", "ALERT", run_id=g.run_id)
        raise Exception('Webhook sumsub sign validation failed')

    # Return some response or handle the valid request further
    log_message(content.decode('utf-8'), "INFO", run_id=g.run_id, data=content.decode('utf-8') )
    return True

And I have also tried to do it converting the payload as text with:

def validate_webhook():
    # Map the header to the appropriate hash function name
    algo_map = {
        'HMAC_SHA1_HEX': hashlib.sha1,
        'HMAC_SHA256_HEX': hashlib.sha256,
        'HMAC_SHA512_HEX': hashlib.sha512,
    }

    # Get the algorithm from the request header
    algo_name = request.headers.get('X-Payload-Digest-Alg')
    if algo_name not in algo_map:
        log_message(f"Unsupported algorithm: {algo_name}", "ALERT", run_id=g.run_id)
        raise RuntimeError('Unsupported algorithm')

    # Get the hash function based on the algorithm
    hash_func = algo_map[algo_name]

    secret_key = 'TZUQlLdW-E5VM7nbcByTbyQx9G_'

    # Calculate the HMAC for the content
    content = request.get_data(as_text=True)  # Gets the data as a text string
    # Explicitly encode the content to UTF-8 before computing HMAC
    encoded_content = content.encode('utf-8')
    computed_signature = hmac.new(secret_key.encode(), encoded_content, hash_func).hexdigest()

    # Get the signature sent in the request headers
    signature = request.headers.get('X-Signature')

    # Check if the computed signature matches the one in the header
    if computed_signature != signature:

        log_message(f"Webhook sumsub sign {content}", "ALERT", run_id=g.run_id)
        raise Exception('Webhook sumsub sign validation failed')

    # exit if the signature is valid
    log_message(content, "INFO", run_id=g.run_id, data=content )
    return True

EDIT 2

This is the payload i receive:

b'{\n  "applicantId" : "6616c4f80cc65e2a1fe1419d",\n  "inspectionId" : "6616c4f80cc65e2a1fe1419e",\n  "applicantType" : "individual",\n  "correlationId" : "7a3450416467276914de43eb75b5916a",\n  "sandboxMode" : true,\n  "externalUserId" : "level-2ce713de-679a-453e-9719-2464082633c5",\n  "type" : "applicantCreated",\n  "reviewStatus" : "init",\n  "createdAt" : "2024-04-17 08:28:07+0000",\n  "createdAtMs" : "2024-04-17 08:28:07.745",\n  "clientId" : "uab_first_digital_trade_europe"\n}'

Solution

The code in your second snippet looks fine. Just make sure you are comparing the correct payload and signature -- the payload in your "Edit 2" is different from the payload in the link to the online tool, so the signatures will be different. When I run your second code snippet on the same payload as the online tool, it produces the same signature.

If you need to continue debugging, I suggest that you (temporarily) log the relevant values:

print("content = {!r}".format(content))
print("expected signature = {!r}".format(signature))
print("computed_signature = {!r}".format(computed_signature))

That will give you a consistent set of values that you can later run tests on.

Some additional notes:

As pointed out in the comments, you need to be very careful around encoding. The best approach is to just read the raw bytes (as you do in your second snippet). Decoding as UTF-8 then re-encoding as you do in your third snippet is asking for trouble. And manually encoding as JSON as you do in your first snippet or using json.dumps() is completely out of the question, because even the smallest difference (e.g. a space) will cause the signature to be different. (The only exception is if you are generating the data yourself, in which case you must encode it. In all other cases, you should treat the payload as an opaque binary blob until you've verified the HMAC: only decode/parse after you've verified the HMAC.)
Ideally, you should compare the signatures using
if not hmac.compare_digest(computed_signature, signature): raise ...
instead of
if computed_signature != signature: raise ...
They return the same result, but compare_digest() is preferable because it protects from timing attacks.