Python 3 XChaCha20 test vectors work for encryption but decryption stage fails

I use the following vectors to test XChaCha20 encryption with AEAD by Poly1305 in python:

Vectors:

https://datatracker.ietf.org/doc/html/draft-arciszewski-xchacha-03#appendix-A.3

pycryptodome:

https://pycryptodome.readthedocs.io/en/latest/src/cipher/chacha20_poly1305.html

The drafts use HEX for the test vectors, if you really need to check me convert using this service:

https://www.asciitohex.com/

import json
from base64 import b64encode
from base64 import b64decode
from Crypto.Cipher import ChaCha20_Poly1305
from Crypto.Random import get_random_bytes

#nonce_xchacha20 = get_random_bytes(24)
nonce_xchacha20 = b64decode("QEFCQ0RFRkdISUpLTE1OT1BRUlNUVVZX")

#header = b"header"
header = b64decode("UFFSU8DBwsPExcbH")
plaintext = b"Ladies and Gentlemen of the class of '99: If I could offer you only one tip for the future, sunscreen would be it."
#key = get_random_bytes(32)
key = b64decode("gIGCg4SFhoeIiYqLjI2Oj5CRkpOUlZaXmJmam5ydnp8=")
cipher = ChaCha20_Poly1305.new(key=key, nonce=nonce_xchacha20)
cipher.update(header)
ciphertext, tag = cipher.encrypt_and_digest(plaintext)

jk = [ 'nonce', 'header', 'ciphertext', 'tag' ]
jv = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ]
result = json.dumps(dict(zip(jk, jv)))
print(result)

# We assume that the key was securely shared beforehand
try:
    b64 = json.loads(result)
    jk = [ 'nonce', 'header', 'ciphertext', 'tag' ]
    jv = {k:b64decode(b64[k]) for k in jk}

    cipher = ChaCha20_Poly1305.new(key=key, nonce=jv['nonce'])
    cipher.update(jv['header'])
    plaintext = cipher.decrypt_and_verify(jv['ciphertext'], jv['tag'])
    print("The message was: " + plaintext)
except (ValueError, KeyError):
    print("Incorrect decryption")

print("sanity check if key values are the same: ")
print(b64encode(jv['nonce']))
print(b64encode(jv['header']))
print(b64encode(jv['ciphertext']))
print(b64encode(jv['tag']))

Why does my decryption stage fail if the test vectors encrypt correctly according to IETF-draft?

{"nonce": "AAAAAFBRUlNUVVZX", "header": "UFFSU8DBwsPExcbH", "ciphertext": "vW0XnT6D1DuVdleUk8DpOVcqFwAlK/rMvtKQLCE5bLtzHH8bC0qmRAvzqC9O2n45rmTGcIxUwhbLlrcuEhO0Ui+Mm6QNtdlFsRtpuYLBu54/P6wrw2lIj3ayODVl0//5IflmTJdjfal2iBL2FcaLE7Uu", "tag": "wIdZJMHHmHlH3q/YeArPSQ=="}
Incorrect decryption
sanity check if key values are the same:
b'AAAAAFBRUlNUVVZX'
b'UFFSU8DBwsPExcbH'
b'vW0XnT6D1DuVdleUk8DpOVcqFwAlK/rMvtKQLCE5bLtzHH8bC0qmRAvzqC9O2n45rmTGcIxUwhbLlrcuEhO0Ui+Mm6QNtdlFsRtpuYLBu54/P6wrw2lIj3ayODVl0//5IflmTJdjfal2iBL2FcaLE7Uu'
b'wIdZJMHHmHlH3q/YeArPSQ=='

When I convert the byte arrays back to base64, they still match the JSON output. So reading my key values from JSON for decryption was done correctly.

Where is the mistake? I literally use a code example from the site offering pycryptodome and encryption was done correctly. It should decrypt just fine.

Solution

The decryption will be done correctly if you replace in the line

jv = [ b64encode(x).decode('utf-8') for x in (cipher.nonce, header, ciphertext, tag) ]

the expression cipher.nonce with nonce_xchacha20. The bug causes an incorrect nonce to be supplied in the JSON.

It seems that cipher.nonce can only be used to determine a randomly generated nonce (a random nonce is generated if no explicit nonce is specified when instantiating the cipher, s. here).

A second (trivial) change is in the line

print("The message was: " + plaintext)

necessary. Here a UTF8 decoding must be performed, i.e. plaintext must be replaced by plaintext.decode('utf8').

In your first post, the AADs were also set incorrectly. But this has been corrected in the meantime.

With these two changes, the code, especially the decryption, works on my machine.