scalanext.jsqr-codezxing

Why can I not encode ScalaPB `toProtoString` into a QR code and decode it again?

Background:

I have a protobuf object called RegistrationInfo, defined as such:

message RegistrationInfo {
  string serverID = 1;
  string serverIP = 2;
  string alias = 3;
  string rootCA = 4;
}

In my Scala backend, I have a QRCodeGenerator class, that creates a QR image from this object using the zxing library:

  def init(): Unit = {
    // create our own RegistrationInfo
    val regInfo = RegistrationInfo(SERVER_ID, SERVER_IP, s"${OS.getOS.toString} Server", Base64.getEncoder.encodeToString(SSLManager.loadRootCertificate().getEncoded))
    encodedQRCode = generateEncodedQRCode(regInfo.toProtoString)
  }

  private def generateEncodedQRCode(text: String): String = {
    val qrCodeWriter = new QRCodeWriter()

    // Set QR code properties
    val hintMap = new util.HashMap[EncodeHintType, Any]()
    hintMap.put(EncodeHintType.CHARACTER_SET, "UTF-8")

    // Generate the QR code as a bit matrix
    val bitMatrix = qrCodeWriter.encode(text, BarcodeFormat.QR_CODE, 150, 150, hintMap)

    // Write the bit matrix to a byte array (PNG format)
    val outputStream = new java.io.ByteArrayOutputStream()
    MatrixToImageWriter.writeToStream(bitMatrix, "PNG", outputStream)

    // Return the byte array representing the image
    Base64.getEncoder.encodeToString(outputStream.toByteArray)
  }

My NextJS application requests this image (as a Base64 encoded string) and provides a download link, like so:


    const [qrResponse, setQrResponse] = useState(null);

    function writeBase64AsPNG(base64String, fileName = "image.png") {
        // Ensure the Base64 string doesn't have the prefix `data:image/png;base64,`
        const cleanBase64String = base64String.replace(/^data:image\/png;base64,/, "");

        // Decode the Base64 string into a binary string
        const byteCharacters = atob(cleanBase64String);

        // Create an array of byte values
        const byteArray = new Uint8Array(byteCharacters.length);

        // Convert the binary string to a byte array
        for (let i = 0; i < byteCharacters.length; i++) {
            byteArray[i] = byteCharacters.charCodeAt(i);
        }

        // Create a Blob from the byte array (with MIME type for PNG images)
        return new Blob([byteArray], { type: "image/png" });
    }

    useEffect(() => {
        if (token && thisServerID) {
            getProto("/web/qr", token, router)
                .then(apiResponse => {
                    if (apiResponse.hasQrresponse()) {
                        setQrResponse(writeBase64AsPNG(apiResponse.getQrresponse().getQrdata()));
                    } else if (apiResponse.hasStatusresponse()) {
                        throw Error(apiResponse.getStatusresponse().getMessage())
                    }
                })
        }
    }, [token, thisServerID]);

...

    const downloadBlob = () => {
        if (!qrResponse) return;

        // Create an object URL for the Blob
        const url = URL.createObjectURL(qrResponse);

        // Create an anchor element and trigger the download
        const link = document.createElement('a');
        link.href = url;
        link.download = thisServerID + "_fingerprint.png"; // File name for download
        link.click();

        // Clean up the URL after the download is triggered
        URL.revokeObjectURL(url);
    };

inside component:
                                        {qrResponse && thisServerID ? (
                                            <a href="#" onClick={downloadBlob}>
                                                Download Digital Fingerprint
                                            </a>
                                        ) : (
                                            <p>Loading Digital Fingerprint...</p>
                                        )}

This process works fine, and I can download the QR image:

enter image description here

I then upload the image into the UI to test the backend processing.

The backend processing (also in QRCodeGenerator):

  def extractInfo(registrationRequest: RegistrationRequest)(implicit system: ActorSystem[_]): RegistrationInfo = {
    try {
      system.log.info(s"Full String: ${registrationRequest.qrData}")
      val cleanBase64String = registrationRequest.qrData.replaceFirst("^data:image\\/[^;]+;base64,", "")

      system.log.info(s"Cleaned String: $cleanBase64String")

      val decodedBytes = Base64.getDecoder.decode(cleanBase64String)

      system.log.info(s"Decoded byte array size: ${decodedBytes.length} bytes")

      val inputStream = new ByteArrayInputStream(decodedBytes)
      val bufferedImage: BufferedImage = ImageIO.read(inputStream)

      if (bufferedImage == null) {
        system.log.error("Error: Failed to read image from byte array.")
        null
      } else {
        // Create a LuminanceSource from the BufferedImage
        val luminanceSource = new BufferedImageLuminanceSource(bufferedImage)
        val binaryBitmap = new BinaryBitmap(new HybridBinarizer(luminanceSource))

        // Create a QRCodeReader instance
        val reader = new QRCodeReader()

        // Decode the QR code from the luminance source
        val result = reader.decode(binaryBitmap)

        JsonFormat.fromJsonString[RegistrationInfo](result.getText)
      }
    } catch {
      case e: NotFoundException =>
        // If no QR code is found, return None
        system.log.error("QR code not found in the image.", e)
        null
      case e: Exception =>
        // Any other errors (e.g., Base64 decoding failure, image reading failure)
        system.log.error(s"Error decoding QR code: ${e.getMessage}", e)
        null
    }
  }

The Problem:

I am catching the NotFoundException, meaning zxing cannot find a valid QR code in the image.

I'm unsure why. I uploaded the image to https://zxing.org/w/decode.jspx and it was able to successfully decode the image and I could see the data correctly.

Update:

I removed NextJS from the equation and just tried processing the results of generateEncodedQRCode through extractInfo. I got the same exception, suggesting there is a problem in one of the functions. I'm leaning towards the extractInfo function, seems as the zxing website was able to decode the QR.

Test init function:

  def init()(implicit system: ActorSystem[_]): Unit = {
    // create our own RegistrationInfo
    val regInfo = RegistrationInfo(SERVER_ID, SERVER_IP, s"${OS.getOS.toString} Server", Base64.getEncoder.encodeToString(SSLManager.loadRootCertificate().getEncoded))
    encodedQRCode = generateEncodedQRCode(regInfo.toProtoString)
    
    // for testing, attempt to reform regInfo
    val reformedRegInfo = extractInfo(encodedQRCode) // <-- throws "NotFoundException"
    system.log.info(s"Reformed RegInfo: ${reformedRegInfo.toProtoString}")
  }

I'm new to this QR generation stuff and the zxing library, so any suggestions what might be wrong?

update 2:

I added further debugging.

In generateEncodedQRCode function:

In extractInfo function:

I can confirm both arrays are identical. Both array sizes are identical. Both encoded strings (result of first function, passed directly to second) are identical.

I do not get to the last log, I get the NotFoundException

Update 3:

I reduced the problem even more by generating a QR code for the string "hello world" and putting it through my extract function. This worked, suggesting there is a problem with parsing toProtoString to and from a QR code.

I opted for toProtoString over using the proto's raw bytes because its human-readable and allows the user to scan it with their phone and verify the contents.

Update 4:

I switched from .toProtoString to:

JsonFormat.printer.print(regInfo)

Which made no difference. I then has a suspicion of the rootCA, as its a Base64 encoded string. I swapped it for a basic string: exampleRootCA and it works!

So how would I get the rootCA into the proto in a human-readable format that still processes as a QR?

Solution

  • So after my debugging steps outlined in the update sections of the question, I have come to the conclusion the error was caused by setting the rootCA field of the RegistraionInfo message to a Base64 encoded string.

    The solution is to wrap the Base64 encoded string into a String raw type like so:

    val rootCAString = new String(Base64.getEncoder.encodeToString(SSLManager.loadRootCertificate().getEncoded))

    And set this as the RegistrationInfo rootCA field.

    This is because zxing expects a string input that it can itself encode, not an already encoded string. Now I have the challenge of handling decoding the rootCA back into a certificate, but that is outside the scope of this question.

    I will leave this here in the hope it will help someone in the same situation in the future.

    EDIT:

    So wrapping it in raw string type was not the entire solution, not really sure how it worked once and never again.

    Turns out the data from encoding a certificate to a string is over the max size of Version 40 of QR. I'm still very confused as to how QR decoding websites were able to decode the QR and I wasn't.

    However, I now use Brotli4j to compress the certificates bytes BEFORE encoding to a QR, reducing the size of the data so it will fit into a QR.

    Example compression/decompression:

      def compressCertificate(cert: X509Certificate): Array[Byte] = {
    val certBytes = cert.getEncoded // Get the certificate bytes

    // Create a ByteArrayOutputStream to hold the compressed data
    val compressedStream = new ByteArrayOutputStream()
    val brotliOutStream = new BrotliOutputStream(compressedStream)

    // Write the certificate bytes to the BrotliOutputStream
    brotliOutStream.write(certBytes)
    brotliOutStream.close()

    // Return the compressed byte array
    compressedStream.toByteArray
  }

  def decompressCertificate(compressedBytes: Array[Byte]): Array[Byte] = {
    val compressedStream = new ByteArrayInputStream(compressedBytes)
    val brotliInStream = new BrotliInputStream(compressedStream)

    // Read all decompressed bytes into a ByteArrayOutputStream
    val decompressedStream = new ByteArrayOutputStream()
    val buffer = new Array[Byte](1024)
    var bytesRead = 0
    while ( {
      bytesRead = brotliInStream.read(buffer); bytesRead != -1
    }) {
      decompressedStream.write(buffer, 0, bytesRead)
    }

    // Return the decompressed byte array as a certificate string
    decompressedStream.toByteArray
  }