[SOLVED] How to get rid of invisible characters from a barcode scanner?

How to get rid of invisible characters from a barcode scanner?

I am using a barcode scanner to auto fill some fields within my program. From monitoring the data variable, I know that the program is working as far as recognizing the barcode scanner and collecting the scanned barcode. The issue is that when I scan a barcode it also adds ChrW(2) to be beginning and ChrW(3) to the end. For example,'Data' ChrW(2) & "GDB00015" & ChrW(3). The barcode is just supposed to scan GDB00015 but with the added text it is throwing off my filter. I have seen something about activating/deactivating invisible characters, but my scanner model does not seem to allow for that. I am using a Symbol brand scanner model Li2208 . I also know the serial port scans in the data in a 32-bit process, but I am unsure if that would be the cause.

    Private Sub SerialPort1_DataReceived(sender As Object, e As IO.Ports.SerialDataReceivedEventArgs) Handles SerialPort1.DataReceived
    
        Dim ErrorReturn As Short = 0 'Used by error handler to store error number
        Dim ErrorText As String 'Used by error handler to store error message
        Dim DataRxd(4096) As Byte 'raw data from serial port
        Dim ReadLength As Short 'length of the data that cam in
        Dim WaitTime As Short 'stores how long we wait for all data to get here
        Dim idx As Short 'misc used for loops
        Dim DataStringRxd As String = "" 'stores string of data we get from serial port

ErrorRetry:
        Try



            'do the waiting
            Sleep(300)

            ReadLength = SerialPort1.BytesToRead

            'get the data
            SerialPort1.Read(DataRxd, 1, 4096)

            For idx = 1 To ReadLength
                DataStringRxd = DataStringRxd & System.Char.ConvertFromUtf32(DataRxd(idx))
            Next idx

            'Debug.Print(DataStringRxd)

            DecodeBarCode(DataStringRxd)

There is more to the sub but it is strictly to send error codes so I did not add it.

I tried adding breakpoints to follow the data as it was taken in, but it seems to be coming from when it is first scanned in. I also presume that it has to do with the utf32 conversion. But I still am not positive on how to go about removing the beginning and end characters.

Solution

So, the issue was coming from the conversion from utf32 to utf16. Since utf32 uses 4 bytes per character and utf16 uses only 2 bytes per character, the conversion to UTF-16 introduces surrogate pairs to handle supplementary characters which was causing the invisible characters I had mentioned. The way I fixed it worked for my situation, but I am unsure if it would be a proper fix.

For idx = 2 To ReadLength - 1
    DataStringRxd = DataStringRxd & System.Char.ConvertFromUtf32(DataRxd(idx))
Next idx

'Debug.Print(DataStringRxd)

DecodeBarCode(DataStringRxd)

I basically now tell my program to ignore the first and last character when completing it conversion and I no longer deal with the surrogate pairs when filtering the barcodes.