vbscriptcharacter-encodingasp-classichebrew

vbscript ADO stream charset output

My webhook is receiving POST request (aplication/JSON) from a 360Dialog (whatsapp) API with escaped Unicode characters like this: \u05db\u05e0\u05e1\u05d2\u05db\u05d9. It should be Hebrew letters.

I'm trying to decode that using JavaScript runat server but seems like it is not changing. I found a potential solution in this question's solution but it still saves the un-escaped Unicode into the database.

<script language="javascript" runat="server">
URL = {
    decode : function(s){return decodeURIComponent(s.replace(/\+/g,  " "))}
}
</script>

<%
  rs("smstext")=URL.decode(body2)
%>

The POST request is coming from 360dialog (a Whatsapp API) and hitting my webhook.

the request sends an application/json POST with information of incoming Whatsapp messages.

It seems the POST itself already has the Hebrew in it as \u05e0\u05e1\u05d9\u05d5\u05df i guess i need to figure out how to set the charset for that?

also, this unanswered question seems like my same problem.

I am trying to convert a request.BinaryRead into utf-8. the output in the database is this: \u05e0\u05e1\u05d9\u05d5\u05df instead of נסיון

I am probably misunderstanding something as the output is not what I expected.

my code is:

<%@LANGUAGE="VBSCRIPT" CODEPAGE="65001"%>
<!--#include virtual="/include/aspjson.asp" -->
<%
If Request.TotalBytes > 0 Then
    Dim lngBytesCount
        lngBytesCount = Request.TotalBytes
    body = BytesToStr(Request.BinaryRead(lngBytesCount))

Set db = CreateObject("ADODB.Connection")
    db.Open "DSN=xxx"
set rs = Server.CreateObject("ADODB.Recordset")
rs.open "SELECT * FROM log_sms", db, 3, 3
  rs.addnew
    rs("smstext")=body
  rs.update
rs.close
End if
Function BytesToStr(bytes)
    Dim Stream
    Set Stream = Server.CreateObject("Adodb.Stream")
        Stream.Type = 1 'adTypeBinary
        Stream.Open
        Stream.Write bytes
        Stream.Position = 0
        Stream.Type = 2 'adTypeText
        Stream.Charset = "utf-8"
        BytesToStr = Stream.ReadText
        Stream.Close
    Set Stream = Nothing
End Function
%>

If I replace rs("smstext")=body with rs("smstext")="נסיון", the value in the database is saved correctly.

Solution

  • The approach is sound the problem is because the text is escaped in the JSON body you will need to unescape those characters before saving the content to a database.

    Would recommend using this particular JSON Parser as it will automatically handle unescaping the characters for you.

    Useful Links