[SOLVED] Trying to guess the encoding of a file using chardet

Trying to guess the encoding of a file using chardet

I'm writing a program that works with CSV files. These files can have a specific encoding. I'm trying to incorporate a procedure to try to guess what the encoding of a file the user wants to open using chardet.

I'm trying with the following code:

rawdata = open('file.csv', "r").read()
result = chardet.detect(rawdata)

But i'm getting the following exeption:

/usr/lib/python3.5/site-packages/chardet/__init__.py in detect(aBuf)
     23     if ((version_info < (3, 0) and isinstance(aBuf, unicode)) or
     24             (version_info >= (3, 0) and not isinstance(aBuf, bytes))):
---> 25         raise ValueError('Expected a bytes object, not a unicode object')
     26 
     27     from . import universaldetector

ValueError: Expected a bytes object, not a unicode object

I've also tried:

result = chardet.detect(bytes(rawdata))

But getting:

TypeError                                 Traceback (most recent call last)
<ipython-input-47-1137b0adb486> in <module>()
----> 1 result = chardet.detect(bytes(rawdata))

TypeError: string argument without an encoding

Here is a part of the file that i'm trying to open:

rawdata

'þFILEPATHþ\x14þidþ\x14þdocidþ\x14þBEGBATESþ\x14þENDBATESþ\x14þBEGATTACHIDþ\x14þENDATTACHIDþ\x14þCUSTODIANþ\x14þRECIPIENTþ\x14þFROMþ\x14þCCþ\x14þBCCþ\x14þDATESENTþ\x14þTIMESENTþ\x14þSUBJECTþ\x14þDATERCVDþ\x14þTIMERCVDþ\x14þMESSAGEIDþ\x14þPARENTIDþ\x14þCREATEDATEþ\x14þCREATETIMEþ\x14þMODDATEþ\x14þMODTIMEþ\x14þLASTACCDATEþ\x14þLASTACCTIMEþ\x14þFILESIZEþ\x14þNATIVELINKþ\x14þMD5HASHþ\x14þSHA1HASHþ\x14þFILENAMEþ\x14þFILEEXTENSþ\x14þTEXTPATH2þ\x14þPSTNAMEþ\x14þMSGFILETYPþ\x14þMIMETYPþ\x14þISNISTþ\x14þFILESIZEþ\x14þHASATTACHþ\x14þATTRIBUTESþ\x14þPRIORITYþ\x14þSENSITIVITYþ\x14þIMPORTANCEþ\x14þISPRIVATEþ\x14þBUSYSTATþ\x14þMSGFILETYPþ\x14þMSGFLAGSþ\x14þKEYWORDSþ\x14þCATEGORIESþ\x14þMSGFILETYPþ\x14þAUTHORþ\x14þATTACHLISTþ\x14þFROMDOMAINþ\x14þTODOMAINþ\x14þMTGWHEREþ\x14þMTGWHENþ\x14þMGTSTARTDATEþ\x14þMTGSTARTTIMEþ\x14þMTGENDDATEþ\x14þMTGENDTIMEþ\x14þMTGDURþ\x14þMTGZONEþ\x14þREMINDDATEþ\x14þREMINDTIMEþ

Solution

How about reading data as binary in the first place?

rawdata = open('file.csv', "rb").read()