I'm trying to migrate Oracle data with 'we8mswin1252
' encoding to my postgres database which has 'UTF8
' encoding.
I'm using foreign data wrapper.
I'm getting
invalid byte sequence error
What should i do?
Such errors can be caused by two things:
There may be zero bytes in your Oracle strings. That is allowed in Oracle (even though it is problematic), but forbidden in PostgreSQL.
It is easy to get data corruption in Oracle, because it is pretty sloppy with encoding checks and allows you to insert arbitrary illegal byte sequences when client encoding and server encoding are the same.
There are two approaches to dealing with this problem:
The correct way: fix the data on the Oracle side. oracle_fdw
will support you by telling you which row in the result set caused the error.
The sloppy way: use a PostgreSQL database with database encoding sql_ascii
, which will allow you to store anything in a string (except zero bytes).