Im doing some data cleansing on some messy data which is being imported into mysql.
The data contains 'pseudo' unicode chars, which are actually embedded into the strings as 'u00e9' etc.
So one field might be.. 'Jalostotitlu00e1n' I need to rip out that clumsy 'u00e1n' and replace it with the corresponding utf character
I can do this in either mysql, using substring and CHR maybe, but Im preprocssing the data via PHP, so I could do it there also.
I already know all about how to configure mysql and php to work with utf data. The problem is really just in the source data Im importing.
Thanks
There's a way. Replace all uXXXX
with their HTML representation and do an html_entity_decode()
I.e. echo html_entity_decode("Jalostotitlán");
Every UTF character in the form u1234
could be printed in HTML as ሴ
. But doing a replace is quite hard, because there could be much false positives if there is no other char that identifies the beginning of an UTF sequence. A simple regex could be
preg_replace('/u([\da-fA-F]{4})/', '&#x\1;', $str)