pythonwordpresscsvplonetransmogrifier

How to parse a WordPress CSV export using Python


I need to import content from WordPress into Plone, a Python-based CMS, and I have a dump of the posts table as a huge CSV vanilla file using ";" as a delimiter.

The problem is the standard CSV reader from the csv module is not smart enough to parse the HTML content inside a row (the post_content field).

For instance, when the parser encounters something like <p>&nbsp;</p> it interprets the semicolon as a field delimiter and I end up with more items than fields and with fields with wrong content.

Is there any other option to solve this kind of issues? Processing the row with a regex seems pretty scary to me.


Solution

  • After some additional research, I discovered the excel-tab dialect by reading the text of the PEP 0305 (which proposed the addition of the cvs module to Python); this is mentioned in the module documentation, but I haven't noticed at first.

    I then re-exported the posts using a tab as a delimiter (\t).

    enter image description here

    I made a test reading a batch of 1,000 rows and found no errors at all.