pythonshlex

shlex include empty strings

sample = ",,"
values = shlex.shlex(sample, posix=True)
values.quotes = '"'
values.whitespace = ','
values.whitespace_split = True

received_output = list(values)

In the above code sample I would like to have ["", "", ""] as the value of received_output, but received_output is simply an empty list []. There does not seem to be any information regarding how to receive this expected behavior.

This works fine with sample.split(','), but I would prefer using shlex since I have complex sentences with tokens which should not be split up if part of a group (such as latitude, longitude in the following example).

Another example:

sample = '9267,BELMONT,KEELER,,62.4,35.2,10/01/2012,Weekday,"(41.93897000, -87.73212000)"'

expected_output = ['9267', 'BELMONT', 'KEELER', '', '62.4', '35.2', '10/01/2012', 'Weekday', '(41.93897000, -87.73212000)']
retrieved_output = ['9267', 'BELMONT', 'KEELER', '62.4', '35.2', '10/01/2012', 'Weekday', '(41.93897000, -87.73212000)']
Solution

  • The shlex docs state:

    • It’s not possible to parse empty strings, even if quoted.

    If you want to include empty strings in your output, the shlex library is the wrong tool for the job.

    As @PadraicCunningham points out in a comment, the csv (Comma Separated Values) library should work fine for this:

    >>> list(csv.reader(['9267,BELMONT,KEELER,,62.4,35.2,10/01/2012,Weekday,"(41.93897000, -87.73212000)"']))[0]
['9267', 'BELMONT', 'KEELER', '', '62.4', '35.2', '10/01/2012', 'Weekday', '(41.93897000, -87.73212000)']
>>> list(csv.reader([',,']))[0]
['', '', '']