pythondbf

Some (Python) dbf fields are filled with None. Why?


In the following code, I'm testing some numeric fields, but a TypeError is being thrown.

import dbf

dbf_orders = dbf.Table('../../data/ORDERS.DBF')
idx_orders_OrDtTm = dbf_orders.create_index(lambda rec: str(rec.ODATE) + rec.OTIME.decode('utf-8') + formatRecNo(rec))

thisRec_Orders = idx_orders_OrDtTm.current_record

if thisRec_Orders['Received'] > 0 \
        and (thisRec_Orders['oTotal'] + thisRec_Orders['Change']) < thisRec_Orders['Received']:

The Change field has a value of None, but the data type is numeric. Output from Table.structure() shows 'CHANGE N(7,2) BINARY'.

However, the value of the Change field is not always None. Most of the time, the field does have a numeric value.

And, it's generally annoying that character fields are represented as binary.

Is there a way to get the data types in a record with valid data for their field types? I would expect this to be the default action.

I don't want to have to check every field for valid data before I use it. I could write a routine to fix data types and values when assigning current_record to a variable, but I'm hoping there is a better way.


Solution

  • The reason None can be returned is that it is possible to have a field be empty. For example, Logical fields should be T or F, but can be ' ' (a space). If empty is not a valid value for the field type, then None is returned (e.g. nothing but spaces is a valid value for a Character field).

    The best solution to that problem is for the creating program to store valid values in the fields.

    The next best solution is to tell dbf which type you want when nothing is stored in the field:

    ... = dbf.Table('...', default_data_types={'N': int, 'L':bool})
    

    and probably create your own function to return tables with your default settings.


    Note: the specification for the default_data_types parameter is

    { data-type: python-type-for-any-or-no-value,
      data-type: (python-type-for-any-value, python-type-for-no-value),
      data-type: (python-type-for-any-value, python-type-for-no-value, python-type-for-null),
      }
    

    If less than all three types are specified, the last one is used to fill out the triple -- so 'N': int is the same as 'N': (int, int, int), meaning that whatever was stored in the field, you'll get an int back.


    Note: some data types, such as date.date and date.datetime do not have an empty/0 possibility, and calling them without a value raises a ValueError or a TypeError.


    Note: The field is showing BINARY because it has the binary flag set in the field header in the dbf file.