My application uses Django non-rel. I don't have access to model.
I have my bulkloader.yaml
file autogenerated by appcfg.py create_bulkloader_config
.
Problem is entities numeric ID's are being imported as string key names. So if I export entity with int ID of, for example, '62', it gets imported as entity with string key name of '61' which screws up Django.
Revelant bulkloader.yaml Fragment:
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
I'm trying to setup download/upload od data using bulkloader, and I want to have data as easy to understand format (like .csv) --- so using bulkloader.py --dump (...)
is not a viable option since it gives me sqlite3 files that have entities contents pickled as a single row.
I tried doing what @Nick suggested and I got an exception:
ErrorOnTransform: Numeric keys are not supported on input at this time.
Does this mean that I have to stick to bulkloader.py (that uses that werid sqlite format) or I messed something? ;)
Header of Transformer:
- kind: auth_user
connector: csv
connector_options:
encoding: utf-8
skip_import_header_row: True
print_export_header_row: True
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
import_transform: transform.create_foreign_key('auth_user', key_is_id=True)
Whole Stacktrace:
Traceback (most recent call last):
File "/opt/google/appengine/google/appengine/tools/adaptive_thread_pool.py", line 150, in WorkOnItems
status, instruction = item.PerformWork(self.__thread_pool)
File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 693, in PerformWork
transfer_time = self._TransferItem(thread_pool)
File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 848, in _TransferItem
self.content = self.request_manager.EncodeContent(self.rows)
File "/opt/google/appengine/google/appengine/tools/bulkloader.py", line 1269, in EncodeContent
entity = loader.create_entity(values, key_name=key, parent=parent)
File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 385, in create_entity
return self.dict_to_entity(input_dict, self.bulkload_state)
File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 131, in dict_to_entity
instance = self.__create_instance(input_dict, bulkload_state_copy)
File "/opt/google/appengine/google/appengine/ext/bulkload/bulkloader_config.py", line 209, in __create_instance
'Numeric keys are not supported on input at this time.')
You've got the export_transform 'key_id_or_name_as_string', but you don't have a corresponding import transform. Try this:
property_map:
- property: __key__
external_name: key
export_transform: transform.key_id_or_name_as_string
import_transform: transform.create_foreign_key('Kind', key_is_id=True)
Where 'Kind' is the name of the kind identified in the config file.