pythonarcgis

Unknown error when trying to 'upsert' with ArcGIS API for Python

I am trying to perform an 'upsert' (update + insert) operation on a feature layer via ArcGIS API for Python. My input data is a geojson file that gets uploaded as per the available guides:

data_item = gis.content.add(
    item_properties={
        'title': 'My Data File',
        'type': 'GeoJson',  
        'overwrite': True,
    },
    data="myfile.geojson"
)

Then I attempt the append operation:

my_layer.append(
    item_id=data_item.id,
    upload_format='geojson', 
    upsert=True,
    upsert_matching_field='an_unique_field',
    update_geometry=True,
)

This returns Exception: Unknown Error (Error Code: 500). The geojson file is valid (checked in geojson.io). Attempts with a smaller file (original has ~4000 polygons) or appending a featureCollection instead of geojson fail with the same error. Manually updating the data via ArcGIS online works fine with the exact same file. Any leads here?

I have succesfully performed the operation using edit_features() instead of append() but it requires querying the layer, mapping OBJECTIDs to an_unique_field, creating separate adds and updates collections, then splitting the collections into 250 features chunks. I'd like to skip all this with a single append call if possible.

Solution

  • It turned out to be due to missing values in the upsert_matching_field. I had missing values in other properties (which are configured to allow null values) too. Correcting those got rid of the unknown error, but the function would still fail to complete (it ran for 2 hours then errored out with a permission denied message).

    Kind of silly, but this could be better documented and the error should be more informative. I assumed the method would skip blank/falsy values. The part about source_table_name being required also seems to be a documentation flaw.