amazon-dynamodbamazon-cloudsearch

DynamoDB table --> CloudSearch == Field "l" does not exist in domain configuration


Summary

I have a DynamoDB table with less than ten items that I am trying index with CloudSearch. The CloudSearch indexer suggests a non-existent attribute named "l". This appears to be somehow sourced from the DynamoDB JSON "list" key (which stores an array of list objects, in my case, all strings). Regardless, even if I remove this key at index field configuration time, I still get the following hard error at document upload time:

{["Field "l" does not exist in domain configuration (near operation with index 3; document_id null)",""add" operation must contain at least one "field" (near operation with index 5; document_id null)",""add" operation must contain at least one "field" (near operation with index 7; document_id null)","Field "l" does not exist in domain configuration (near operation with index 11; document_id null)",""add" operation must contain at least one "field" (near operation with index 13; document_id null)",""add" operation must contain at least one "field" (near operation with index 15; document_id null)","Field "l" does not exist in domain configuration (near operation with index 19; document_id null)",""add" operation must contain at least one "field" (near operation with index 21; document_id null)",""add" operation must contain at least one "field" (near operation with index 23; document_id null)","Field "l" does not exist in domain configuration (near operation with index 27; document_id null)",""add" operation must contain at least one "field" (near operation with index 29; document_id null)",""add" operation must contain at least one "field" (near operation with index 31; document_id null)","Field "l" does not exist in domain configuration (near operation with index 35; document_id null)",""add" operation must contain at least one "field" (near operation with index 37; document_id null)",""add" operation must contain at least one "field" (near operation with index 39; document_id null)"] }

Detail

Hello! I have a DynamoDB table containing fewer than ten items that have ~35 variously typed attributes (strings, numbers, lists of strings, and maps). I am trying to index this table with CloudSearch via the aws console gui. When I proceed to the index field configuration page most of my attributes show -- notably missing are the lists. However, the indexer does suggest "l" as per the following image:

enter image description here

I assume that to be the DynamoDB JSON list key. I've tried keeping and removing this suggestion and moving on and creating the search domain. Either way, ten minutes later, when I try to upload documents I get the above error.

I've found that removing all lists from my table items results in this "l" suggestion disappearing and eventual document upload proceeding as expected.

The DynamoDB table items are created via the api (using boto3's DynamoDB "Table" interface as per the following:

body = {}
# body is populated as a regular python dict
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table("table_name")
table.put_item(Item = body, ConditionExpression = 'attribute_not_exists(ds_name)')

I was thinking perhaps there is a edge case here and I should use the more explicit boto3 DynamoDB client interface to create the items -- but using the above method works without error and the item appears to be completely correct and normal in the DynamoDB gui viewer (both in tree and text mode) as well as from an api ".get_item()" call.

Possibly related: Amazon CloudSearch creates Null ID from DynamoDB


Solution

  • Got a response from AWS Support -- as of this writing, the list and map types are not supported by CloudSearch. There is an open request to add them but "there is no ETA on when they will be supported". A suggested alternative is to use String Set in place of a list of strings which should work for my use case.