jythonstreamsets

I'm trying to create and pass a new record which contains a map from the jython processor in streamsets but getting this error?


I want the newRecord to containt a map of column names and column values. I am getting the following error which I am not able to resolve - Record1-Error Record1 SCRIPTING_04 - Script sent record to error: write(): 1st arg can't be coerced to com.streamsets.pipeline.stage.util.scripting.ScriptRecord : (View Stack Trace... )

from datetime import datetime
metadata_dict = {}
for metadata in sdc.records[0].value['XMLData']['Metadata'][0]['FieldDefinitions'][0]['FieldDefinition']:
  metadata_dict [metadata['attr|id']] = metadata ['attr|alias']
    
  
for record in sdc.records:
  try: 
    for row in record.value['XMLData']['Record']:
      newRecord = sdc.createRecord(str(datetime.now()))
      newRecord = sdc.createMap (False)
      value = row ['Field']
      for values in value:
        column_id = values ['attr|id']
        column_name = metadata_dict [column_id]
        for a in values:
          if a == 'value':
            column_value = values ['value']
          elif a == 'ListValues':
            column_value = values ['ListValues']
          elif a == 'Groups':
            column_value = values ['Groups']
          elif a == 'Users':
            column_value = values ['Users']
        newRecord[column_name] = column_value
      sdc.output.write(newRecord)
  except Exception as e:
    sdc.error.write(record, str(e))

Solution

  • You have a bug in your code:

    newRecord = sdc.createMap (False)
    

    Here, you create a map and place it into a newRecord variable.

    sdc.output.write(newRecord)
    

    Here, you're trying to write a map, not a record, into the output.

    You should do something like:

    newRecord = sdc.createRecord(...
    myMap = sdc.createMap(...)
    myMap['foo'] = 'bar'
    newRecord.value = myMap