How to set index.mapping.ignore_malformed on ASP.NET MVC with ElasticsearchClientSettings

I'm using elastic search on ASP.NET MVC to audit data with Audit.NET library but sometimes I get below error whenever I tried to call db.SaveChanges function.

Elastic.Transport.TransportException: 'The remote server returned an error: (400) Bad Request. Call: Status code 400 from: PUT /index_name/_doc/a03cf431-de33-4bc6-946f-aa583977e07f. ServerError: Type: document_parsing_exception Reason: "[1:219] failed to parse field [entityFrameworkEvent.entries.changes.originalValue] of type [date] in document with id 'a03cf431-de33-4bc6-946f-aa583977e07f'. Preview of field's value: 'b67265c6-8653-4224-8106-c594c8341eb3'" CausedBy: "Type: illegal_argument_exception Reason: "failed to parse date field [b67265c6-8653-4224-8106-c594c8341eb3] with format [strict_date_optional_time||epoch_millis]" CausedBy: "Type: date_time_parse_exception Reason: "Failed to parse with all enclosed parsers"""'

From different sources it seems to set index.mapping.ignore_malformed might help to solve the problem but I don't find any documentation on how to do it. Below is my setup on startup file.

var settings = new ElasticsearchClientSettings(new Uri("IP Address"))                
    .DefaultIndex("index_name")
    .CertificateFingerprint("certificate_fingerprint")
    .Authentication(new BasicAuthentication("username", "password"));

Audit.Core.Configuration.Setup()
    .UseElasticsearch(config => config
    .Client(new ElasticsearchClient(settings)));

Any help will be appreciated

The issue is caused by Elasticsearch's Dynamic Field Mapping.

The NewValue/OriginalValue properties in EventEntryChange class are of type object, meaning Elasticsearch determines its type based on the first indexed document. If the first indexed document contains a DateTime value for OriginalValue, Elasticsearch will map the field as a date. Consequently, if a subsequent document contains a different data type for OriginalValue, indexing will fail due to a type mismatch.

To avoid this issue, you have a couple of options:

1. Use a Custom Index Template (For New Indexes Only)

You can define an index template to explicitly map the field as a TextProperty, ensuring consistent data types across all indexed documents:

var result = await elasticClient.Indices.PutIndexTemplateAsync("target_object", x => x
    .IndexPatterns(indexName)
    .Template(map =>
        map.Mappings(m => m.Properties(new Properties() 
        { 
            { "entityFrameworkEvent.entries.changes.originalValue", new TextProperty() }, 
            { "entityFrameworkEvent.entries.changes.newValue", new TextProperty() }
        })))); 

This approach, however, will only work for newly created indexes.

2. Convert Object Values to JSON Strings

Alternatively, you can override object values and convert them to JSON strings before indexing. This ensures that Elasticsearch treats all values as string data, preventing type conflicts:

Audit.Core.Configuration.AddCustomAction(ActionType.OnEventSaving, scope =>
{
    foreach (var entry in scope.Event.GetEntityFrameworkEvent().Entries)
    {
        // Convert Changes properties to JSON strings
        if (entry.Changes != null)
        {
            foreach (var change in entry.Changes)
            {
                change.NewValue = ToJson(change.NewValue);
                change.OriginalValue = ToJson(change.OriginalValue);
            }
        }
        
        // Convert ColumnValues properties to JSON strings
        if (entry.ColumnValues != null)
        {
            foreach (var colKey in entry.ColumnValues.Keys.ToList())
            {
                entry.ColumnValues[colKey] = ToJson(entry.ColumnValues[colKey]);
            }
        }
        
        // Convert Primary Key properties to JSON strings
        if (entry.PrimaryKey != null)
        {
            foreach (var pkKey in entry.PrimaryKey.Keys.ToList())
            {
                entry.PrimaryKey[pkKey] = ToJson(entry.PrimaryKey[pkKey]);
            }
        }
    }
});

private static string ToJson(object obj)
{
    return obj == null ? null : JsonSerializer.Serialize(obj);
}

Please also check the following Issues: 152, 499, 504, and 591.