javaspring-bootelasticsearchjsonbhibernate-search

Hibernate Search JsonB indexing


I am struggling with indexing jsonB column into Elasicsearch backend, using Hibernate Search 6.0.2

This is my entity:

@Data
@NoArgsConstructor
@Entity
@Table(name = "examples")
public class Example {

    @Id
    @GeneratedValue(strategy = GenerationType.AUTO)
    private UUID id;

    @NotNull
    @Column(name = "fields")
    @Type(type = "jsonb")
    private Map<String, Object> fields;
}

and this is my programmatic mapping of elasticsearch backend for Hibernate Search:

@Configuration
@RequiredArgsConstructor
public class ElasticsearchMappingConfig implements HibernateOrmSearchMappingConfigurer {

    private final JsonPropertyBinder jsonPropertyBinder;

    @Override
    public void configure(HibernateOrmMappingConfigurationContext context) {
        var mapping = context.programmaticMapping();
        var exampleMapping = mapping.type(Example.class);
        exampleMapping.indexed();
        exampleMapping.property("fields").binder(jsonPropertyBinder);
    }
}

I based my custom property binder implementation on Hibernate Search 6.0.2 documentation.

@Component
public class JsonPropertyBinder implements PropertyBinder {

    @Override
    public void bind(PropertyBindingContext context) {
        context.dependencies().useRootOnly();
        var schemaElement = context.indexSchemaElement();
        var userMetadataField = schemaElement.objectField("metadata");
        context.bridge(Map.class, new Bridge(userMetadataField.toReference()));
    }

    @RequiredArgsConstructor
    private static class Bridge implements PropertyBridge<Map> {

        private final IndexObjectFieldReference fieldReference;

        @Override
        public void write(DocumentElement target, Map bridgedElement, PropertyBridgeWriteContext context) {
            var map = target.addObject(fieldReference);
            ((Map<String, Object>) bridgedElement).forEach(map::addValue);
        }
    }

}

I am aware that documentation defines multiple templates for what an Object in Map can be (like in MultiTypeUserMetadataBinder example), but I really do not know what can be inside. All I know, it is a valid json and my goal is to put it into Elasticsearch as valid json structure under "fields": {...}

In my case jsonB column may contain something like this:

    {
        "testString": "298",
        "testNumber": 123,
        "testBoolean": true,
        "testNull": null,
        "testArray": [
            5,
            4,
            3
        ],
        "testObject": {
            "testString": "298",
            "testNumber": 123,
            "testBoolean": true,
            "testNull": null,
            "testArray": [
                5,
                4,
                3
            ]
        }

but it throws an exception:

org.hibernate.search.util.common.SearchException: HSEARCH400609: Unknown field 'metadata.testNumber'.

I have also set dynamic_mapping to true in my spring application:

...
spring.jpa.properties.hibernate.search.backend.hosts=127.0.0.3:9200
spring.jpa.properties.hibernate.search.backend.dynamic_mapping=true
...

Any other ideas how can I approach this problem? Or maybe I made an error somewhere?


Solution

  • I am aware that documentation defines multiple templates for what an Object in Map can be (like in MultiTypeUserMetadataBinder example), but I really do not know what can be inside. All I know, it is a valid json and my goal is to put it into Elasticsearch as valid json structure under "fields": {...}

    If you don't know what the type of each field is, Hibernate Search won't be able to help much. If you really want to stuff that into your index, I'd suggest declaring a native field and pushing the JSON as-is. But then you won't be able to apply predicates to the metadata fields easily, except using native JSON.

    Something like this:

    @Component
    public class JsonPropertyBinder implements PropertyBinder {
    
        @Override
        public void bind(PropertyBindingContext context) {
            context.dependencies().useRootOnly();
            var schemaElement = context.indexSchemaElement();
    
            // CHANGE THIS
            IndexFieldReference<JsonElement> userMetadataField = schemaElement.field( 
                    "metadata",
                    f -> f.extension(ElasticsearchExtension.get())
                            .asNative().mapping("{\"type\": \"object\", \"dynamic\":\"true\"}");
            )
                    .toReference();
    
            context.bridge(Map.class, new Bridge(userMetadataField));
        }
    
        @RequiredArgsConstructor
        private static class Bridge implements PropertyBridge<Map> {
            private static final Gson GSON = new Gson();
    
            private final IndexFieldReference<JsonElement> fieldReference;
    
            @Override
            public void write(DocumentElement target, Map bridgedElement, PropertyBridgeWriteContext context) {
                // CHANGE THIS
                target.addValue(fieldReference, GSON.toJsonTree(bridgedElement));
            }
        }
    
    }
    

    Alternatively, you can just declare all fields as strings. Then all features provided by Hibernate Search on string types will be available. But of course things like range predicates or sorts will lead to strange results on numeric values (2 is before 10, but "2" is after "10").

    Something like this:

    @Component
    public class JsonPropertyBinder implements PropertyBinder {
    
        @Override
        public void bind(PropertyBindingContext context) {
            context.dependencies().useRootOnly();
            var schemaElement = context.indexSchemaElement();
            var userMetadataField = schemaElement.objectField("metadata");
    
            // ADD THIS
            userMetadataField.fieldTemplate( 
                    "userMetadataValueTemplate_default",
                    f -> f.asString().analyzer( "english" )
            );
    
            context.bridge(Map.class, new Bridge(userMetadataField.toReference()));
        }
    
        @RequiredArgsConstructor
        private static class Bridge implements PropertyBridge<Map> {
    
            private final IndexObjectFieldReference fieldReference;
    
            @Override
            public void write(DocumentElement target, Map bridgedElement, PropertyBridgeWriteContext context) {
                var map = target.addObject(fieldReference);
                // CHANGE THIS
                ((Map<String, Object>) bridgedElement).forEach(entry -> map.addValue( entry.getKey(), String.valueOf(entry.getValue())));
            }
        }
    
    }