javaspring-bootelasticsearchspring-data-elasticsearch

How to add multiple elastic search connections to a Spring Boot project?


I have 2 elastic search servers that I need to add the connection to the spring boot project. For one connection following is the code I have,

@Configuration
@EnableElasticsearchRepositories(basePackages = {"packageName"})
public class ES {

    @Bean
    public RestHighLevelClient client() {
        CredentialsProvider cp = new BasicCredentialsProvider();
        cp.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials("username", "password"));

        return new RestHighLevelClient(RestClient.builder(new HttpHost("host", "port", "https"))
                        .setHttpClientConfigCallback(httpAsyncClientBuilder -> httpAsyncClientBuilder.setDefaultCredentialsProvider(cp)
                                        .setKeepAliveStrategy(new DefaultConnectionKeepAliveStrategy())));
    }
}

So how can I add the other server connection? If I duplicate this class and add other server connection details will it work? And also I need to separate the indexes. I mean for example I need the index entity1 must be only in server 1 and the index entity2 must be only in server 2. Is that possible? Really appreciate if anyone can assist me.


Solution

  • This is currently not possible with Spring Data Elasticsearch. A repository uses the ElasticsearchOperations bean provided by the Spring context, and this bean uses the configured RestHighLevelClient. There is no way to configure multiple connections to different clusters.

    Edit:

    You could have one connection configured as the default using the beans and then you can use this injected as ElasticsearchOperations and used by the repository method.

    For the second connection you'd need to create a new RestHighLevelClient like you already do and then create an additional ElasticsearchRestTemplate passing in this second client; use an injected ElasticsearchConverter as the second constructor parameter. You can check the code in org.springframework.data.elasticsearch.config.AbstractElasticsearchConfiguration how it's done there.

    With this second ElasticsearchOperations instance you can work against the second cluster, but you will have to use the methods of this interface and cannot use repository methods for that.

    Edit 29.01.2024:

    I recently wrote a short article about doing this: https://www.sothawo.com/post/2023/10/18/use-two-different-elasticsearch-clusters-with-spring-data-elasticsearch/