I am upgrading a Maven project from Spring Boot 2.7 to Spring Boot 3.1.5, and with it moving to Spring Data Elasticsearch 5.1 and Elasticsearch 7.17.2 to 8.7.
Following the update to Spring Data Elasticsearch, they have removed the dependency which contains:
org.elasticsearch.index.query.QueryBuilders;
So it ships by default with a new set of query objects under:
co.elastic.clients.elasticsearch._types.query_dsl.QueryBuilders;
However, the new Query type is incompatible with Elasticsearchoperations, which is what I was able to use in the previous version of Elasticsearch.
I have tried replacing it with:
co.elastic.clients.elasticsearch.ElasticsearchClient;
Because this uses the new Query class, but as a commenter suggested I seemed to have got my wires crossed. If I try to @Autowire the ElasticsearchClient and run my search, I get the error "Connection is closed":
Unsatisfied dependency expressed through constructor parameter 4: Error creating bean with name 'myESRepository' defined in org.MyProject....MyESRepository defined in @EnableElasticsearchRepositories declared on IndexingConfiguration: Failed to instantiate [org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository]: Constructor threw exception
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:801)
at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:240)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1352)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1189)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:560)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:973)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:950)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:616)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:738)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:440)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:316)
at org.springframework.boot.test.context.SpringBootContextLoader.lambda$loadContext$3(SpringBootContextLoader.java:137)
at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:58)
at org.springframework.util.function.ThrowingSupplier.get(ThrowingSupplier.java:46)
at org.springframework.boot.SpringApplication.withHook(SpringApplication.java:1406)
at org.springframework.boot.test.context.SpringBootContextLoader$ContextLoaderHook.run(SpringBootContextLoader.java:545)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:137)
at org.springframework.boot.test.context.SpringBootContextLoader.loadContext(SpringBootContextLoader.java:108)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContextInternal(DefaultCacheAwareContextLoaderDelegate.java:187)
at org.springframework.test.context.cache.DefaultCacheAwareContextLoaderDelegate.loadContext(DefaultCacheAwareContextLoaderDelegate.java:119)
... 73 more
Caused by: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'myESRepository' defined in org.blah.MyESRepository defined in @EnableElasticsearchRepositories declared on IndexingConfiguration: Failed to instantiate [org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository]: Constructor threw exception
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1770)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:598)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:520)
at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:325)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:323)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:199)
at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:254)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1417)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1337)
at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:910)
at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:788)
... 97 more
Caused by: org.springframework.beans.BeanInstantiationException: Failed to instantiate [org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository]: Constructor threw exception
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:224)
at org.springframework.data.repository.core.support.RepositoryFactorySupport.lambda$instantiateClass$5(RepositoryFactorySupport.java:571)
at java.base/java.util.Optional.map(Optional.java:260)
at org.springframework.data.repository.core.support.RepositoryFactorySupport.instantiateClass(RepositoryFactorySupport.java:571)
at org.springframework.data.repository.core.support.RepositoryFactorySupport.getTargetRepositoryViaReflection(RepositoryFactorySupport.java:536)
at org.springframework.data.elasticsearch.repository.support.ElasticsearchRepositoryFactory.getTargetRepository(ElasticsearchRepositoryFactory.java:79)
at org.springframework.data.repository.core.support.RepositoryFactorySupport.getRepository(RepositoryFactorySupport.java:317)
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.lambda$afterPropertiesSet$5(RepositoryFactoryBeanSupport.java:279)
at org.springframework.data.util.Lazy.getNullable(Lazy.java:245)
at org.springframework.data.util.Lazy.get(Lazy.java:114)
at org.springframework.data.repository.core.support.RepositoryFactoryBeanSupport.afterPropertiesSet(RepositoryFactoryBeanSupport.java:285)
at org.springframework.data.elasticsearch.repository.support.ElasticsearchRepositoryFactoryBean.afterPropertiesSet(ElasticsearchRepositoryFactoryBean.java:69)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1817)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1766)
... 108 more
Caused by: org.springframework.dao.DataAccessResourceFailureException: Connection is closed
at org.springframework.data.elasticsearch.client.elc.ElasticsearchExceptionTranslator.translateExceptionIfPossible(ElasticsearchExceptionTranslator.java:107)
at org.springframework.data.elasticsearch.client.elc.ElasticsearchExceptionTranslator.translateException(ElasticsearchExceptionTranslator.java:63)
at org.springframework.data.elasticsearch.client.elc.ChildTemplate.execute(ChildTemplate.java:73)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.doExists(IndicesTemplate.java:177)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.exists(IndicesTemplate.java:169)
at org.springframework.data.elasticsearch.repository.support.SimpleElasticsearchRepository.<init>(SimpleElasticsearchRepository.java:83)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:211)
... 121 more
Caused by: java.lang.RuntimeException: Connection is closed
at org.springframework.data.elasticsearch.client.elc.ElasticsearchExceptionTranslator.translateException(ElasticsearchExceptionTranslator.java:62)
... 131 more
Caused by: org.apache.http.ConnectionClosedException: Connection is closed
at org.elasticsearch.client.RestClient.extractAndWrapCause(RestClient.java:920)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:300)
at org.elasticsearch.client.RestClient.performRequest(RestClient.java:288)
at co.elastic.clients.transport.rest_client.RestClientTransport.performRequest(RestClientTransport.java:153)
at co.elastic.clients.elasticsearch.indices.ElasticsearchIndicesClient.exists(ElasticsearchIndicesClient.java:620)
at org.springframework.data.elasticsearch.client.elc.IndicesTemplate.lambda$doExists$2(IndicesTemplate.java:177)
at org.springframework.data.elasticsearch.client.elc.ChildTemplate.execute(ChildTemplate.java:71)
... 130 more
Caused by: org.apache.http.ConnectionClosedException: Connection is closed
at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.endOfInput(HttpAsyncRequestExecutor.java:356)
at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:261)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:87)
at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:40)
at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:591)
at java.base/java.lang.Thread.run(Thread.java:833)
In the previous version, I could use @Autowired on ElasticsearchOperations, and in my integration tests, it would correctly connect to a running Elasticsearch Testcontainer.
I have multiple tests that share the test container; any test which uses it simply has to implement this interface:
@SpringBootTest
@Testcontainers
public interface ElasticsearchIntegrationTest {
final DockerImageName ES_IMAGE = DockerImageName.parse("docker.elastic.co/elasticsearch/elasticsearch:8.7.0");
static ElasticsearchContainer elasticsearchContainer = new ElasticsearchContainer(ES_IMAGE);
@DynamicPropertySource
static void elasticsearchProperties(final DynamicPropertyRegistry dynamicPropertyRegistry) {
final String username = "elastic";
final String password = "password";
elasticsearchContainer.withPassword(password);
elasticsearchContainer.start();
dynamicPropertyRegistry.add("spring.elasticsearch.username", () -> username);
dynamicPropertyRegistry.add("spring.elasticsearch.password", () -> password);
dynamicPropertyRegistry.add("spring.elasticsearch.uris",
() -> List.of(elasticsearchContainer.getHttpHostAddress()));
}
}
If I choose to manually include the dependency that was removed from Spring Data Elasticsearch (which feels wrong):
<dependency>
<groupId>org.elasticsearch</groupId>
<artifactId>elasticsearch</artifactId>
<version>8.11.3</version>
</dependency>
Then I can still use the old Query types, except for NativeQuery, which has been deprecated.
So if I write my Query with the new API, I can't use Elasticsearchoperations, but if I stick to the old one, I can't use NativeQuery.
My test class with Elasticsearchoperations looks something like this:
class MyIndexIntegrationTest
implements ElasticsearchIntegrationTest {
@Autowired
private Elasticsearchoperations elasticsearchOperations;
@Test
void testSearchMyIndex() {
final String matchThis = "hello";
final Query field1Query = new NativeSearchQueryBuilder() // Removed in Spring Data 5
.withQuery(QueryBuilders.matchQuery("field1",
matchThis))
.build();
final List<MyIndex> actualIndexed = elasticsearchOperations
.search(field1Query, MyIndex.class)
.stream()
.map(SearchHit::getContent)
.toList();
// assertions etc
}
}
This worked in the previous version of Spring Data Elasticsearch, but now NativeSearchQueryBuilder has been removed. I could potentially get around it if I can find a way to convert matchQuery to a standard Query that Elasticsearchoperations.search can take as its first argument, but that doesn't seem to be possible, and it seems wrong to be relying on an old API (I don't even know if Elasticsearch 8 containers will work with that API).
So what should I do? How can I run a search using a deprecated NativeQuery? Or alternatively, how can I make a search with the new QueryBuilders from the new API?
The "Connection is closed" error indeed came from SSL enforcement. I was able to get around that by adding this to my DynamicPropertySource:
elasticsearchContainer.getEnvMap().put("xpack.security.enabled", "false");
As for Elasticsearchoperations, I was never able to get it to work with the latest API, but I found I could completely do without it by adding methods to my ElasticsearchRepository interface that do the searches I needed. In the example above it would be:
@Repository
public interface MyIndexRepository extends ElasticsearchRepository<MyIndex, UUID> {
List<MyIndex> findByField1(String field1);
}
And the test class:
final List<MyIndex> actualIndexed = myIndexRepository.findByField1("hello");
This worked both in retaining the original API's Query object, by manually including the org.elasticsearch.elasticsearch dependency (even up to 8.11.2), or indeed by using the new Query API from co.elastic.clients.elasticsearch._types.query_dsl.QueryBuilders