elasticsearchelasticsearch-bulk-api

Setting Time To Live (TTL) from Java - sample requested


EDIT: This is basically what I want to do, only in Java

Using ElasticSearch, we add documents to an index bypassing IndexRequest items to a BulkRequestBuilder.

I would like for the documents to be dropped from the index after some time has passed (time to live/ttl)

This can be done either by setting a default for the index, or on a per-document basis. Either approach is fine by me.

The code below is an attempt to do it per document. It does not work. I think it's because TTL is not enabled for the index. Either show me what Java code I need to add to enable TTL so the code below works, or show me different code that enables TTL + sets default TTL value for the index in Java I know how to do it from the REST API but I need to do it from Java code, if at all possible.

    logger.debug("Indexing record ({}): {}", id, map);
    final IndexRequest indexRequest = new IndexRequest(_indexName, _documentType, id);
    final long debug = indexRequest.ttl();
    if (_ttl > 0) {
        indexRequest.ttl(_ttl);
        System.out.println("Setting TTL to " + _ttl);
        System.out.println("IndexRequest now has ttl of " + indexRequest.ttl());
    }
    indexRequest.source(map);
    indexRequest.operationThreaded(false);
    bulkRequestBuilder.add(indexRequest);
}

// execute and block until done.
BulkResponse response;
try {
    response = bulkRequestBuilder.execute().actionGet();

Later I check in my unit test by polling this method, but the document count never goes down.

public long getDocumentCount() throws Exception {
    Client client = getClient();
    try {
        client.admin().indices().refresh(new RefreshRequest(INDEX_NAME)).actionGet();

        ActionFuture<CountResponse> response = client.count(new CountRequest(INDEX_NAME).types(DOCUMENT_TYPE));
        CountResponse countResponse = response.get();
        return countResponse.getCount();
    } finally {
        client.close();
    }
}

Solution

  • After a LONG day of googling and writing test programs, I came up with a working example of how to use ttl and basic index/object creation from the Java API. Frankly most of the examples in the docs are trivial, and some JavaDoc and end-to-end examples would go a LONG way to help those of us who are using the non-REST interfaces.

    Ah well.

    Code here: Adding mapping to a type from Java - how do I do it?