djangopython-2.7amazon-web-serviceselasticsearchdjango-haystack

Connection error while indexing in Elastic search in aws


I want to use elastic search from aws for my usage. Following code works totally fine in my local elastic search but always give error when trying to connect to aws elasticsearch service. I am using python 2.7, django 1.10 and elastic search 5.1.1. Following is the error

ConnectionError(HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //search-test-abc-jlpfzhi64qcrhhqxycov5rzgcq.ap-south-1.es.amazonaws.com/:443/test-index/tweet/1 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e2e5e2090>: Failed to establish a new connection: [Errno -2] Name or service not known',))) caused by: ConnectionError(HTTPSConnectionPool(host='https', port=443): Max retries exceeded with url: //search-test-abc-jlpfzhi64qcrhhqxycov5rzgcq.ap-south-1.es.amazonaws.com/:443/test-index/tweet/1 (Caused by NewConnectionError('<urllib3.connection.VerifiedHTTPSConnection object at 0x7f1e2e5e2090>: Failed to establish a new connection: [Errno -2] Name or service not known',)))

Also, here is the code that i am using

host = AWS_ELASTIC_SEARCH_URL
awsauth = AWS4Auth(AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_ELASTIC_SEARCH_REGION, 'es')

es = Elasticsearch(
    hosts=[{'host': host, 'port': 443}],
    http_auth=awsauth,
    use_ssl=True,
    verify_certs=True,
    connection_class=elasticsearch.RequestsHttpConnection
)

doc = {
'author': 'kimchy',
'text': 'Elasticsearch: cool. bonsai cool.',
'timestamp': datetime.now(),
}
res = es.index(index="test-index", doc_type='tweet', id=1, body=doc)

It is giving error in last line. Also I have made full access to the elastic search url.


Solution

  • I finally figured it out. In my case, I was writing host url as "https://example.com/", but it should be given as "example.com" only. Took a lot of while to hit this in my mind. Following is my working code to connect to aws ElasticSearch (5.1) by using python 2.7 and django 1.9.

    def bulk_indexing():
    
         host = 'example.com'  #not https://example.com"
         awsauth = AWS4Auth('access key', 'secret', region, 'es')
         es = Elasticsearch(
             hosts=[{'host': host, 'port': 443}],
             http_auth=awsauth,
             use_ssl=True,
             verify_certs=True,
             connection_class=RequestsHttpConnection
         )
         payload = {'abc' : 'def'}
         es.index('abc-index', 'doc', payload)