elasticsearchelasticsearch-bulk-api

Post request for bulk API is giving status code of 406, How to resolve it?


I am using Elastic search 6.1 version My data is appending correctly and I am adding '\n' at the end of the request.

My code is as follows:

def insert_in_bulk(self, filee, rtype):
    U = urljoin(self.args.host, "/_bulk")
    body = []
    f = open(filee)
    for line in f:
        action = {
                'index' :{
                '_index' : self.args.index,
                '_type' : rtype,
                  }
                }
        item = {
            'word' : line.strip()
            }

        body.append(json.dumps(action))
        body.append(json.dumps(item))

    f.close()

    body = '\n'.join(body)+'\n'
    success = False
    try:
        r = requests.post(U, data=body)
        self.log.info("after request")
        if r.status_code == 200:
            success = True
        r = r.json()
        self.log.info("inserted %s items of type = %s", self.args.index , rtype)
    except (SystemExit, KeyboardInterrupt): raise
    except:
          self.log.exception("during bulk index")

    if not success:
             self.log.error("failed to index records of type = %s", rtype)

I am using the python to connect to elastic search.

I got the answer from this link Bulk index document from JSON file into ElasticSearch

I have to pass the header to the request as application/x-ndjson.


Solution

  • Though it is quite some time question is asked, but i want to give a solution that has worked for me in most case,

    def insert_in_bulk(self, filee, rtype):
        U = urljoin(self.args.host, "/_bulk")
        body = []
        f = open(filee)
        for line in f:
            action = {
                    'index' :{
                    '_index' : self.args.index,
                    '_type' : rtype,
                      }
                    }
            item = {
                'word' : line.strip()
                }
    
            body.append(json.dumps(action))
            body.append(json.dumps(item))
    
        f.close()
    
    
        payload = ""
        for l in body:
            payload = payload + f"{l} \n"
        data = payload.encode('utf-8')
    
        r = requests.post(U, data=data, headers={"Content-Type": "application/x-ndjson"})
        print(r.text)