How to fix "elasticsearch.exceptions.AuthenticationException: ... missing authentication token for REST request"?

Question

I am working on using an ElasticSearch database to store data I am pulling from online. However, when I try to index the data in the database I receive an error.

Here is my code for creating and indexing the data:

es = Elasticsearch()

es.index(index='weather', doc_type='data', body=doc)

However when I run this program, the second of those lines causes an error, here is the complete traceback:

Traceback (most recent call last):
File "weatherScraper.py", line 79, in <module>
  main()
File "weatherScraper.py", line 73, in main
  es.index(index='weather', doc_type='data', body=doc)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/client/utils.py", line 73, in _wrapped
  return func(*args, params=params, **kwargs)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/client/__init__.py", line 298, in index
  _make_path(index, doc_type, id), params=params, body=body)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/transport.py", line 312, in perform_request
  status, headers, data = connection.perform_request(method, url, params, body, ignore=ignore, timeout=timeout)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/connection/http_urllib3.py", line 128, in perform_request
  self._raise_error(response.status, raw_data)
File "/home/alec/.local/lib/python2.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
  raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.AuthenticationException: TransportError(401, u'security_exception', u'missing authentication token for REST request [/weather/data]')

Answer 1

''missing authentication token' means you need to authenticate before you can talk to this Elasticsearch instance. To index documents, the user must have write access. You can include a username and password in a URL like this: http://user:password@hostname:port

For example, in a shell:

export ES_ENDPOINT="http://usernameWithWriteAccess:password@localhost:9200"

Then in python:

es = Elasticsearch(os.environ['ES_ENDPOINT'])

Answer 2

The HTTP basic auth can be passed to a http_auth parameter when creating the ElasticSearch client:

client = Elasticsearch(
    hosts=['localhost:5000'],
    http_auth=('username', 'password'),
)
s = Search(using=client, index='something')

This assumes you are using the underlying Urllib3HttpConnection transport class which has the http_auth parameter.

class elasticsearch.connection.Urllib3HttpConnection(host='localhost', 
                                                     http_auth=None, 
                                                     ...,
                                                     **kwargs) 

Default connection class using the urllib3 library and the http protocol.

Parameters:

• http_auth – optional http auth information as either ‘:’ separated string or a tuple

For SSL and other params to authentication, see the SSL and Authentication section of the docs:

from ssl import create_default_context

context = create_default_context(cafile="path/to/cert.pem")
es = Elasticsearch(
    ['localhost', 'otherhost'],
    http_auth=('user', 'secret'),
    scheme="https",
    port=443,
    ssl_context=context,
)