Kibana fails to pick up date from elasticsearch when I include the hour and minute

Question

I'm really stuggling to get this specific time format into elasticsearch so I can graph it in Kibana. I cannot change this format. My elasticsearch data and mapping is in this format:

STEP 1: Setup Mapping

PUT http://<>.com:5101/myindex6/_doc/1

{
  "mappings": {
    "properties": {
      "date": {
        "type":   "date",
        "format": "HH:mm yyyy-MM-dd"
      },
        "data": {
        "type":   "integer"
      }
    }
  }
}

Step 2: Add Data

PUT http://<>.com:5101/myindex6

{
  "test" : [ {
    "data" : "119050300",
    "date" : "10:00 2019-06-03"
  } ]
}

In Kibana it wont find this as a date and wont allow me to map it as one. However, if I remove the time aspect and use the date, and do this instead, it works fine:

Data

{
  "test" : [ {
    "data" : "119050300",
    "date" : "2019-06-03"
  } ]
}

Map

{
  "mappings": {
    "properties": {
      "date": {
        "type":   "date",
        "format": "yyyy-MM-dd"
      },
        "data": {
        "type":   "integer"
      }
    }
  }
}

Can someone please tell me how to include the time and not have it break, so I can filter on time in kibana.

Answer 1

There is the difference between the mapping and the structure document that you are indexing. Also the endpoints you are using seems be swapped. Follow the steps below:

1. Create index

PUT myindex6
{
  "mappings": {
    "properties": {
      "date": {
        "type":   "date",
        "format": "HH:mm yyyy-MM-dd"
      },
        "data": {
        "type":   "integer"
      }
    }
  }
}

2. Index document

POST myindex6/_doc/1
{
  "data": "119050300",
  "date": "10:00 2019-06-03"
}

Notice the endpoints used to create index and then to index a document. Also notice the structure of document is in line with the mapping. In your case you are indexing a document with a test field which is an array of object with fields data and date. This structure doesn't match to the mapping created in step 1.

Answer 2

As already pointed in the above answer, the data which you are indexing doesn't match with the mapping created. What you need to do is update your mapping with test field as nested.

{
  "mappings": {
    "properties": {
        "test": {
            "type": "nested",
            "properties": {
                "date": {
                    "type": "date",
                    "format": "HH:mm yyyy-MM-dd"
                },
                "data": {
                    "type": "integer"
                }
            }
        }
    }
}}

And then re-index your data. The date field will appear in Kibana after that.