# Elasticsearch "Failed to parse date time value" bug

**URL:** <https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836>\
**Category:** Uncategorized\
**Created:** [September 13, 2018, 2:07pm UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836 "2018-09-13T14:07:17Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![nathanemmison](https://avatars.discourse-cdn.com/v4/letter/n/edb3f5/32.png) [@nathanemmison](https://community.dremio.com/u/nathanemmison)\
**Post date:** [September 13, 2018, 2:07pm UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/1 "2018-09-13T14:07:17Z")

</div>

Hello!

We have our ES cluster with an index and one of the fields is mapped as such:

“fieldname”: {  
“type”: “date”,  
“format”: “dd-MM-yyyy”  
},

When trying to use Dremio (Windows, UI version) it then struggles to connect to the index with the following error (date value has been masked):

_Failed to parse date time value \_\_\_\_\_\_ in field fieldname._

Upon looking at the logs it appears it’s trying parse this as a date AND time field rather than just date.

at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.WriteHolders.getMillis(WriteHolders.java:370) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.WriteHolders.access$100(WriteHolders.java:45) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.WriteHolders$TimestampWriteHolder.write(WriteHolders.java:356) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.WriteHolders$TimestampWriteHolder.writeMap(WriteHolders.java:347) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.FieldReadDefinition.writeMap(FieldReadDefinition.java:132) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:328) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:324) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:324) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeToVector(ElasticsearchJsonReader.java:214) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]  
at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.write(ElasticsearchJsonReader.java:182) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]

Is this a bug that needs to be fixed or is there something we can do to fix it?

Thanks

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [September 14, 2018, 3:05am UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/2 "2018-09-14T03:05:10Z")

</div>

Hi @nathanemmison

Can you please provide us with the job profile?

> **[How To Share A Query Profile](https://www.dremio.com/tutorials/share-query-profile-dremio/)**
>
> Tutorial explaining how to share a query profile in Dremio.

Thanks  
@balaji.ramaswamy

---

<div class="post-metadata">

**Author:** ![nathanemmison](https://avatars.discourse-cdn.com/v4/letter/n/edb3f5/32.png) [@nathanemmison](https://community.dremio.com/u/nathanemmison)\
**Post date:** [September 14, 2018, 8:28am UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/3 "2018-09-14T08:28:58Z")

</div>

Hi,

Due to the confidential nature of our data and system I can’t share that.

Is there something specific in the profile that you are looking for?

It definitely appears as though it is trying to run a function called .getMillis() when this is only a date value and not a time.

In addition, this error also comes up even when I’m not selecting this field in a query.

@balaji.ramaswamy

---

<div class="post-metadata">

**Author:** ![anthony](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/anthony/32/670_2.png) [@anthony](https://community.dremio.com/u/anthony)\
**Post date:** [September 14, 2018, 1:18pm UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/4 "2018-09-14T13:18:30Z")

</div>

A query profile helps understand what is being run, what Dremio’s query planner is doing behind the scene, and ultimately a deeper look into the possible error’s stacktrace. Ultimately, it would help us repro the issue internally.

---

<div class="post-metadata">

**Author:** ![nathanemmison](https://avatars.discourse-cdn.com/v4/letter/n/edb3f5/32.png) [@nathanemmison](https://community.dremio.com/u/nathanemmison)\
**Post date:** [September 14, 2018, 1:48pm UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/5 "2018-09-14T13:48:13Z")

</div>

Unfortunately I will not be able to share it.

For job summary it has this:

> No planning phase information to show

> State: FAILED  
> Coordinator: _IP ADDRESS_  
> Threads: 0  
> Planning Time: Still planning

The Threads, Nodes and Operators tables are all empty.

The full stack trace:

```
 2018-09-14 14:40:51,359 [2464459d-766a-f3aa-945c-bc735b7e3c00:foreman] INFO c.d.p.elastic.execution.WriteHolders - User Error Occurred [ErrorId: 858c8969-6591-4af3-b193-50def2bda6d4]
 com.dremio.common.exceptions.UserException: Failed to parse date time value _____in field_______.
    	at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746) ~[dremio-common-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.WriteHolders.getMillis(WriteHolders.java:370) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.WriteHolders.access$100(WriteHolders.java:45) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.WriteHolders$TimestampWriteHolder.write(WriteHolders.java:356) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.WriteHolders$TimestampWriteHolder.writeMap(WriteHolders.java:347) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.FieldReadDefinition.writeMap(FieldReadDefinition.java:132) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:328) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:324) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeDeclaredMap(ElasticsearchJsonReader.java:324) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.writeToVector(ElasticsearchJsonReader.java:214) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchJsonReader.write(ElasticsearchJsonReader.java:182) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.execution.ElasticsearchRecordReader.next(ElasticsearchRecordReader.java:351) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.ElasticTableBuilder.getSampledSchema(ElasticTableBuilder.java:322) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.ElasticTableBuilder.populate(ElasticTableBuilder.java:190) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.ElasticTableBuilder.buildIfNecessary(ElasticTableBuilder.java:161) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.plugins.elastic.ElasticTableBuilder.getDataset(ElasticTableBuilder.java:142) [dremio-elasticsearch-plugin-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.DatasetManager.getTableFromPlugin(DatasetManager.java:298) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.DatasetManager.getTable(DatasetManager.java:191) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.CatalogImpl.getTable(CatalogImpl.java:135) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.DelegatingCatalog.getTable(DelegatingCatalog.java:58) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.CachingCatalog.getTable(CachingCatalog.java:66) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.DremioCatalogReader.getTable(DremioCatalogReader.java:94) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.catalog.DremioCatalogReader.getTable(DremioCatalogReader.java:71) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at org.apache.calcite.sql.validate.EmptyScope.getTableNamespace(EmptyScope.java:76) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.DelegatingScope.getTableNamespace(DelegatingScope.java:197) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.IdentifierNamespace.resolveImpl(IdentifierNamespace.java:102) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.IdentifierNamespace.validateImpl(IdentifierNamespace.java:120) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:943) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:924) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2971) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom(SqlValidatorImpl.java:2956) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect(SqlValidatorImpl.java:3197) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SelectNamespace.validateImpl(SelectNamespace.java:60) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.AbstractNamespace.validate(AbstractNamespace.java:84) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace(SqlValidatorImpl.java:943) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery(SqlValidatorImpl.java:924) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.SqlSelect.validate(SqlSelect.java:226) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression(SqlValidatorImpl.java:899) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at org.apache.calcite.sql.validate.SqlValidatorImpl.validate(SqlValidatorImpl.java:609) [calcite-core-1.16.0-201807312206090258-e1b02e7.jar:1.16.0-201807312206090258-e1b02e7]
    	at com.dremio.exec.planner.sql.SqlConverter.validate(SqlConverter.java:168) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.planner.sql.handlers.PrelTransformer.validateNode(PrelTransformer.java:176) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert(PrelTransformer.java:165) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert(PrelTransformer.java:161) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.planner.sql.handlers.query.NormalHandler.getPlan(NormalHandler.java:43) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan(HandlerToExec.java:69) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:291) [dremio-sabot-kernel-2.1.4-201808302048550610-0981242.jar:2.1.4-201808302048550610-0981242]
    	at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.8.0_181]
    	at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.8.0_181]
    	at java.lang.Thread.run(Unknown Source) [na:1.8.0_181]

```

It seems another user had a similar problem:

> [@Elasticsearch datetime problems](https://community.dremio.com/t/elasticsearch-datetime-problems/1088):
>
> I am using the latest Dremo (2.0.1) on Windows. I am trying to read an index from Elasticsearch. If the date time format/index metadata is defined as follows the Query will fail with the error message (Failed to parse date time value 2017/11/01 00:01:05.908 in field source\_data.will\_fail\_timestamp): will\_fail\_timestamp ": { “format”: “Y/M/d H:m:s.SSS”, “type”: “date”}, If the date time of an Elasticsearch index is defined without the format it will succeed as in: “will\_succeed\_timestamp”: { …

@HokieKev

---

<div class="post-metadata">

**Author:** ![anshal](https://avatars.discourse-cdn.com/v4/letter/a/c0e974/32.png) [@anshal](https://community.dremio.com/u/anshal)\
**Post date:** [November 9, 2018, 5:56am UTC](https://community.dremio.com/t/elasticsearch-failed-to-parse-date-time-value-bug/1836/6 "2018-11-09T05:56:00Z")

</div>

We are also facing this issue .  
We use ES 5.5 with date format YYYY-MM-DD HH:MM:SS  
The error that we get on the UI is Failed to parse date time value 2018-07-25 09:15:51 in field XXX