# Column not found for Elasticsearch as a source **URL:** \ **Category:** Dremio University\ **Created:** [December 29, 2020, 10:51pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787 "2020-12-29T22:51:44Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [December 29, 2020, 10:51pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/1 "2020-12-29T22:51:44Z")
Hello Everyone, We are using 3 nodes Dremio cluster (2 slaves + 1 master) and we have configured Elasticsearch(v6.8) as one of the data sources. we have below index mapping in ES, > { test-alias:{ > index-1:{ > source:{ > type:keyword > } > }, > index-2:{ > source:{ > type:keyword > } > }, > index-2:{ > source:{ > type:keyword > } > } > } while executing **select \* from test-alias** we get proper results and where as sometimes we face below exception, (org.apache.calcite.runtime.CalciteContextException) From line 26, column 19 to line 26, column 24: Column ‘source’ not found in any table sun.reflect.GeneratedConstructorAccessor264.newInstance():-1 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.sql.SqlUtil.newContextException():803 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4470 org.apache.calcite.sql.validate.DelegatingScope.fullyQualify():253 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit():5332 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit():5314 org.apache.calcite.sql.SqlIdentifier.accept():344 org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild():134 org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild():101 org.apache.calcite.sql.SqlOperator.acceptCall():859 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visitScoped():5367 org.apache.calcite.sql.validate.SqlScopedShuttle.visit():50 org.apache.calcite.sql.validate.SqlScopedShuttle.visit():33 org.apache.calcite.sql.SqlCall.accept():138 org.apache.calcite.sql.validate.SqlValidatorImpl.expand():4921 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWhereClause():3659 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3200 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():924 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2971 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2956 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3197 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():924 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2971 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2956 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3197 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.AbstractNamespace.getRowType():115 org.apache.calcite.sql.validate.SelectNamespace.getRowType():30 org.apache.calcite.sql.validate.AbstractNamespace.getType():126 org.apache.calcite.sql.validate.SelectNamespace.getType():30 org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeTypeIfKnown():1538 org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType():1523 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWithItem():3449 org.apache.calcite.sql.validate.WithNamespace.validateImpl():53 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWith():3433 org.apache.calcite.sql.SqlWith.validate():71 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():899 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():609 com.dremio.exec.planner.sql.SqlConverter.validate():199 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateNode():178 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert():167 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert():163 com.dremio.exec.planner.sql.handlers.query.NormalHandler.getPlan():43 com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan():68 com.dremio.exec.work.foreman.AttemptManager.plan():383 com.dremio.exec.work.foreman.AttemptManager.lambda$run$0():292 com.dremio.service.commandpool.CommandWrapper.run():62 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 Caused By (org.apache.calcite.sql.validate.SqlValidatorException) Column ‘source’ not found in any table sun.reflect.GeneratedConstructorAccessor263.newInstance():-1 sun.reflect.DelegatingConstructorAccessorImpl.newInstance():45 java.lang.reflect.Constructor.newInstance():423 org.apache.calcite.runtime.Resources$ExInstWithCause.ex():463 org.apache.calcite.runtime.Resources$ExInst.ex():572 org.apache.calcite.sql.SqlUtil.newContextException():803 org.apache.calcite.sql.SqlUtil.newContextException():788 org.apache.calcite.sql.validate.SqlValidatorImpl.newValidationError():4470 org.apache.calcite.sql.validate.DelegatingScope.fullyQualify():253 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit():5332 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visit():5314 org.apache.calcite.sql.SqlIdentifier.accept():344 org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild():134 org.apache.calcite.sql.util.SqlShuttle$CallCopyingArgHandler.visitChild():101 org.apache.calcite.sql.SqlOperator.acceptCall():859 org.apache.calcite.sql.validate.SqlValidatorImpl$Expander.visitScoped():5367 org.apache.calcite.sql.validate.SqlScopedShuttle.visit():50 org.apache.calcite.sql.validate.SqlScopedShuttle.visit():33 org.apache.calcite.sql.SqlCall.accept():138 org.apache.calcite.sql.validate.SqlValidatorImpl.expand():4921 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWhereClause():3659 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3200 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():924 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2971 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2956 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3197 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateQuery():924 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2971 org.apache.calcite.sql.validate.SqlValidatorImpl.validateFrom():2956 org.apache.calcite.sql.validate.SqlValidatorImpl.validateSelect():3197 org.apache.calcite.sql.validate.SelectNamespace.validateImpl():60 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.AbstractNamespace.getRowType():115 org.apache.calcite.sql.validate.SelectNamespace.getRowType():30 org.apache.calcite.sql.validate.AbstractNamespace.getType():126 org.apache.calcite.sql.validate.SelectNamespace.getType():30 org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeTypeIfKnown():1538 org.apache.calcite.sql.validate.SqlValidatorImpl.getValidatedNodeType():1523 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWithItem():3449 org.apache.calcite.sql.validate.WithNamespace.validateImpl():53 org.apache.calcite.sql.validate.AbstractNamespace.validate():84 org.apache.calcite.sql.validate.SqlValidatorImpl.validateNamespace():943 org.apache.calcite.sql.validate.SqlValidatorImpl.validateWith():3433 org.apache.calcite.sql.SqlWith.validate():71 org.apache.calcite.sql.validate.SqlValidatorImpl.validateScopedExpression():899 org.apache.calcite.sql.validate.SqlValidatorImpl.validate():609 com.dremio.exec.planner.sql.SqlConverter.validate():199 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateNode():178 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert():167 com.dremio.exec.planner.sql.handlers.PrelTransformer.validateAndConvert():163 com.dremio.exec.planner.sql.handlers.query.NormalHandler.getPlan():43 com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan():68 com.dremio.exec.work.foreman.AttemptManager.plan():383 com.dremio.exec.work.foreman.AttemptManager.lambda$run$0():292 com.dremio.service.commandpool.CommandWrapper.run():62 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 on rebooting dremio cluster it works for several attempts and after that, it fails with the same exception. can anyone please help us on this? ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [December 30, 2020, 7:30am UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/2 "2020-12-30T07:30:17Z")
@murtazahzaveri53 Are you doing a run or preview? Can you please share the query profile? > **[How To Share A Query Profile](https://www.dremio.com/tutorials/share-query-profile-dremio/)** > > Tutorial explaining how to share a query profile in Dremio. ---
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [December 30, 2020, 10:11am UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/3 "2020-12-30T10:11:07Z")
Hi Balaji, thanks for your quick response. I am doing a run. and above logs are taken from job profile console. ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [December 30, 2020, 9:56pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/4 "2020-12-30T21:56:34Z")
@murtazahzaveri53 The above snippet does not show the full issue, want to check the profile and see why Dremio says “Column ‘source’ not found in any table”, in particular want to see if metadata found it, which will be under the planning tab Thanks Bali ---
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [January 3, 2021, 11:00pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/5 "2021-01-03T23:00:53Z")
Hi Balaji, Thanks for your quick response. i have found one interesting point here, when we query to individual indices in apache dremio it gets result with all the columns whereas if we query to alias(pointing to multiple indices) it throws this exception. note : all the indices containing in aliases has column ‘source’ in it. ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 4, 2021, 8:20am UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/6 "2021-01-04T08:20:23Z")
Thanks for the update @murtazahzaveri53, do send us the complete profile and we will review and get back to you ---
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [January 4, 2021, 3:25pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/7 "2021-01-04T15:25:46Z")
Hi Balaji, I cant share an actual profile with you as we have few of PHI details in it but I can assure you that all the indices of ES have **source** fields in it. Also, the same dremio query works sometimes without any errors. Under the planning tab, we see below things, **elastic.: Permission\_cache\_hit(0ms)** **elastic.: Permission\_cache\_hit(0ms)** **elastic.: Permission\_cached\_metadata(0ms)** **Non-default options :** [{ kind : LONG, type : SYSTEM, name : store.plugin.max\_metadata\_leaf\_columns num\_val:1800 } ] **Job Summary** State : FAILED co-ordinator : dremio\_node\_url Threads : 0 Command pool wait : 0ms Planning time : still planning Please let me know incase of any additional details is required. Thanks, Murtaza Zaveri ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 5, 2021, 7:59am UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/8 "2021-01-05T07:59:55Z")
@murtazahzaveri53 I believe you that the columns exists, I want to examine the planning information on what Dremio found. Are you able to send the profile to a Dremio email? Thanks Bali ---
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [January 5, 2021, 12:13pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/9 "2021-01-05T12:13:32Z")
Hi Balaji, Kindly find attached dremio profile\_attempt. Many thanks, Murtaza Zaveri[profile\_attempt\_0.zip](https://community.dremio.com/uploads/short-url/zmbmM5TMU1VperI0SDXfQ8xpiVe.zip) (3.8 KB) ---
**Author:** ![murtazahzaveri53](https://avatars.discourse-cdn.com/v4/letter/m/9f8e36/32.png) [@murtazahzaveri53](https://community.dremio.com/u/murtazahzaveri53)\ **Post date:** [January 6, 2021, 9:56pm UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/10 "2021-01-06T21:56:57Z")
Hi Balaji, Any updates on this? Please let me know in case any more details are required. Thanks and regards, Murtaza Zaveri ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 7, 2021, 8:51am UTC](https://community.dremio.com/t/column-not-found-for-elasticsearch-as-a-source/6787/11 "2021-01-07T08:51:11Z")
@murtazahzaveri53 Download the zip file from the Dremio jobs page and send us the zip file as is, this will help us render the profile and see what is going on