Hello Team,
I have created the DREMIO HA Cluster with total 4 nodes, 2 Coordinator and 2 executor nodes.
I have added the HIVE data source in dremio and executing the query from tableau dashboard. But I am not getting stable performance.
Sometime the same query jobs take 10 to 20 seconds to complete but suddenly after some time it took minimum 3 min or more to complete.
We have configured the NAS mount point for metadata storage which is stored across all nodes, also we have configured the HDFS as distributed storage but we as of now are not using reflection.
We have checked the server.log file and got multiple error log when query take more time but the same error logs are generating at time when jobs get completed within 10to 20 seconds.
I have attached the both jobs profile generated after completion. and below are the some error logs in our server.log file.
“2020-07-06 13:21:05,315 [out-of-band-observer] INFO query.logger - {“queryId”:“20fcdabe-19bb-f71d-5520-ec4874836400”,“queryText”:“SELECT “SUBCOL” AS “COL”\nFROM (\n SELECT 1 AS “SUBCOL”\n) “SUBQUERY”\nGROUP BY 2”,“start”:1594041665312,“finish”:1594041665313,“outcome”:“FAILED”,“username”:“dremio_tableau”}
2020-07-06 13:21:05,326 [20fcdabe-3271-c1cb-b5c3-de05bee57400/0:foreman-planning] INFO c.d.exec.planner.sql.SqlConverter - User Error Occurred [ErrorId: a7b22ed7-1f24-44f7-b2e2-db786f309d8f]
com.dremio.common.exceptions.UserException: Failure parsing the query.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:802)
at com.dremio.exec.planner.sql.SqlConverter.parseMultipleStatementsImpl(SqlConverter.java:191)
at com.dremio.exec.planner.sql.SqlConverter.parseSingleStatementImpl(SqlConverter.java:201)
at com.dremio.exec.planner.sql.SqlConverter.parse(SqlConverter.java:217)
at com.dremio.exec.planner.sql.handlers.commands.CommandCreator.getSqlCommand(CommandCreator.java:255)
at com.dremio.exec.planner.sql.handlers.commands.CommandCreator.toCommand(CommandCreator.java:158)
at com.dremio.exec.work.foreman.AttemptManager.plan(AttemptManager.java:386)
at com.dremio.exec.work.foreman.AttemptManager.lambda$run$1(AttemptManager.java:310)
at com.dremio.service.commandpool.CommandWrapper.run(CommandWrapper.java:62)
at com.dremio.common.concurrent.ContextMigratingExecutorService.lambda$decorate$3(ContextMigratingExecutorService.java:192)
at com.dremio.common.concurrent.ContextMigratingExecutorService$ComparableRunnable.run(ContextMigratingExecutorService.java:174)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: org.apache.calcite.sql.parser.SqlParseException: Encountered “INTO” at line 2, column 1.”
error 2- Caused By (org.apache.calcite.sql.validate.SqlValidatorException) Table ‘json’ not found
error 3 - foreman-planning failed com.dremio.common.exceptions.UserException: Table
Could you please help me to resolve this issue
Thank you.