I’m throwing a query that counts rows from MongoDB through ODBC as follows:
select count(*) from (select * from "local.drywall"."accounts")
And I get below error.
HY000 1040 [Dremio][Connector] (1040) Dremio failed to execute the query: select count( *) from ( select * from "local.drywall"."accounts") as count_source
[30029]Query execution error. Details:[
SYSTEM ERROR: MongoCommandException: Command failed with error 16403: '$projection requires at least one output field' on server 127.0.0.1:27017. The full response is { "ok" : 0.0, "errmsg" : "$projection requires at least one output field", "code" : 16403 }
SqlOperatorImpl MONGO_SUB_SCAN
Location 0:0:2
SqlOperatorImpl MON
[RODBC] ERROR: Could not SQLExecDirect 'select count(*) from (select * from "local.drywall"."accounts") as count_source'
However, if I run same query from Dremio UI, it works fine.
Did I miss anything?
Dremio
And I’m using ODBC which is downloaded from here.
https://download.dremio.com/odbc-driver/1.3.11.1034/DremioODBC-1.3.11.1034.dmg
Yes, it is querying the MongoDB physical data set.
Actually, It seems it only works for first preview on Dremio UI. If I change the button from Preview To Run, it fails.
And I’m seeing this in server.json file.
{"timestamp":"2017-10-09 09:49:10,208","host":"Hidetakas-MacBook-Pro.local","thread":"e0 - 26245779-540c-d740-1bcd-b095bae0e400:frag:0:0","levelName":"ERROR","levelValue":40000,"logger":"com.dremio.sabot.driver.SmartOp","message":"MongoCommandException: Command failed with error 16403: '$projection requires at least one output field' on server 127.0.0.1:27017. The full response is { \"ok\" : 0.0, \"errmsg\" : \"$projection requires at least one output field\", \"code\" : 16403 }","stack_trace":"com.dremio.common.exceptions.UserException: MongoCommandException: Command failed with error 16403: '$projection requires at least one output field' on server 127.0.0.1:27017. The full response is { \"ok\" : 0.0, \"errmsg\" : \"$projection requires at least one output field\", \"code\" : 16403 }\n\tat com.dremio.common.exceptions.UserException$Builder.build(UserException.java:648) ~[dremio-common-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.SmartOp.contextualize(SmartOp.java:125) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.SmartOp$SmartProducer.setup(SmartOp.java:556) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.Pipe$SetupVisitor.visitProducer(Pipe.java:79) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.Pipe$SetupVisitor.visitProducer(Pipe.java:63) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.SmartOp$SmartProducer.accept(SmartOp.java:522) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.StraightPipe.setup(StraightPipe.java:102) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.StraightPipe.setup(StraightPipe.java:102) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.StraightPipe.setup(StraightPipe.java:102) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.StraightPipe.setup(StraightPipe.java:102) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.Pipeline.setup(Pipeline.java:58) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution(FragmentExecutor.java:331) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:227) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.exec.fragment.FragmentExecutor.access$800(FragmentExecutor.java:83) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run(FragmentExecutor.java:577) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.task.AsyncTaskWrapper.run(AsyncTaskWrapper.java:92) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.task.slicing.SlicingThread.run(SlicingThread.java:71) [dremio-extra-sabot-scheduler-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\nCaused by: com.mongodb.MongoCommandException: Command failed with error 16403: '$projection requires at least one output field' on server 127.0.0.1:27017. The full response is { \"ok\" : 0.0, \"errmsg\" : \"$projection requires at least one output field\", \"code\" : 16403 }\n\tat com.mongodb.connection.ProtocolHelper.getCommandFailureException(ProtocolHelper.java:77) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.CommandProtocol.createCommandResult(CommandProtocol.java:140) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.CommandProtocol.receiveMessage(CommandProtocol.java:131) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.CommandProtocol.execute(CommandProtocol.java:84) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.DefaultServer$DefaultServerProtocolExecutor.execute(DefaultServer.java:155) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.DefaultServerConnection.executeProtocol(DefaultServerConnection.java:219) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.connection.DefaultServerConnection.command(DefaultServerConnection.java:146) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:182) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.CommandOperationHelper.executeWrappedCommandProtocol(CommandOperationHelper.java:166) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.AggregateOperation$1.call(AggregateOperation.java:199) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.AggregateOperation$1.call(AggregateOperation.java:196) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.OperationHelper.withConnectionSource(OperationHelper.java:195) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.OperationHelper.withConnection(OperationHelper.java:168) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.AggregateOperation.execute(AggregateOperation.java:196) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.operation.AggregateOperation.execute(AggregateOperation.java:62) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.Mongo.execute(Mongo.java:738) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.Mongo$2.execute(Mongo.java:725) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.OperationIterable.iterator(OperationIterable.java:47) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.mongodb.AggregateIterableImpl.iterator(AggregateIterableImpl.java:92) ~[mongo-java-driver-3.0.2.jar:na]\n\tat com.dremio.plugins.mongo.planning.MongoPipeline.getCursor(MongoPipeline.java:258) ~[dremio-extra-plugin-mongo-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.plugins.mongo.execution.MongoRecordReader.next(MongoRecordReader.java:162) ~[dremio-extra-plugin-mongo-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.plugins.mongo.execution.MongoRecordReader.setup(MongoRecordReader.java:147) ~[dremio-extra-plugin-mongo-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.exec.store.CoercionReader.setup(CoercionReader.java:109) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:178) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:174) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat java.security.AccessController.doPrivileged(Native Method) ~[na:1.8.0_144]\n\tat javax.security.auth.Subject.doAs(Subject.java:422) ~[na:1.8.0_144]\n\tat org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) ~[hadoop-common-2.8.0.jar:na]\n\tat com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser(ScanOperator.java:174) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.op.scan.ScanOperator.setupReader(ScanOperator.java:166) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.op.scan.ScanOperator.setup(ScanOperator.java:147) ~[dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\tat com.dremio.sabot.driver.SmartOp$SmartProducer.setup(SmartOp.java:552) [dremio-sabot-kernel-1.2.1-201710030121530889-8e49316.jar:1.2.1-201710030121530889-8e49316]\n\t... 14 common frames omitted\n"}
Hi,
Wondering if you can download the profile from the failed job and send it to us?
If you click on jobs and on the left hand side click on the actual job itself, on the right pane of the window, scroll to the bottom and “Download Profile”
Thanks,
Hi Balaji,
Shared with Dropbox
Hi,
Thanks for uploading the profile. This is a known issue which we are addressing in our upcoming release. For now you can use the _id field in the count, like count(_id), only restriction is that there cannot be any NULL values in the _id field
Thanks,
