"Unexpected error occurred" (NullPointerException) when querying TSV file on Hadoop

#1

I am trying to import a folder of tab separated value files on a Hadoop box via Dremio running stand-alone with default settings on a Ubuntu desktop. The TSV files are compressed via bz2 and have no header line. I have selected Text (delimited) format and Tab and LF as the field delimiter and line delimiter respectively. Inside ./log/server.log I have the following:

2017-07-24 16:55:16,265 [FABRIC-rpc-event-queue] INFO query.logger - {“queryId”:“2689e7a3-73a0-214f-0866-711cfd14f000”,“schema”:"",“queryText”:“select * from table(Hadoop.[pathToTheFolder] (type => ‘text’, fieldDelimiter => ‘,’, comment => ‘#’, “escape” => '”’, quote => ‘"’, lineDelimiter => ‘\n’, extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true))",“start”:1500911708371,“finish”:1500911716265,“outcome”:“FAILED”,“username”:“dremio”,“commandDescription”:“execute; query”}
2017-07-24 16:55:16,270 [FABRIC-rpc-event-queue] WARN c.d.exec.work.foreman.AttemptManager - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).
2017-07-24 16:55:18,063 [FABRIC-rpc-event-queue] INFO c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 2689e79f-c7ea-4b07-6840-4bf3577cb600:0:0
2017-07-24 16:55:18,200 [e0 - 2689e79f-c7ea-4b07-6840-4bf3577cb600:frag:0:0] ERROR com.dremio.sabot.driver.SmartOp - NullPointerException
com.dremio.common.exceptions.UserException: NullPointerException
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:622) ~[dremio-common-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp.contextualize(SmartOp.java:108) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp$SmartProducer.outputData(SmartOp.java:495) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.StraightPipe.pump(StraightPipe.java:56) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.Pipeline.doPump(Pipeline.java:82) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.Pipeline.pumpOnce(Pipeline.java:72) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run(FragmentExecutor.java:279) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run(FragmentExecutor.java:275) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_131]
at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_131]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) [hadoop-common-2.8.0.jar:na]
at com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:238) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor.access$800(FragmentExecutor.java:81) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run(FragmentExecutor.java:567) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.task.AsyncTaskWrapper.run(AsyncTaskWrapper.java:91) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.task.slicing.SlicingThread.run(SlicingThread.java:71) [dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
Caused by: java.lang.NullPointerException: null
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.readFirstLineForColumnNames(CompliantTextRecordReader.java:174) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.generateColumnNames(CompliantTextRecordReader.java:230) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.setup(CompliantTextRecordReader.java:124) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:179) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:175) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_131]
at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_131]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) [hadoop-common-2.8.0.jar:na]
at com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser(ScanOperator.java:175) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator.setupReader(ScanOperator.java:167) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator.outputData(ScanOperator.java:218) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp$SmartProducer.outputData(SmartOp.java:493) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
… 13 common frames omitted
2017-07-24 16:55:18,205 [FABRIC-rpc-event-queue] INFO c.d.exec.work.foreman.QueryManager - Fragment 2689e79f-c7ea-4b07-6840-4bf3577cb600:0:0 failed, cancelling remaining fragments.
2017-07-24 16:55:18,206 [FABRIC-rpc-event-queue] INFO query.logger - {“queryId”:“2689e79f-c7ea-4b07-6840-4bf3577cb600”,“schema”:"",“queryText”:“select * from table(Hadoop.[pathToTheFolder] (type => ‘text’, fieldDelimiter => ‘\t’, comment => ‘#’, “escape” => '”’, quote => ‘"’, lineDelimiter => ‘\n’, extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true))",“start”:1500911711245,“finish”:1500911718206,“outcome”:“FAILED”,“username”:“dremio”,“commandDescription”:“execute; query”}
2017-07-24 16:55:18,217 [FABRIC-rpc-event-queue] WARN c.d.exec.work.foreman.AttemptManager - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).

Meanwhile I have in the error in the jobs UI query profile:

SYSTEM ERROR: NullPointerException

SqlOperatorImpl TEXT_SUB_SCAN
Location 0:0:6
Fragment 0:0

[Error Id: 299dd914-8775-4b28-8d37-a1c7fd76d572 on PC987:31010]

(java.lang.NullPointerException) null
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.readFirstLineForColumnNames():174
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.generateColumnNames():230
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.setup():124
com.dremio.sabot.op.scan.ScanOperator$1.run():179
com.dremio.sabot.op.scan.ScanOperator$1.run():175
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1807
com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser():175
com.dremio.sabot.op.scan.ScanOperator.setupReader():167
com.dremio.sabot.op.scan.ScanOperator.outputData():218
com.dremio.sabot.driver.SmartOp$SmartProducer.outputData():493
com.dremio.sabot.driver.StraightPipe.pump():56
com.dremio.sabot.driver.Pipeline.doPump():82
com.dremio.sabot.driver.Pipeline.pumpOnce():72
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():279
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():275
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1807
com.dremio.sabot.exec.fragment.FragmentExecutor.run():238
com.dremio.sabot.exec.fragment.FragmentExecutor.access$800():81
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():567
com.dremio.sabot.task.AsyncTaskWrapper.run():91
com.dremio.sabot.task.slicing.SlicingThread.run():71

SqlOperatorImpl TEXT_SUB_SCAN
Location 0:0:6
Fragment 0:0

com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:174)
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:230)
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:124)
com.dremio.sabot.op.scan.ScanOperator$1(ScanOperator.java:179)
com.dremio.sabot.op.scan.ScanOperator$1(ScanOperator.java:175)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1807)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:175)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:167)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:218)
com.dremio.sabot.driver.SmartOp$SmartProducer(SmartOp.java:493)
com.dremio.sabot.driver.StraightPipe(StraightPipe.java:56)
com.dremio.sabot.driver.Pipeline(Pipeline.java:82)
com.dremio.sabot.driver.Pipeline(Pipeline.java:72)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:279)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:275)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1807)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:238)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:81)
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl(FragmentExecutor.java:567)
com.dremio.sabot.task.AsyncTaskWrapper(AsyncTaskWrapper.java:91)
com.dremio.sabot.task.slicing.SlicingThread(SlicingThread.java:71)

Any ideas what could be causing this?

#2

Looks like you have at least one file that has only line delimiter “\n” in it - no other data. Or first line is empty in the file. Could you take a look and if possible remove that file?

#3

I don’t think this is the cause of your issue but looks like the field delimiter is set to comma “,” and not the Tab character. Could you confirm you set Tab as the field delimiter and re-share any exceptions you get?

#4

I believe I do have some TSVs in the folder which are empty…(or at least they are empty when decompressed from .bz2) though there aren’t any with just a newline I don’t think. It would be nice for Dremio to deal with this situation with an option…

Here is the full log when I change to Tab and \n. For some reason it throws the error in between selecting Tab in the UI and selecting \n (instead of the default \r\n)…

2017-07-25 09:40:47,428 [FABRIC-rpc-event-queue] INFO query.logger - {“queryId”:“2688fbf8-be7c-e421-b481-c1c777eeb100”,“schema”:"",“queryText”:“select * from table(Hadoop.[pathToTheFolder] (type => ‘text’, fieldDelimiter => ‘\t’, comment => ‘#’, “escape” => '”’, quote => ‘"’, lineDelimiter => ‘\r\n’, extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true))",“start”:1500972038279,“finish”:1500972047428,“outcome”:“FAILED”,“username”:“dremio”,“commandDescription”:“execute; query”}
2017-07-25 09:40:47,435 [FABRIC-rpc-event-queue] WARN c.d.exec.work.foreman.AttemptManager - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).
2017-07-25 09:40:48,194 [FABRIC-rpc-event-queue] INFO c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 2688fbf7-57f5-ca4f-eb3d-5a593372e500:0:0
2017-07-25 09:40:48,361 [e0 - 2688fbf7-57f5-ca4f-eb3d-5a593372e500:frag:0:0] ERROR com.dremio.sabot.driver.SmartOp - NullPointerException
com.dremio.common.exceptions.UserException: NullPointerException
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:622) ~[dremio-common-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp.contextualize(SmartOp.java:108) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp$SmartProducer.outputData(SmartOp.java:495) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.StraightPipe.pump(StraightPipe.java:56) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.Pipeline.doPump(Pipeline.java:82) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.Pipeline.pumpOnce(Pipeline.java:72) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run(FragmentExecutor.java:279) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run(FragmentExecutor.java:275) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_131]
at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_131]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) [hadoop-common-2.8.0.jar:na]
at com.dremio.sabot.exec.fragment.FragmentExecutor.run(FragmentExecutor.java:238) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor.access$800(FragmentExecutor.java:81) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run(FragmentExecutor.java:567) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.task.AsyncTaskWrapper.run(AsyncTaskWrapper.java:91) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.task.slicing.SlicingThread.run(SlicingThread.java:71) [dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
Caused by: java.lang.NullPointerException: null
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.readFirstLineForColumnNames(CompliantTextRecordReader.java:174) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.generateColumnNames(CompliantTextRecordReader.java:230) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.setup(CompliantTextRecordReader.java:124) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:179) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator$1.run(ScanOperator.java:175) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.security.AccessController.doPrivileged(Native Method) [na:1.8.0_131]
at javax.security.auth.Subject.doAs(Subject.java:422) [na:1.8.0_131]
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807) [hadoop-common-2.8.0.jar:na]
at com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser(ScanOperator.java:175) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator.setupReader(ScanOperator.java:167) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.op.scan.ScanOperator.outputData(ScanOperator.java:218) ~[dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.sabot.driver.SmartOp$SmartProducer.outputData(SmartOp.java:493) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
… 13 common frames omitted
2017-07-25 09:40:48,366 [FABRIC-rpc-event-queue] INFO c.d.exec.work.foreman.QueryManager - Fragment 2688fbf7-57f5-ca4f-eb3d-5a593372e500:0:0 failed, cancelling remaining fragments.
2017-07-25 09:40:48,368 [FABRIC-rpc-event-queue] INFO query.logger - {“queryId”:“2688fbf7-57f5-ca4f-eb3d-5a593372e500”,“schema”:"",“queryText”:“select * from table(Hadoop.[pathToTheFolder] (type => ‘text’, fieldDelimiter => ‘\t’, comment => ‘#’, “escape” => '”’, quote => ‘"’, lineDelimiter => ‘\n’, extractHeader => false, skipFirstLine => false, autoGenerateColumnNames => true, trimHeader => true))",“start”:1500972040234,“finish”:1500972048367,“outcome”:“FAILED”,“username”:“dremio”,“commandDescription”:“execute; query”}
2017-07-25 09:40:48,379 [FABRIC-rpc-event-queue] WARN c.d.exec.work.foreman.AttemptManager - Dropping request to move to COMPLETED state as query is already at FAILED state (which is terminal).

And the verbose error message in the Jobs UI is:

SYSTEM ERROR: NullPointerException

SqlOperatorImpl TEXT_SUB_SCAN
Location 0:0:6
Fragment 0:0

[Error Id: 299dd914-8775-4b28-8d37-a1c7fd76d572 on PC987:31010]

(java.lang.NullPointerException) null
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.readFirstLineForColumnNames():174
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.generateColumnNames():230
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader.setup():124
com.dremio.sabot.op.scan.ScanOperator$1.run():179
com.dremio.sabot.op.scan.ScanOperator$1.run():175
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1807
com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser():175
com.dremio.sabot.op.scan.ScanOperator.setupReader():167
com.dremio.sabot.op.scan.ScanOperator.outputData():218
com.dremio.sabot.driver.SmartOp$SmartProducer.outputData():493
com.dremio.sabot.driver.StraightPipe.pump():56
com.dremio.sabot.driver.Pipeline.doPump():82
com.dremio.sabot.driver.Pipeline.pumpOnce():72
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():279
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():275
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1807
com.dremio.sabot.exec.fragment.FragmentExecutor.run():238
com.dremio.sabot.exec.fragment.FragmentExecutor.access$800():81
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():567
com.dremio.sabot.task.AsyncTaskWrapper.run():91
com.dremio.sabot.task.slicing.SlicingThread.run():71

SqlOperatorImpl TEXT_SUB_SCAN
Location 0:0:6
Fragment 0:0

com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:174)
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:230)
com.dremio.exec.store.easy.text.compliant.CompliantTextRecordReader(CompliantTextRecordReader.java:124)
com.dremio.sabot.op.scan.ScanOperator$1(ScanOperator.java:179)
com.dremio.sabot.op.scan.ScanOperator$1(ScanOperator.java:175)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1807)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:175)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:167)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:218)
com.dremio.sabot.driver.SmartOp$SmartProducer(SmartOp.java:493)
com.dremio.sabot.driver.StraightPipe(StraightPipe.java:56)
com.dremio.sabot.driver.Pipeline(Pipeline.java:82)
com.dremio.sabot.driver.Pipeline(Pipeline.java:72)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:279)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:275)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1807)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:238)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:81)
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl(FragmentExecutor.java:567)
com.dremio.sabot.task.AsyncTaskWrapper(AsyncTaskWrapper.java:91)
com.dremio.sabot.task.slicing.SlicingThread(SlicingThread.java:71)

#5

And when I try again on a folder that I don’t think has any empty files everything works, so I guess that was the issue.

#6

Yes, we are looking into the issue where you have first line empty or empty file altogether. Will update you on the progress.