We have MaRFs hive partition table, while reading one particular partition or select * from throws below exception
when we create maprfs source and format folder then we are able to read same parquet file/partition without any issue.
Only when MapRFS Hive + Hive Query its gives this arrayindexoutofboound exception. when we copy or re-create particular partition then its working. its looks very strange. please help us find out the issue. [ is issue at MaprFs Hive or dremio code? [ we are using 3.3.1]
SqlOperatorImpl HIVE_SUB_SCAN
Location 7:91:5
Fragment 7:0
com.mapr.fs.jni.MapRClient(MapRClient.java:-2)
com.mapr.fs.MapRClientImpl(MapRClientImpl.java:523)
com.mapr.fs.MapRFileSystem(MapRFileSystem.java:2087)
com.mapr.fs.MapRFileSystem(MapRFileSystem.java:2108)
org.apache.hadoop.fs.FileSystem(FileSystem.java:746)
com.dremio.exec.store.dfs.FileSystemWrapper(FileSystemWrapper.java:1128)
com.dremio.exec.store.parquet.UnifiedParquetReader(UnifiedParquetReader.java:171)
com.dremio.exec.store.parquet.UnifiedParquetReader(UnifiedParquetReader.java:135)
com.dremio.exec.store.hive.exec.FileSplitParquetRecordReader(FileSplitParquetRecordReader.java:219)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:225)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1633)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:224)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:196)
com.dremio.sabot.op.scan.ScanOperator(ScanOperator.java:270)
com.dremio.sabot.driver.SmartOp$SmartProducer(SmartOp.java:521)
com.dremio.sabot.driver.StraightPipe(StraightPipe.java:56)
com.dremio.sabot.driver.Pipeline(Pipeline.java:109)
com.dremio.sabot.driver.Pipeline(Pipeline.java:99)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:328)
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper(FragmentExecutor.java:324)
…(:0)
org.apache.hadoop.security.UserGroupInformation(UserGroupInformation.java:1633)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:281)
com.dremio.sabot.exec.fragment.FragmentExecutor(FragmentExecutor.java:92)
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl(FragmentExecutor.java:671)
com.dremio.sabot.task.AsyncTaskWrapper(AsyncTaskWrapper.java:104)
com.dremio.sabot.task.slicing.SlicingThread(SlicingThread.java:226)
com.dremio.sabot.task.slicing.SlicingThread(SlicingThread.java:156)
Thanks