Memory leak issue

1

Looking for help solving an out of memory issue. Wondering if anyone can help…

**Apache arrow version: 0.14.0-20190718172626-fa1a62d422-dremio

(org.apache.arrow.memory.OutOfMemoryException) Unable to allocate buffer of size 32768 (rounded from 22488) due to memory limit. Current allocation: 24652052
org.apache.arrow.memory.BaseAllocator.buffer():307
org.apache.arrow.memory.BaseAllocator.buffer():277
com.dremio.parquet.pages.BaseReaderIterator.renderDataPage():110
com.dremio.parquet.pages.IncrementalPageReaderIterator.getPage():174
com.dremio.parquet.pages.MemoizingPageIterator.getPage():41
com.dremio.parquet.pages.PageIterator.nextPage():118
com.dremio.parquet.pages.PageIterator.hasNextPage():63
com.dremio.parquet.reader.column.generics.VarCharDeltaReader.evalNextBatch():95
com.dremio.parquet.reader.FilteringRowGroupReader.eval():41
com.dremio.extra.exec.store.dfs.parquet.ParquetVectorizedReader.next():320
com.dremio.exec.store.parquet.UnifiedParquetReader.next():236
com.dremio.exec.store.hive.exec.FileSplitParquetRecordReader.next():252
com.dremio.sabot.op.scan.ScanOperator.outputData():249
com.dremio.sabot.driver.SmartOp$SmartProducer.outputData():521
com.dremio.sabot.driver.StraightPipe.pump():56
com.dremio.sabot.driver.Pipeline.doPump():109
com.dremio.sabot.driver.Pipeline.pumpOnce():99
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():328
com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():324
java.security.AccessController.doPrivileged():-2
javax.security.auth.Subject.doAs():422
org.apache.hadoop.security.UserGroupInformation.doAs():1633
com.dremio.sabot.exec.fragment.FragmentExecutor.run():281
com.dremio.sabot.exec.fragment.FragmentExecutor.access$1300():92
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():671
com.dremio.sabot.task.AsyncTaskWrapper.run():104
com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop():226
com.dremio.sabot.task.slicing.SlicingThread.run():156

2

bump… .anyone have any thoughts?

3

@ou101

When Dremio runs out of direct memory, it should dump all the allocators including the all the query id’s and how much memory it used. I do not see that. What version of Dremio is this? How much direct memory do you have per executor and can we have the profile?

4

Thank you. 0.14.0-20190718172626-fa1a62d422-dremio is the version. I will get the direct memory info shortly.