Exception from create Page List From Column And Offset Index

1

Dremio 17.0.0

I am getting the following error when running a query:

c.d.p.reader.metadata.RowGroups - Exception from createPageListFromColumnAndOffsetIndex java.lang.ArrayIndexOutOfBoundsException

The message is logged from multiple threads while running the query. The query is not being accelerated, and I am unable to cancel the job.

The same query on a different but similar setup with less data is working fine and is being accelerated by the expected reflections.

Same dremio version, same table structure, same reflections and same query.

NB! I had to split the method name in the title for the post to be accepted (Title seems unclear, one or more words is very long)

2

@dotjdk Kindly share the query profile

3

Unfortunately I can’t. It contains sensitive information.

Seems to be the same issue reported here: Parquet Reader plugin fails to read metadata with ArrayIndexOutOfBoundsException

4

@dotjdk Would you be able to send output of parquet-tools meta <parquet-file>

5

There is no information in the logs about which files it has trouble reading. Only repeated WARN logs containing nothing but the text in my original post.

6

@dotjdk Would you be able to send the parquet file schema using parquet-tools?