Dremio 4 failed to read parquet

Hi Dremio 4.0.2 running in container and failed reading parquet file:

Error in parquet reader (complex). Message: Failure in setting up reader Parquet Metadata: ParquetMetaData{FileMetaData{schema: message spark_schema { optional int64 dsid; optional binary consumer_id (STRING); optional binary person_id (STRING); optional binary derived_key; } , metadata: {org.apache.spark.sql.parquet.row.metadata={"type":"struct","fields":[{"name":"dsid","type":"long","nullable":true,"metadata":{}},{"name":"consumer_id","type":"string","nullable":true,"metadata":{}},{"name":"person_id","type":"string","nullable":true,"metadata":{}},{"name":"derived_key","type":"binary","nullable":true,"metadata":{}}]}}}, blocks: [BlockMetaData{567053, 60113622 [ColumnMetaData{SNAPPY [dsid] optional int64 dsid [RLE, BIT_PACKED, PLAIN], 4}, ColumnMetaData{SNAPPY [consumer_id] optional binary consumer_id (STRING) [RLE, BIT_PACKED, PLAIN], 2939903}, ColumnMetaData{SNAPPY [person_id] optional binary person_id (STRING) [RLE, BIT_PACKED, PLAIN], 21515693}, ColumnMetaData{SNAPPY [derived_key] optional binary derived_key [RLE, BIT_PACKED, PLAIN], 40091600}]}]}

Note: if I run the same distr with the same config from edge node on the cluster all is fine. Some libs missing?

Update: Logs show java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available

Turned out that on container it was jdk11 and su.misc.Unsafe was moved out of there. Switching back to jdk8 resolved the issue

Thanks for the update @comphead

I have installed dremio in ubuntu, I am receiving below error while loading sample csv file in Dremio

sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available

I also removed jdk 11 and my jdk version is JDK 8 now, Still i do receive same error.Please help ASAP

java -version
openjdk version “1.8.0_232”
OpenJDK Runtime Environment (build 1.8.0_232-8u232-b09-0ubuntu1~18.04.1-b09)
OpenJDK 64-Bit Server VM (build 25.232-b09, mixed mode)

I would recommend double check that your java env when you starting dremio is exactly 1.8, not just java -version


best is to the below line in dremio-env and restart Dremio
export JAVA_HOME=path

1- JAVA_HOME is set executing below command for same session.
export JAVA_HOME
export PATH

2- echo $JAVA_HOME

3-service --status-all

[ - ] dremio

sudo systemctl restart dremio


Job for dremio.service failed because the control process exited with error code.
See “systemctl status dremio.service” and “journalctl -xe” for details.

4- systemctl status dremio.service


● dremio.service - LSB: This service manages the Dremio server
Loaded: loaded (/etc/init.d/dremio; generated)
Active: failed (Result: exit-code) since Thu 2019-12-26 16:02:35 IST; 14s ago
Docs: man:systemd-sysv-generator(8)
Process: 31446 ExecStart=/etc/init.d/dremio start (code=exited, status=1/FAILU
Tasks: 120 (limit: 4915)
CGroup: /system.slice/dremio.service
└─1016 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Djava.util.loggin

pam_unix(runuser:session): session opened
| Error: JAVA_HOME is not set and Jav
| Dremio requires Java 1.8 or later.
pam_unix(runuser:session): session closed
dremio.service: Control process exited, code=
dremio.service: Failed with result 'exit-code
Failed to start LSB: This service manages the

From the above commands you could see even setting up JAVA_HOME for particular session, dremio services are not starting up and tries to refer JDK 11, which was uninstalled



Did you say you set JAVA_HOME, inside dremio-env and still running into issues?

If not please set it in dremio-env
Also can you try to start dremio from “bin/dremio start” instead of systemctl

The issue is resolved, after removing all the links to jdk 11, as dreimo services were referring jdk 11.

this’s how it looks now after referringto jdk 8

CGroup: /system.slice/dremio.service
└─1621 /usr/lib/jvm/java-8-openjdk-amd64/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -X