Hello Dremio Community,
currently I am setting up Dremio on an AWS machine. I was able to start the service and status looks good immediately after launching it:
● dremio.service - Dremio Daemon Server
Loaded: loaded (/etc/systemd/system/dremio.service; enabled; vendor preset: enabled)
Active: active (running) since Mon 2018-06-11 09:26:52 UTC; 2s ago
Docs: https://docs.dremio.com
Process: 3385 ExecStop=/opt/dremio/bin/dremio stop (code=exited, status=0/SUCCESS)
Main PID: 3668 (java)
Tasks: 10
Memory: 80.0M
CPU: 2.091s
CGroup: /system.slice/dremio.service
└─3668 /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+Pri
Jun 11 09:26:52 ip-172-30-0-244 systemd[1]: Started Dremio Daemon Server.
Jun 11 09:26:52 ip-172-30-0-244 dremio[3668]: starting dremio, logging to var/log/dremio/server.out
However, it crashes within 3 seconds. I get this status
● dremio.service - Dremio Daemon Server
Loaded: loaded (/etc/systemd/system/dremio.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Mon 2018-06-11 09:26:58 UTC; 15s ago
Docs: https://docs.dremio.com
Process: 3751 ExecStop=/opt/dremio/bin/dremio stop (code=exited, status=0/SUCCESS)
Process: 3668 ExecStart=/opt/dremio/bin/dremio start-fg (code=exited, status=1/FAILURE)
Main PID: 3668 (code=exited, status=1/FAILURE)
Jun 11 09:26:52 ip-172-30-0-244 systemd[1]: Started Dremio Daemon Server.
Jun 11 09:26:52 ip-172-30-0-244 dremio[3668]: starting dremio, logging to var/log/dremio/server.out
Jun 11 09:26:58 ip-172-30-0-244 systemd[1]: dremio.service: Main process exited, code=exited, status=1/FAILURE
Jun 11 09:26:58 ip-172-30-0-244 dremio[3751]: no dremio to stop because kill -0 of pid 3668 failed with status 1
Jun 11 09:26:58 ip-172-30-0-244 systemd[1]: dremio.service: Unit entered failed state.
Jun 11 09:26:58 ip-172-30-0-244 systemd[1]: dremio.service: Failed with result ‘exit-code’.
Looking at the most recent entry in server.out it seems like a LOCK file can’t be opened:
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 3900
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 3900
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Exception in thread “main” org.rocksdb.RocksDBException: while open a file for lock: /opt/dremio/data/db/catalog/LOCK: Permission denied
at org.rocksdb.RocksDB.open(Native Method)
at org.rocksdb.RocksDB.open(RocksDB.java:286)
at com.dremio.datastore.ByteStoreManager.openDB(ByteStoreManager.java:204)
at com.dremio.datastore.ByteStoreManager.start(ByteStoreManager.java:160)
at com.dremio.datastore.CoreStoreProviderImpl.start(CoreStoreProviderImpl.java:167)
at com.dremio.datastore.LocalKVStoreProvider.start(LocalKVStoreProvider.java:88)
at com.dremio.dac.daemon.DremioDaemon.checkVersion(DremioDaemon.java:86)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:125)
The most recent entry in the according dremio.log file is:
2018-06-11 08:45:27,965 [main] INFO c.d.common.scanner.BuildTimeScan - Loaded prescanned packages [com.dremio.service.jobs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.service.users, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.provision, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.service.jobs, com.dremio.service.namespace, com.dremio.service.users, com.dremio.exec.store, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.storage, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.service.users, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.storage, org.apache.hadoop.hive, com.dremio.exec.fn.hive, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.namespace, com.dremio.service.users, org.apache.hadoop.hive, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.service.users, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.jobs, com.dremio.service.namespace, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.exec.store] from locations [jar:file:/opt/dremio/jars/dremio-services-jobs-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-backend-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-datastore-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-common-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-common-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-logical-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-hive-plugin-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-accelerator-2.0.5-201806021755080191-767cfb5.jar!/META-INF/dremio-module-scan/registry.json]
2018-06-11 08:45:29,469 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.plugins.s3.store, com.dremio.plugins.adl, com.dremio.service.jobs, com.dremio.service.users, com.dremio.exec.store.jdbc, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.plugins.mongo, com.dremio.provision.yarn.service, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.plugins.elastic, com.dremio.exec.store, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.extras.plugins.elastic, org.apache.hadoop.hive, com.dremio.exec.fn.hive, com.dremio.exec.store.hbase, com.dremio.exec.expr.fn.impl.conv, com.dremio.service.namespace, com.dremio.provision, com.dremio.plugins.mongo, com.dremio.exec.store.jdbc, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.extra.exec.store.dfs, com.dremio.exec.planner.acceleration.substitution, org.apache.hadoop.hive] in locations [jar:file:/opt/dremio/jars/dremio-s3-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-hbase-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-jdbc-plugin-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-2.0.5-201806021755080191-767cfb5.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-2.0.5-201806021755080191-767cfb5.jar!/] took 1491ms
2018-06-11 08:45:29,544 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider
2018-06-11 08:45:29,728 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopping LocalKVStoreProvider
2018-06-11 08:45:29,731 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopped LocalKVStoreProvider
Also I can provide the access permissions for the LOCK file @ /opt/dremio/data/db/catalog:
-rwxr-xr-x 1 hadoop_user hadoop_group 0 Jun 8 14:44 LOCK
Any help is appreciated!
Thanks in advance!