Investigation 1
I saw:
Starting dremio, logging to /opt/dremio/log/server.out
Check the log file to make sure dremio started successfully and did not run into any issues.
[root@915468-ElasticDemo ~]# netstat -pluton | grep 9047
[root@915468-ElasticDemo ~]# netstat -pluton | grep 9047
The port was not opened. And so I looked in /opt/dremio/log/server.out:
17:35:10,937 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[null] - Active log file name: /opt/dremio/log/hive.deprecated.function.warning.log
17:35:10,937 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[null] - File property is set to [/opt/dremio/log/hive.deprecated.function.warning.log]
17:35:10,938 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@2d710f1a - Propagating ERROR level on Logger[ROOT] onto the JUL framework
17:35:10,938 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - ROOT level set to ERROR
17:35:10,940 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [text] to Logger[ROOT]
17:35:10,940 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [json] to Logger[ROOT]
17:35:10,940 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:35:10,941 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@13fd2ccd - Registering current configuration as safe fallback point
Dremio is exiting. Failure while starting services.
java.lang.UnsatisfiedLinkError: /tmp/librocksdbjni6790346469476059877.so: /tmp/librocksdbjni6790346469476059877.so: failed to map segment from shared object: Operation not permitted
at java.lang.ClassLoader$NativeLibrary.load(Native Method)
at java.lang.ClassLoader.loadLibrary0(ClassLoader.java:1934)
at java.lang.ClassLoader.loadLibrary(ClassLoader.java:1817)
at java.lang.Runtime.load0(Runtime.java:809)
at java.lang.System.load(System.java:1086)
at org.rocksdb.NativeLibraryLoader.loadLibraryFromJar(NativeLibraryLoader.java:78)
at org.rocksdb.NativeLibraryLoader.loadLibrary(NativeLibraryLoader.java:56)
at org.rocksdb.RocksDB.loadLibrary(RocksDB.java:64)
at org.rocksdb.RocksDB.<clinit>(RocksDB.java:35)
at org.rocksdb.Options.<clinit>(Options.java:25)
at com.dremio.datastore.ByteStoreManager.start(ByteStoreManager.java:201)
at com.dremio.datastore.CoreStoreProviderImpl.start(CoreStoreProviderImpl.java:192)
at com.dremio.datastore.LocalKVStoreProvider.start(LocalKVStoreProvider.java:157)
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:160)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:99)
Suppressed: java.lang.IllegalStateException: #start was not invoked, so metadataManager is not available
So, I looked “https://stackoverflow.com/questions/13502156/what-are-possible-causes-of-failed-to-map-segment-from-shared-object-operation” and saw:
Permission issue. Need to remount /tmp. The following command works for me (Centos 7):
sudo mount /tmp -o remount,exec
So, I applied:
[root@915468-ElasticDemo tmp]# sudo mount /tmp -o remount,exec
[root@915468-ElasticDemo tmp]# netstat -pluton | grep 9047
[root@915468-ElasticDemo tmp]#
Investigation 2
And restarted:
[root@915468-ElasticDemo tmp]# service dremio restart
stopping dremio
.
#######################################################################
WARNING:
Please increase the maximum number of file descriptors to at least 65536.
Dremio utilizes a large number of file descriptors.
The maximum number of file descriptors, currently set at 8192, is not sufficient.
#######################################################################
Starting dremio, logging to /opt/dremio/log/server.out
Check the log file to make sure dremio started successfully and did not run into any issues.
I updated the live maximum number of descriptors:
echo “65536” > /proc/sys/fs/file-max
The, updated for reboot:
vi /etc/sysctl.conf
fs.file-max = 65536
And looked at the logs
[root@915468-ElasticDemo tmp]# vi /opt/dremio/log/server.out
17:42:52,783 |-INFO in ch.qos.logback.classic.jul.LevelChangePropagator@2d710f1a - Propagating ERROR level on Logger[ROOT] onto the JUL framework
17:42:52,783 |-INFO in ch.qos.logback.classic.joran.action.LevelAction - ROOT level set to ERROR
17:42:52,785 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [text] to Logger[ROOT]
17:42:52,785 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [json] to Logger[ROOT]
17:42:52,785 |-INFO in ch.qos.logback.classic.joran.action.ConfigurationAction - End of configuration.
17:42:52,786 |-INFO in ch.qos.logback.classic.joran.JoranConfigurator@13fd2ccd - Registering current configuration as safe fallback point
Dremio is exiting. Failure while starting services.
com.dremio.common.exceptions.UserException: No Cluster Identity found
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:776)
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:172)
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:162)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:99)
According to “https://community.dremio.com/t/dremio-dockers-stop-as-soon-as-i-started-them/2523” we need to “remove” the data/db directory:
[root@915468-ElasticDemo tmp]# cd /opt/dremio/data
[root@915468-ElasticDemo data]# mkdir ~/dremio-backup
[root@915468-ElasticDemo data]# mv db ~/dremio-backup/
[root@915468-ElasticDemo data]# service dremio restart
[root@915468-ElasticDemo data]# vi /opt/dremio/log/server.out
And then SUCCESS:
