Dremio in Docker

#1

So I’ve been working on trying to containerize Dremio in order for me to more effectively build from source/port the app or whatever use case you can think of with Docker containers and Dremio. However what I’m tripping on is the startup command for Dremio. Yes, the start-in command for Dremio, depending on the system is either running the application (MacOS) or by running service Dremio start.

However, docker throws out the concept of a system manager and expects the startup command to be the command that actually runs the app. In other words, if I were to run Dremio’s true startup command from the command line, my terminal would become consumed by the process. In order for Dremio to correctly run inside a docker container, I need this command.

According to the Dremio bash script, the following line is what I have come to believe to be the true startup command for Dremio:
exec $JAVA $DREMIO_JAVA_OPTS -cp "$DREMIO_CLASSPATH" com.dremio.dac.daemon.DremioDaemon $command "$@" start >> "$logout" 2>&1 &

Which according to Dremio.service file, some of these variables are defined in the config file used to start up the service (dremio/conf). If I were to run this command from the command line alone, what would it look like?

1 Like
#2

<path_to_dremio_install>/bin/dremio [–config <dremio_conf_dir>] start

Where <dremio_conf_dir> is by default <path_to_dremio_install>/conf

#3

However that command will stop executing after its done, and thus the docker container will close when that command is done. I said it needs to be the true start up command, not the one thats used through the bash script.

#4

As you pointed out here is the “real” java command

Why don’t you remove “&” at the end of the command and still use bash script to start command, so script won’t exit

#5

Agreed, the dremio script is missing an option to run in the foreground instead of detaching itself, which would be useful to run in containerized environment like Docker (but not only).

You can probably figure out the specific commandline by running the daemon in the background, and using ps to get it (or by using /proc/<pid>/cmdline), but it should looks something similar to this:

${JAVA_HOME}/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:<path/to/log>/server.gc -Ddremio.log.path=<path/to/log> -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=<path/to/log> -Xms4096m -cp <path/to/conf>:<path/to/dremio>/jars/*:<path/to/dremio>/jars/ext/*:<path/to/dremio>/jars/3rdparty/* com.dremio.dac.daemon.DremioDaemon dremio start

Note that only the path to the configuration is added to the classpath, not the actual configuration file. Dremio actually expects to find a dremio.conf in the application classpath (and if none is found, default configuration will be used instead as a fallback).

1 Like
#6

Its working. Thanks a million.

1 Like
#7

Jouster, could you please share the container if you are happy with it?

Thanks

#8

FYI I’ve created a PR for adding Docker support.

BP

#9

Update on Docker: Official Docker Image for Dremio

#10

More on tools for Docker and Kubernetes here.

#11

Question on the dremio docker image - what is the root password ?

#12

As far as I know, no root password is configured for the image. That said, you should be able to use docker exec -u root <container id> to get a shell as root in a running image.

#13

Ahh - of course. I was trying by exec -it and defaults to the dremio user. Just tried and that worked. Thanks!