No logs in Dremio Docker image

1

Hi,
When I run the Dremio Docker image, there’s no longer server.log file produced in the logs folder, I’ve tried changing it to a different path but it only writes the server.out file.

Is there some option that I need to enable?

Thank You

2

Hi @dobodo5322

Did you configure systemctl? Check journal

journalctl _PID=<Dremio-PID>

journalctl -u dremio.service
3

When I connect to the Dremio docker and try to run you command I get:
“bash: journalctl: command not found”

And just to add to my original post, this is on a docker image that I just downloaded, I didn’t change any settings and the logs don’t work right from the beginning

4

Hi @dobodo5322

When you start the Dremio docker, something like below, the same window should be tailing the logs interactively on the screen (standard out)

-v is to map a local folder on to the docker image

docker run -v /Users/balajiramaswamy/apache-hive-1.2.2-bin/data:/opt/hive -p 9047:9047 -p 31010:31010 -p 45678:45678 dremio/dremio-oss

Thanks
@balaji.ramaswamy

5

Hi. Is there still no way how to “enable” log files (as opposed to on screen) in Dremio docker container?

6

@david-kubecka

You can configure logback.xml to write logs to a persistent disk, in your values.yaml first do the below

V1 Template

Edit dremio-master.yaml , dremio-coordinator.yaml, dremio-executor.yaml

1 2 3 name: DREMIO_JAVA_EXTRA_OPTS value: >- -Ddremio.log.path="/opt/dremio/data/blah"

V2 Template

1 2 extraStartParams: >- -Ddremio.log.path="/opt/dremio/data/blah"

7

I’m just playing directly with the docker container (via simple custom mode compose file). I’ve tried to put this into my compose file:

  environment:
    DREMIO_JAVA_EXTRA_OPTS: '-Ddremio.log.path="/opt/dremio/data/custom.log"'

But still I don’t see any /opt/dremio/data/custom.log.

Is there any way how to configure logs in docker container?

8

@david-kubecka

If you open your dremio-env, what does the entry for the below point to?

DREMIO_LOG_DIR=${DREMIO_HOME}/log

Also, if you do a ps -ef | grep dremio , what does -Ddremio.log.path point to?

9

@balaji.ramaswamy

In the docker container $DREMIO_HOME is set to /opt/dremio.

Regarding the Java env it’s -Ddremio.log.path="/opt/dremio/data/custom.log". But that file doesn’t exist.

10

@david-kubecka

Can you please send me the output of ps -ef | grep dremio

11 
root@6921c353f5cb:/opt/dremio#  ps -ef | grep dremio
dremio       1     0 99 09:56 ?        00:00:30 /usr/local/openjdk-8/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Ddremio.plugins.path=/opt/dremio/plugins -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+PrintClassHistogramAfterFullGC -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/dremio -Dio.netty.maxDirectMemory=0 -Dio.netty.tryReflectionSetAccessible=true -DMAPR_IMPALA_RA_THROTTLE -DMAPR_MAX_RA_STREAMS=400 -XX:+UseG1GC -cp /opt/dremio/conf:/opt/dremio/jars/*:/opt/dremio/jars/ext/*:/opt/dremio/jars/3rdparty/*:/usr/local/openjdk-8/lib/tools.jar com.dremio.dac.daemon.DremioDaemon
root       530   510  0 09:57 pts/0    00:00:00 grep dremio

I’m using dremio docker image (dremio/dremio-oss:latest) so I guess you should be able to reproduce even yourself if needed.

12

@david-kubecka

Have your tried docker logs?

13

I realize that this is a very old question. You need to set an environment variable with the path to the log file - “dremio.log.path” Here is an example:

docker run -dti -e DREMIO_JAVA_SERVER_EXTRA_OPTS=-Ddremio.log.path=/var/log/dremio dremio/dremio-oss:latest dremio

The server.log will be written to /var/log/dremio

Thanks.