Dremio Docker Container Stuck on Startup with Apple M3 Chip

1

Hello,

I am trying to run Dremio for the first time using the Docker image on my Rancher Desktop. I’m using an Apple M3 chip, and I encountered an issue where the Dremio container gets stuck during startup.

Steps to Reproduce

  1. Pull the Dremio Docker image:
    docker pull dremio/dremio-oss
    2 . Run the Dremio Docker container:
    docker run -p 9047:9047 -p 31010:31010 -p 45678:45678 -p 32010:32010 --name try-dremio dremio/dremio-oss

Observed Behavior

The container logs show the following output and then get stuck:
[2024-08-12T14:39:49.973+0000][4.638s][info][gc ] GC(4) Pause Young (Normal) (G1 Evacuation Pause) 73M->42M(152M) 10.427ms
[2024-08-12T14:39:49.973+0000][4.638s][info][gc,cpu ] GC(4) User=0.02s Sys=0.00s Real=0.01s
2024-08-12 14:39:50,287 [main] INFO com.dremio.dac.daemon.DremioDaemon - Starting dremio daemon

The logs do not proceed past this point, and I am unable to access the Dremio web UI at http://localhost:9047.

Environment Details

  • Host Machine: Apple M3 chip
  • Docker Environment: Rancher Desktop
  • Dremio Docker Image: dremio/dremio-oss

Attempts to Resolve

  1. Verified that the container is running using docker ps.
  2. Checked the container logs using docker logs -f try-dremio.
  3. Specified the platform using --platform linux/amd64 in the docker run command.
  4. Ensured sufficient system resources (CPU, memory).
  5. No additional logs were generated after the initial startup message.

Any guidance or suggestions on how to resolve this issue would be greatly appreciated.
Thank you!

2

@Sarwan Are you able to send server.log and sever.out after it gets stuck?

3

Hi @balaji.ramaswamy

Thank you for your response. I’m quite new to Docker and would appreciate it if you could guide me on how to retrieve the server.log and server.out files from the Dremio container after it gets stuck. Could you please provide the steps or any commands I need to run?

Thank you for your help!

Best regards,
Sarwan

Feel free to customize it as needed!

4

@Sarwan

You would have to look at the container logs

1 Like
5

@balaji.ramaswamy below is the full log am seeing

starting dremio
[2024-08-15T16:42:34.153+0000][0.053s][info][gc,heap] Heap region size: 1M
[2024-08-15T16:42:34.169+0000][0.069s][info][gc ] Using G1
[2024-08-15T16:42:34.169+0000][0.069s][info][gc,heap,coops] Heap address: 0x0000000700000000, size: 4096 MB, Compressed Oops mode: Zero based, Oop shift amount: 3
[2024-08-15T16:42:34.172+0000][0.072s][info][gc,cds ] Mark closed archive regions in map: [0x00000007bfe00000, 0x00000007bfe6bff8]
[2024-08-15T16:42:34.173+0000][0.073s][info][gc,cds ] Mark open archive regions in map: [0x00000007bfc00000, 0x00000007bfc46ff8]
[2024-08-15T16:42:36.198+0000][2.098s][info][gc,start ] GC(0) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:36.199+0000][2.099s][info][gc,task ] GC(0) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:36.247+0000][2.147s][info][gc,phases ] GC(0) Pre Evacuate Collection Set: 0.3ms
[2024-08-15T16:42:36.247+0000][2.147s][info][gc,phases ] GC(0) Evacuate Collection Set: 38.4ms
[2024-08-15T16:42:36.247+0000][2.147s][info][gc,phases ] GC(0) Post Evacuate Collection Set: 4.6ms
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,phases ] GC(0) Other: 6.1ms
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,heap ] GC(0) Eden regions: 24->0(13)
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,heap ] GC(0) Survivor regions: 0->3(3)
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,heap ] GC(0) Old regions: 2->15
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,heap ] GC(0) Humongous regions: 7->7
[2024-08-15T16:42:36.248+0000][2.148s][info][gc,metaspace ] GC(0) Metaspace: 781K(4864K)->781K(4864K) NonClass: 748K(4352K)->748K(4352K) Class: 32K(512K)->32K(512K)
[2024-08-15T16:42:36.249+0000][2.149s][info][gc ] GC(0) Pause Young (Normal) (G1 Evacuation Pause) 31M->22M(126M) 51.009ms
[2024-08-15T16:42:36.249+0000][2.149s][info][gc,cpu ] GC(0) User=0.11s Sys=0.06s Real=0.05s
[2024-08-15T16:42:36.681+0000][2.581s][info][gc,start ] GC(1) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:36.681+0000][2.581s][info][gc,task ] GC(1) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:36.699+0000][2.599s][info][gc,phases ] GC(1) Pre Evacuate Collection Set: 0.2ms
[2024-08-15T16:42:36.699+0000][2.599s][info][gc,phases ] GC(1) Evacuate Collection Set: 16.9ms
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,phases ] GC(1) Post Evacuate Collection Set: 0.8ms
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,phases ] GC(1) Other: 0.7ms
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,heap ] GC(1) Eden regions: 8->0(20)
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,heap ] GC(1) Survivor regions: 3->2(2)
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,heap ] GC(1) Old regions: 15->21
[2024-08-15T16:42:36.699+0000][2.600s][info][gc,heap ] GC(1) Humongous regions: 12->12
[2024-08-15T16:42:36.700+0000][2.600s][info][gc,metaspace ] GC(1) Metaspace: 966K(4864K)->966K(4864K) NonClass: 921K(4352K)->921K(4352K) Class: 45K(512K)->45K(512K)
[2024-08-15T16:42:36.700+0000][2.600s][info][gc ] GC(1) Pause Young (Normal) (G1 Evacuation Pause) 35M->33M(126M) 19.076ms
[2024-08-15T16:42:36.700+0000][2.600s][info][gc,cpu ] GC(1) User=0.05s Sys=0.00s Real=0.02s
[2024-08-15T16:42:37.494+0000][3.395s][info][gc,start ] GC(2) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:37.494+0000][3.395s][info][gc,task ] GC(2) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:37.502+0000][3.403s][info][gc,phases ] GC(2) Pre Evacuate Collection Set: 0.1ms
[2024-08-15T16:42:37.502+0000][3.403s][info][gc,phases ] GC(2) Evacuate Collection Set: 6.5ms
[2024-08-15T16:42:37.502+0000][3.403s][info][gc,phases ] GC(2) Post Evacuate Collection Set: 1.1ms
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,phases ] GC(2) Other: 0.5ms
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,heap ] GC(2) Eden regions: 20->0(25)
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,heap ] GC(2) Survivor regions: 2->3(3)
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,heap ] GC(2) Old regions: 21->24
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,heap ] GC(2) Humongous regions: 17->17
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,metaspace ] GC(2) Metaspace: 2773K(5760K)->2773K(5760K) NonClass: 2502K(5120K)->2502K(5120K) Class: 271K(640K)->271K(640K)
[2024-08-15T16:42:37.503+0000][3.403s][info][gc ] GC(2) Pause Young (Normal) (G1 Evacuation Pause) 58M->41M(126M) 8.437ms
[2024-08-15T16:42:37.503+0000][3.403s][info][gc,cpu ] GC(2) User=0.02s Sys=0.00s Real=0.00s
[2024-08-15T16:42:38.152+0000][4.053s][info][gc,start ] GC(3) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:38.152+0000][4.053s][info][gc,task ] GC(3) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,phases ] GC(3) Pre Evacuate Collection Set: 0.1ms
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,phases ] GC(3) Evacuate Collection Set: 8.4ms
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,phases ] GC(3) Post Evacuate Collection Set: 0.6ms
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,phases ] GC(3) Other: 0.6ms
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,heap ] GC(3) Eden regions: 25->0(28)
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,heap ] GC(3) Survivor regions: 3->2(4)
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,heap ] GC(3) Old regions: 24->26
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,heap ] GC(3) Humongous regions: 17->17
[2024-08-15T16:42:38.162+0000][4.062s][info][gc,metaspace ] GC(3) Metaspace: 4422K(6656K)->4422K(6656K) NonClass: 3913K(5888K)->3913K(5888K) Class: 508K(768K)->508K(768K)
[2024-08-15T16:42:38.162+0000][4.062s][info][gc ] GC(3) Pause Young (Normal) (G1 Evacuation Pause) 66M->42M(126M) 9.932ms
[2024-08-15T16:42:38.162+0000][4.063s][info][gc,cpu ] GC(3) User=0.03s Sys=0.01s Real=0.01s
[2024-08-15T16:42:39.176+0000][5.076s][info][gc,start ] GC(4) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:39.176+0000][5.076s][info][gc,task ] GC(4) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:39.184+0000][5.084s][info][gc,phases ] GC(4) Pre Evacuate Collection Set: 0.0ms
[2024-08-15T16:42:39.184+0000][5.084s][info][gc,phases ] GC(4) Evacuate Collection Set: 7.0ms
[2024-08-15T16:42:39.184+0000][5.084s][info][gc,phases ] GC(4) Post Evacuate Collection Set: 0.8ms
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,phases ] GC(4) Other: 0.5ms
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,heap ] GC(4) Eden regions: 28->0(32)
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,heap ] GC(4) Survivor regions: 2->3(4)
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,heap ] GC(4) Old regions: 26->26
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,heap ] GC(4) Humongous regions: 17->17
[2024-08-15T16:42:39.184+0000][5.085s][info][gc,metaspace ] GC(4) Metaspace: 8355K(9088K)->8355K(9088K) NonClass: 7414K(7936K)->7414K(7936K) Class: 940K(1152K)->940K(1152K)
[2024-08-15T16:42:39.185+0000][5.085s][info][gc ] GC(4) Pause Young (Normal) (G1 Evacuation Pause) 70M->43M(126M) 8.612ms
[2024-08-15T16:42:39.185+0000][5.085s][info][gc,cpu ] GC(4) User=0.03s Sys=0.00s Real=0.01s
2024-08-15 16:42:39,618 [main] INFO com.dremio.dac.daemon.DremioDaemon - Starting dremio daemon
[2024-08-15T16:42:39.721+0000][5.621s][info][gc,start ] GC(5) Pause Young (Normal) (G1 Evacuation Pause)
[2024-08-15T16:42:39.721+0000][5.621s][info][gc,task ] GC(5) Using 3 workers of 8 for evacuation
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,phases ] GC(5) Pre Evacuate Collection Set: 0.1ms
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,phases ] GC(5) Evacuate Collection Set: 7.1ms
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,phases ] GC(5) Post Evacuate Collection Set: 0.6ms
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,phases ] GC(5) Other: 0.4ms
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,heap ] GC(5) Eden regions: 32->0(41)
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,heap ] GC(5) Survivor regions: 3->3(5)
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,heap ] GC(5) Old regions: 26->26
[2024-08-15T16:42:39.729+0000][5.629s][info][gc,heap ] GC(5) Humongous regions: 17->17
[2024-08-15T16:42:39.729+0000][5.630s][info][gc,metaspace ] GC(5) Metaspace: 10158K(10752K)->10158K(10752K) NonClass: 9029K(9472K)->9029K(9472K) Class: 1128K(1280K)->1128K(1280K)
[2024-08-15T16:42:39.729+0000][5.630s][info][gc ] GC(5) Pause Young (Normal) (G1 Evacuation Pause) 75M->43M(126M) 8.547ms
[2024-08-15T16:42:39.729+0000][5.630s][info][gc,cpu ] GC(5) User=0.01s Sys=0.01s Real=0.01s

6

Could you please suggest any alternative options for launching and exploring Dremio locally?

7

@Sarwan Are you ok to use tar ball on a Linux machine? or you can use rpm