Hello,
My dremio master node works fine, but I cannot open dremio executor because
Could not bind to port 45678
I tried to kill the process using port 45678 in master node or executor node. But the issue still unsolved.
Can I change the port in config file? what is the dremio-env to change the port and should i make change on master or executor?
Thank you.
Here is the documentation on dremio.conf content: http://docs.dremio.com/advanced-administration/dremio-conf.html
Look for fabric: { port: 45678
Even I changed the port, it gives me the same error log.
Catastrophic failure occurred. Exiting. Information follows: Failed to start services, daemon exiting.
com.dremio.common.exceptions.UserException: Could not bind to port 56789.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:746)
at com.dremio.exec.rpc.BasicServer.bind(BasicServer.java:220)
at com.dremio.services.fabric.FabricServer.bind(FabricServer.java:72)
at com.dremio.services.fabric.FabricServiceImpl.start(FabricServiceImpl.java:92)
at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)
at com.dremio.dac.daemon.NonMasterSingletonRegistry.start(NonMasterSingletonRegistry.java:54)
at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:177)
at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:183)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:112)
Caused by: java.net.BindException: Address already in use
Do you have a firewall preventing port binding? Or do you have another process running on those specific ports? You can check via netstat -tulpn for example
To add here - are you trying by any chance to start executor on the same node as master? That combination is not really working.
helllo @yufeldman,
No, my master node and executor node are two different node.
@anthony, netstat -tulpn | grep 56789 on master gives
tcp 0 0 0.0.0.0:56789 0.0.0.0:* LISTEN 121054/java
netstat -tulpn | grep 56789 on executor gives
tcp 0 0 0.0.0.0:56789 0.0.0.0:* LISTEN 117107/java
And so what are those processes?
Especially 117107 ?
you can find it out by running ps -aef | grep 117107
@yufeldman
Here is the output
pz50apl+ 117107 1 1 13:09 ? 00:00:37 /usr/bin/java/jdk1.8.0_181/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/opt/dremio/log/server.gc -Ddremio.log.path=/opt/dremio/log -Xmx4096m -XX:MaxDirectMemorySize=40960m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/opt/dremio/log -Dio.netty.maxDirectMemory=0 -DMAPR_IMPALA_RA_THROTTLE -DMAPR_MAX_RA_STREAMS=400 -cp /opt/dremio/conf:/opt/dremio/jars/:/opt/dremio/jars/ext/:/opt/dremio/jars/3rdparty/* com.dremio.dac.daemon.DremioDaemon dremio start
pz50apl+ 128410 121897 0 13:48 pts/0 00:00:00 grep --color 117107
So Dremio is running? Is it that somehow happening that you have two processes starting and second one has an issue, as address is already in use?
I don’t understand how can this happen.
Because when I do dremio status, it gives me dremio is not running.
Can you manually kill all those processes and then restart Dremio with coordinator first then executors?
I killed all processes.
But still get the same results.
Master was able to run, but not executor.
Hi @carol
To change the fabric port use below document
https://docs.dremio.com/advanced-administration/dremio-conf.html?h=45678
Thanks
@balaji.ramaswamy