Just trying a simple EC2 install, one coordinator, one executor. but I’m unable to see the executor in the admin console. Any suggestions on my config?
Coordinate config:
master: {
name: 10.0.0.1,
port: 45678
}
paths: {
local: “/data/dremio”
}
services: {
coordinator.enabled: true,
executor.enabled: false
}
Executor Config:
master: {
name: 10.0.0.1,
port: 45678
}
paths: {
local: “/data/dremio”
}
services: {
coordinator.enabled: false,
executor.enabled: true
}
Security Group Config:
Do you have two nodes: one with coordinator, one with executor, or you try from the same node?
Also is your “master” internal or external IP? I would recommend internal hostname
Yes, I’m trying to setup a two node system, one coordinator and one executor. These are both Amazon Linux instances with internal IP’s only.
Could you see /opt/dremio/log/server.log on executor node if it can connect to coordinator? Ports you listed are not the only ones needed, but I would assume internally it should not be an issue. E.g. executor should be able to connect to port 2181 (Zookeeper) on coordinator.
Yep opening port 2181 worked. And it was clearly documented here: https://docs.dremio.com/deployment/system-requirements.html#network
Thanks!
Hugo
I am trying to reproduce your environment, but I am receiving the error: No executors currently available.
I can access the web app correctly.
Tested on Java 1.7 and 1.8 (Oracle)
Coordinator conf:
master: {
name: 127.0.0.1,
port: 45678
}
paths: {
local: "/data/dremio"
}
services: {
coordinator: {
enabled: true,
web: {
enabled: true,
port: 80
}
},
executor.enabled: false
}
Executor conf:
master: {
name: coordinator private ip,
port: 45678
}
paths: {
local: "/data/dremio"
}
services: {
coordinator.enabled: false,
executor.enabled: true
}
I can telnet from executor to coordinator on port 45678 and 2181.
My Security Groups are:
Cordinator:
Executor:
Could you change master to internal hostname/ip of itself instead of 127.0.0.1?
I tried both internal name (dremio-coordinator) and internal ip, and both didn’t work…
The executor’s log shows in both cases:
2017-08-07 15:20:46,885 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to RECONNECTED
2017-08-07 15:20:54,440 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to RECONNECTED
2017-08-07 15:21:01,839 [main] INFO c.d.d.s.exec.MasterStatusListener - Waiting for master COORDINATOR-INTERNAL_IP:45678
The “COORDINATOR-INTERNAL_IP” is the internal ip on coordinator, I can telnet from executor to it on ports 45678 and 2181.
I can not telnet from coordinator to executor on port 2181 and 45678, the coordinator need to reach the executors on those ports?
Not really. I wonder what logs on executor say in terms of trying connecting back to coordinator/zookeeper
I replaced the coordinator IP for X.X.X.X
Executor server.out
Tue Aug 8 11:24:28 BRT 2017 Starting dremio on dremio-executor-01
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 59758
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 59758
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
Executor server.log
2017-08-08 11:24:30,966 [main] INFO com.dremio.exec.util.GuavaPatcher - Google's Stopwatch patched for old HBase Guava version.
2017-08-08 11:24:30,990 [main] INFO com.dremio.exec.util.GuavaPatcher - Google's Closeables patched for old HBase Guava version.
2017-08-08 11:24:31,145 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 139ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
2017-08-08 11:24:31,587 [main] INFO c.d.common.scanner.BuildTimeScan - Loaded prescanned packages [com.dremio.exec.store, com.dremio.service.jobs, com.dremio.service.accelerator, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.service.namespace, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.service.jobs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.exec.store, com.dremio.service.jobs, com.dremio.service.accelerator, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.storage, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.users, com.dremio.service.namespace, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users] from locations [jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json]
2017-08-08 11:24:32,564 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.plugins.s3.store, com.dremio.service.namespace, com.dremio.provision.yarn.service, org.apache.hadoop.hive, org.apache.hadoop.hive, com.dremio.exec.fn.hive, com.dremio.plugins.mongo, com.dremio.service.jobs, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.extras.plugins.elastic, com.dremio.plugins.mongo, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.hbase, com.dremio.exec.expr.fn.impl.conv, com.dremio.exec.store.jdbc, com.dremio.plugins.elastic, com.dremio.service.accelerator, com.dremio.exec.store, com.dremio.exec.store.jdbc, com.dremio.service.users, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.extra.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot] in locations [jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/] took 973ms
2017-08-08 11:24:32,606 [main] INFO com.dremio.dac.daemon.DACDaemon - This node is not master, waiting on X.X.X.X. This node acts as an executor.
2017-08-08 11:24:32,644 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 28ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
2017-08-08 11:24:32,987 [main] INFO c.d.s.fabric.FabricServiceImpl - fabric service has 104857600 bytes reserved
2017-08-08 11:24:33,000 [main] INFO c.dremio.dac.daemon.DACDaemonModule - Internal user/group service is configured.
2017-08-08 11:24:33,377 [main] INFO c.d.s.coordinator.zk.ZKClusterClient - Starting ZKClusterClient
2017-08-08 11:24:33,386 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to CONNECTED
2017-08-08 11:24:33,448 [main] INFO c.d.s.c.zk.ZKClusterCoordinator - ZKClusterCoordination is up
2017-08-08 11:24:33,458 [main] INFO c.d.s.coordinator.zk.ZKClusterClient - Starting ZKClusterClient
2017-08-08 11:24:33,461 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to CONNECTED
2017-08-08 11:24:33,470 [main] INFO c.d.s.c.zk.ZKClusterCoordinator - ZKClusterCoordination is up
2017-08-08 11:24:33,470 [main] INFO c.d.d.s.exec.MasterStatusListener - Starting MasterStatusListener with master node X.X.X.X:45678
2017-08-08 11:24:33,471 [main] INFO c.d.d.s.exec.MasterStatusListener - MasterStatusListener is up X.X.X.X:45678
2017-08-08 11:24:33,471 [main] INFO c.d.d.s.exec.MasterStatusListener - Waiting for master X.X.X.X:45678
2017-08-08 11:25:33,474 [main] INFO c.d.d.s.exec.MasterStatusListener - Waiting for master X.X.X.X:45678What do coordinator logs show?
Coordinator server.out
Tue Aug 8 11:54:48 BRT 2017 Starting dremio on dremio-coordinator
core file size (blocks, -c) 0
data seg size (kbytes, -d) unlimited
scheduling priority (-e) 0
file size (blocks, -f) unlimited
pending signals (-i) 59758
max locked memory (kbytes, -l) 64
max memory size (kbytes, -m) unlimited
open files (-n) 1024
pipe size (512 bytes, -p) 8
POSIX message queues (bytes, -q) 819200
real-time priority (-r) 0
stack size (kbytes, -s) 8192
cpu time (seconds, -t) unlimited
max user processes (-u) 59758
virtual memory (kbytes, -v) unlimited
file locks (-x) unlimited
11:54:57,769 |-INFO in LogbackRequestLog - Will use configuration resource [/logback-access.xml]
11:54:57,775 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - debug attribute not set
11:54:57,777 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - About to instantiate appender of type [ch.qos.logback.core.rolling.RollingFileAppender]
11:54:57,777 |-INFO in ch.qos.logback.core.joran.action.AppenderAction - Naming appender as [access-text]
11:54:57,778 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use gz compression
11:54:57,778 |-INFO in c.q.l.core.rolling.TimeBasedRollingPolicy - Will use the pattern /opt/dremio/log/archive/access.%d{yyyy-MM-dd}.log for the active file
11:54:57,778 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - The date pattern is 'yyyy-MM-dd' from file name pattern '/opt/dremio/log/archive/access.%d{yyyy-MM-dd}.log.gz'.
11:54:57,778 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Roll-over at midnight.
11:54:57,778 |-INFO in c.q.l.core.rolling.DefaultTimeBasedFileNamingAndTriggeringPolicy - Setting initial period to Tue Aug 08 11:54:57 BRT 2017
11:54:57,778 |-INFO in ch.qos.logback.core.joran.action.NestedComplexPropertyIA - Assuming default type [ch.qos.logback.access.PatternLayoutEncoder] for [encoder] property
11:54:57,788 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[access-text] - Active log file name: /opt/dremio/log/access.log
11:54:57,788 |-INFO in ch.qos.logback.core.rolling.RollingFileAppender[access-text] - File property is set to [/opt/dremio/log/access.log]
11:54:57,788 |-INFO in ch.qos.logback.core.joran.action.AppenderRefAction - Attaching appender named [access-text] to null
11:54:57,788 |-INFO in ch.qos.logback.access.joran.action.ConfigurationAction - End of configuration.
11:54:57,788 |-INFO in ch.qos.logback.access.joran.JoranConfigurator@7d599553 - Registering current configuration as safe fallback point
Dremio Daemon Started as masterCoordinator server.log
2017-08-08 11:54:50,045 [main] INFO com.dremio.exec.util.GuavaPatcher - Google's Stopwatch patched for old HBase Guava version.
2017-08-08 11:54:50,080 [main] INFO com.dremio.exec.util.GuavaPatcher - Google's Closeables patched for old HBase Guava version.
2017-08-08 11:54:50,217 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 125ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
2017-08-08 11:54:50,607 [main] INFO c.d.common.scanner.BuildTimeScan - Loaded prescanned packages [com.dremio.exec.store, com.dremio.service.jobs, com.dremio.service.accelerator, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.service.namespace, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.service.jobs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.exec.store, com.dremio.service.jobs, com.dremio.service.accelerator, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.namespace, com.dremio.storage, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.service.users, com.dremio.service.namespace, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.service.users] from locations [jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/META-INF/dremio-module-scan/registry.json]
2017-08-08 11:54:51,600 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.plugins.s3.store, com.dremio.service.namespace, com.dremio.provision.yarn.service, org.apache.hadoop.hive, org.apache.hadoop.hive, com.dremio.exec.fn.hive, com.dremio.plugins.mongo, com.dremio.service.jobs, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.extras.plugins.elastic, com.dremio.plugins.mongo, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.hbase, com.dremio.exec.expr.fn.impl.conv, com.dremio.exec.store.jdbc, com.dremio.plugins.elastic, com.dremio.service.accelerator, com.dremio.exec.store, com.dremio.exec.store.jdbc, com.dremio.service.users, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.extra.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot] in locations [jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/] took 989ms
2017-08-08 11:54:51,642 [main] INFO com.dremio.dac.daemon.DACDaemon - Dremio daemon write path: /data/dremio
2017-08-08 11:54:51,645 [main] INFO com.dremio.dac.daemon.DACDaemon - This node is the master node, X.X.X.X. This node acts as a coordinator.
2017-08-08 11:54:51,689 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 39ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
2017-08-08 11:54:51,926 [main] INFO c.d.s.fabric.FabricServiceImpl - fabric service has 104857600 bytes reserved
2017-08-08 11:54:51,973 [main] INFO c.dremio.dac.daemon.DACDaemonModule - Internal user/group service is configured.
2017-08-08 11:54:52,252 [main] INFO com.dremio.dac.daemon.ZkServer - Starting Zookeeper
2017-08-08 11:54:53,259 [main] INFO com.dremio.dac.daemon.ZkServer - Zookeeper is up at localhost:2181
2017-08-08 11:54:53,373 [main] INFO c.d.s.coordinator.zk.ZKClusterClient - Starting ZKClusterClient
2017-08-08 11:54:53,389 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to CONNECTED
2017-08-08 11:54:53,450 [main] INFO c.d.s.c.zk.ZKClusterCoordinator - ZKClusterCoordination is up
2017-08-08 11:54:53,451 [main] INFO c.d.s.coordinator.zk.ZKClusterClient - Starting ZKClusterClient
2017-08-08 11:54:53,453 [Curator-ConnectionStateManager-0] INFO c.d.s.coordinator.zk.ZKClusterClient - ZK connection state changed to CONNECTED
2017-08-08 11:54:53,462 [main] INFO c.d.s.c.zk.ZKClusterCoordinator - ZKClusterCoordination is up
2017-08-08 11:54:53,462 [main] INFO c.d.d.s.exec.MasterStatusListener - Starting MasterStatusListener with master node X.X.X.X:45678
2017-08-08 11:54:53,462 [main] INFO c.d.d.s.exec.MasterStatusListener - MasterStatusListener is up X.X.X.X:45678
2017-08-08 11:54:53,521 [main] INFO c.d.services.fabric.FabricServer - [FABRIC]: Server started on port 45678.
2017-08-08 11:54:53,524 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider
2017-08-08 11:54:55,071 [main] INFO c.d.datastore.LocalKVStoreProvider - LocalKVStoreProvider is up
2017-08-08 11:54:55,090 [main] INFO c.d.sabot.rpc.user.UserRPCServer - [USER]: Server started on port 31010.
2017-08-08 11:54:55,091 [main] INFO c.dremio.exec.server.ContextService - IFaces [dremio-coordinator/127.0.0.1] bound to the host: dremio-coordinator
2017-08-08 11:54:56,262 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.plugins.s3.store, com.dremio.service.namespace, com.dremio.provision.yarn.service, org.apache.hadoop.hive, org.apache.hadoop.hive, com.dremio.exec.fn.hive, com.dremio.plugins.mongo, com.dremio.service.jobs, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.extras.plugins.elastic, com.dremio.plugins.mongo, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.hbase, com.dremio.exec.expr.fn.impl.conv, com.dremio.exec.store.jdbc, com.dremio.plugins.elastic, com.dremio.service.accelerator, com.dremio.exec.store, com.dremio.exec.store.jdbc, com.dremio.service.users, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.extra.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical.PlannerSettings, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot] in locations [jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/] took 703ms
2017-08-08 11:54:56,607 [main] INFO c.d.e.e.f.FunctionImplementationRegistry - Function registry loaded. 1375 functions loaded in 1352 ms.
2017-08-08 11:54:56,611 [main] WARN c.d.exec.compile.JDKClassCompiler - JDK Java compiler not available - probably you're running Dremio with a JRE and not a JDK
2017-08-08 11:54:56,706 [main] INFO c.d.exec.store.StoragePluginStarter - Starting storage plugins.
2017-08-08 11:54:57,087 [main] INFO c.d.exec.store.StoragePluginStarter - Storage plugins startup successful.
Total Time: 380ms
Successes: 8
Pending: 0
Failures: 0
Specific Plugin Results
$scratch: success (379ms)
__accelerator: success (378ms)
__datasetDownload: success (374ms)
__home: success (377ms)
__jobResultsStore: success (377ms)
__logs: success (376ms)
__materialization: success (369ms)
__support: success (368ms)
2017-08-08 11:54:57,151 [main] INFO c.d.s.s.LocalSchedulerService - SchedulerService is up
2017-08-08 11:54:57,151 [main] INFO c.d.service.jobs.LocalJobsService - Starting JobsService
2017-08-08 11:54:57,191 [main] INFO c.d.service.jobs.LocalJobsService - JobsService is up
2017-08-08 11:54:57,214 [main] INFO c.d.s.a.AccelerationServiceImpl - Starting acceleration service
2017-08-08 11:54:57,389 [main] INFO c.d.s.a.AccelerationServiceImpl - Acceleration service is up
2017-08-08 11:54:57,389 [main] INFO c.d.p.s.ProvisioningServiceImpl - Starting provisioning service
2017-08-08 11:54:57,420 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 26ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-s3-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive-exec-shaded-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-datastore-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-hive-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-elasticsearch-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-mongo-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-scheduler-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hbase-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-plugin-jdbc-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-extra-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-1.0.8-201707190805180330-27f36e1.jar!/sabot-module.conf
2017-08-08 11:54:57,515 [main] INFO c.d.exec.server.NodeRegistration - Starting NodeRegistration for dremio-coordinator:31010
2017-08-08 11:54:57,541 [main] INFO c.d.exec.server.NodeRegistration - NodeRegistration is up for dremio-coordinator:31010
2017-08-08 11:54:58,408 [main] INFO c.d.d.s.TimingApplicationEventListener - ApplicationEventListener.onEvent INITIALIZATION_START
2017-08-08 11:54:59,381 [main] INFO c.d.d.s.TimingApplicationEventListener - ApplicationEventListener.onEvent INITIALIZATION_APP_FINISHED
2017-08-08 11:54:59,382 [main] INFO c.d.d.s.TimingApplicationEventListener - ApplicationEventListener.onEvent INITIALIZATION_FINISHED
2017-08-08 11:54:59,407 [main] INFO com.dremio.dac.server.WebServer - Started on http://localhost:80
2017-08-08 11:55:24,484 [26762f24-7dda-deb5-e5e4-8f5b5b964b00:foreman] WARN c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal. Should be fixed once decimal is implemented.
2017-08-08 11:55:24,555 [26762f24-7dda-deb5-e5e4-8f5b5b964b00:foreman] INFO c.d.e.p.s.h.c.ExecutionPlanCreator - User Error Occurred [ErrorId: 1f218c52-7539-47ab-b440-3ed777464e0f]
com.dremio.common.exceptions.UserException: No executors currently available.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:622) ~[dremio-common-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.planner.sql.handlers.commands.ExecutionPlanCreator.getExecutionPlan(ExecutionPlanCreator.java:58) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan(HandlerToExec.java:68) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:285) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_80]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_80]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_80]
2017-08-08 11:55:24,591 [26762f24-7dda-deb5-e5e4-8f5b5b964b00:foreman] INFO query.logger - {"queryId":"26762f24-7dda-deb5-e5e4-8f5b5b964b00","schema":"","queryText":"select\n 'green' as status,\n nodes.hostname name,\n nodes.ip_address ip,\n nodes.fabric_port port,\n cpu cpu,\n memory memory \nfrom\n sys.nodes,\n (select\n hostname,\n fabric_port,\n sum(cast(cpuTime as float) / cores) cpu \n from\n sys.threads \n group by\n hostname,\n fabric_port) cpu,\n (select\n hostname,\n fabric_port,\n direct_current * 100.0 / direct_max as memory \n from\n sys.memory) memory \nwhere\n nodes.hostname = cpu.hostname \n and nodes.fabric_port = cpu.fabric_port \n and nodes.hostname = memory.hostname \n and nodes.fabric_port = memory.fabric_port \norder by\n name,\n port","start":1502204123155,"finish":1502204124555,"outcome":"FAILED","username":"bernardelli","commandDescription":"<unknown>"}
2017-08-08 11:56:24,376 [26762ee7-aacb-32ed-a23c-f08b6f87a700:foreman] WARN c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal. Should be fixed once decimal is implemented.
2017-08-08 11:56:24,383 [26762ee7-aacb-32ed-a23c-f08b6f87a700:foreman] INFO c.d.e.p.s.h.c.ExecutionPlanCreator - User Error Occurred [ErrorId: 99631005-ea8e-4be8-9d2c-49712e24d7ee]
com.dremio.common.exceptions.UserException: No executors currently available.
at com.dremio.common.exceptions.UserException$Builder.build(UserException.java:622) ~[dremio-common-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.planner.sql.handlers.commands.ExecutionPlanCreator.getExecutionPlan(ExecutionPlanCreator.java:58) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan(HandlerToExec.java:68) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at com.dremio.exec.work.foreman.AttemptManager.run(AttemptManager.java:285) [dremio-sabot-kernel-1.0.8-201707190805180330-27f36e1.jar:1.0.8-201707190805180330-27f36e1]
at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source) [na:1.7.0_80]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source) [na:1.7.0_80]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_80]
2017-08-08 11:56:24,384 [26762ee7-aacb-32ed-a23c-f08b6f87a700:foreman] INFO query.logger - {"queryId":"26762ee7-aacb-32ed-a23c-f08b6f87a700","schema":"","queryText":"select\n 'green' as status,\n nodes.hostname name,\n nodes.ip_address ip,\n nodes.fabric_port port,\n cpu cpu,\n memory memory \nfrom\n sys.nodes,\n (select\n hostname,\n fabric_port,\n sum(cast(cpuTime as float) / cores) cpu \n from\n sys.threads \n group by\n hostname,\n fabric_port) cpu,\n (select\n hostname,\n fabric_port,\n direct_current * 100.0 / direct_max as memory \n from\n sys.memory) memory \nwhere\n nodes.hostname = cpu.hostname \n and nodes.fabric_port = cpu.fabric_port \n and nodes.hostname = memory.hostname \n and nodes.fabric_port = memory.fabric_port \norder by\n name,\n port","start":1502204183976,"finish":1502204184383,"outcome":"FAILED","username":"bernardelli","commandDescription":"<unknown>"}Feels like the above could be a potential issue, though not sure, but definitely some networking.
Now I can connect, but I could not understand why.
On my executor’s /etc/hosts I added line:
X.X.X.X dremio-coordinator
And I changed the executor config to:
master: {
name: dremio-coordinator,
port: 45678
}
...
Where X.X.X.X is the coordinator’s internal IP. And now the executor can connect to the coordinator.
Best not to use IPs, but rather internal hostnames