No executors are available for data with hard affinity

I’ve been successfully using Dremio for the last couple of days, but now I keep getting this error whenever I try to view any data:

“No executors are available for data with hard affinity. You may consider using “registration.publish-host” property if your network rules change.”

I’ve tried many of the suggestions from this topic to no avail. Under “node activity” it is showing 1 node with my local machine IP.

Dremio.conf:

master: {
  # the name of the master server. If this node matches the name, it starts the master service
  name: localhost,
  port: 45678
}

paths: {
  # the local path for dremio to store data.
 local: ${HOME}"/Library/Dremio/data"

  # the distributed path Dremio data including job results, downloads, uploads, etc
  #dist: "pdfs://"${paths.local}"/pdfs"
}

services: {
  coordinator.enabled: true,
  coordinator.auto-upgrade: true
  executor.enabled: true
}

#allows network to change and dremio to keep working
registration.publish-host: "localhost"

Server.log:

2017-10-04 09:41:45,179 [262af0b5-baa4-af8e-07ec-a810014e8e00:foreman] INFO  query.logger - {"queryId":"262af0b5-baa4-af8e-07ec-a810014e8e00","schema":"","queryText":"SELECT *\nFROM \"@dylanstark\".CampaignMember","start":1507135305132,"finish":1507135305178,"outcome":"FAILED","username":"dylanstark","commandDescription":"<unknown>"}
2017-10-04 09:44:19,472 [262af01c-73e6-7c98-9a31-f1ec592a4200:foreman] WARN  c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal.  Should be fixed once decimal is implemented.
2017-10-04 09:44:19,632 [262af01c-73e6-7c98-9a31-f1ec592a4200:foreman] INFO  c.d.s.fabric.FabricConnectionManager - [FABRIC]: No connection active, opening new connection to localhost:45678.
2017-10-04 09:44:19,706 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262af01c-73e6-7c98-9a31-f1ec592a4200:2:0
2017-10-04 09:44:19,725 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262af01c-73e6-7c98-9a31-f1ec592a4200:1:0
2017-10-04 09:44:19,730 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262af01c-73e6-7c98-9a31-f1ec592a4200:0:0
2017-10-04 09:44:19,731 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262af01c-73e6-7c98-9a31-f1ec592a4200:3:0
2017-10-04 09:44:19,736 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262af01c-73e6-7c98-9a31-f1ec592a4200:4:0
2017-10-04 09:44:20,085 [e4 - 262af01c-73e6-7c98-9a31-f1ec592a4200:frag:4:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262af01c-73e6-7c98-9a31-f1ec592a4200:4:0: State to report: FINISHED
2017-10-04 09:44:20,085 [e3 - 262af01c-73e6-7c98-9a31-f1ec592a4200:frag:3:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262af01c-73e6-7c98-9a31-f1ec592a4200:3:0: State to report: FINISHED
2017-10-04 09:44:20,119 [e0 - 262af01c-73e6-7c98-9a31-f1ec592a4200:frag:2:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262af01c-73e6-7c98-9a31-f1ec592a4200:2:0: State to report: FINISHED
2017-10-04 09:44:20,136 [e1 - 262af01c-73e6-7c98-9a31-f1ec592a4200:frag:1:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262af01c-73e6-7c98-9a31-f1ec592a4200:1:0: State to report: FINISHED
2017-10-04 09:44:20,267 [e2 - 262af01c-73e6-7c98-9a31-f1ec592a4200:frag:0:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262af01c-73e6-7c98-9a31-f1ec592a4200:0:0: State to report: FINISHED
2017-10-04 09:44:20,269 [FABRIC-rpc-event-queue] INFO  query.logger - {"queryId":"262af01c-73e6-7c98-9a31-f1ec592a4200","schema":"","queryText":"select\n   'green' as status,\n   nodes.hostname name,\n   nodes.ip_address ip,\n   nodes.fabric_port port,\n   cpu cpu,\n   memory memory \nfrom\n   sys.nodes,\n   (select\n      hostname,\n      fabric_port,\n      sum(cast(cpuTime as float) / cores) cpu \n   from\n      sys.threads \n   group by\n      hostname,\n      fabric_port) cpu,\n   (select\n      hostname,\n      fabric_port,\n      direct_current * 100.0 / direct_max as memory \n   from\n      sys.memory) memory  \nwhere\n   nodes.hostname = cpu.hostname \n   and nodes.fabric_port = cpu.fabric_port  \n   and nodes.hostname = memory.hostname \n   and nodes.fabric_port = memory.fabric_port \norder by\n   name,\n   port","start":1507135459137,"finish":1507135460269,"outcome":"COMPLETED","username":"dylanstark","commandDescription":"execute; query"}
2017-10-04 09:45:21,291 [262aefde-7f72-2a50-e056-b01da49afe00:foreman] WARN  c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal.  Should be fixed once decimal is implemented.
2017-10-04 09:45:21,306 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefde-7f72-2a50-e056-b01da49afe00:1:0
2017-10-04 09:45:21,307 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefde-7f72-2a50-e056-b01da49afe00:0:0
2017-10-04 09:45:21,308 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefde-7f72-2a50-e056-b01da49afe00:3:0
2017-10-04 09:45:21,309 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefde-7f72-2a50-e056-b01da49afe00:2:0
2017-10-04 09:45:21,315 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefde-7f72-2a50-e056-b01da49afe00:4:0
2017-10-04 09:45:21,359 [e4 - 262aefde-7f72-2a50-e056-b01da49afe00:frag:4:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefde-7f72-2a50-e056-b01da49afe00:4:0: State to report: FINISHED
2017-10-04 09:45:21,365 [e2 - 262aefde-7f72-2a50-e056-b01da49afe00:frag:3:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefde-7f72-2a50-e056-b01da49afe00:3:0: State to report: FINISHED
2017-10-04 09:45:21,377 [e3 - 262aefde-7f72-2a50-e056-b01da49afe00:frag:2:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefde-7f72-2a50-e056-b01da49afe00:2:0: State to report: FINISHED
2017-10-04 09:45:21,388 [e0 - 262aefde-7f72-2a50-e056-b01da49afe00:frag:1:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefde-7f72-2a50-e056-b01da49afe00:1:0: State to report: FINISHED
2017-10-04 09:45:21,410 [e1 - 262aefde-7f72-2a50-e056-b01da49afe00:frag:0:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefde-7f72-2a50-e056-b01da49afe00:0:0: State to report: FINISHED
2017-10-04 09:45:21,412 [FABRIC-rpc-event-queue] INFO  query.logger - {"queryId":"262aefde-7f72-2a50-e056-b01da49afe00","schema":"","queryText":"select\n   'green' as status,\n   nodes.hostname name,\n   nodes.ip_address ip,\n   nodes.fabric_port port,\n   cpu cpu,\n   memory memory \nfrom\n   sys.nodes,\n   (select\n      hostname,\n      fabric_port,\n      sum(cast(cpuTime as float) / cores) cpu \n   from\n      sys.threads \n   group by\n      hostname,\n      fabric_port) cpu,\n   (select\n      hostname,\n      fabric_port,\n      direct_current * 100.0 / direct_max as memory \n   from\n      sys.memory) memory  \nwhere\n   nodes.hostname = cpu.hostname \n   and nodes.fabric_port = cpu.fabric_port  \n   and nodes.hostname = memory.hostname \n   and nodes.fabric_port = memory.fabric_port \norder by\n   name,\n   port","start":1507135521110,"finish":1507135521412,"outcome":"COMPLETED","username":"dylanstark","commandDescription":"execute; query"}
2017-10-04 09:46:23,173 [262aefa1-7e54-0280-38de-218222de1300:foreman] WARN  c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal.  Should be fixed once decimal is implemented.
2017-10-04 09:46:23,276 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefa1-7e54-0280-38de-218222de1300:2:0
2017-10-04 09:46:23,277 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefa1-7e54-0280-38de-218222de1300:0:0
2017-10-04 09:46:23,278 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefa1-7e54-0280-38de-218222de1300:3:0
2017-10-04 09:46:23,278 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefa1-7e54-0280-38de-218222de1300:1:0
2017-10-04 09:46:23,283 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aefa1-7e54-0280-38de-218222de1300:4:0
2017-10-04 09:46:23,323 [e2 - 262aefa1-7e54-0280-38de-218222de1300:frag:4:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefa1-7e54-0280-38de-218222de1300:4:0: State to report: FINISHED
2017-10-04 09:46:23,326 [e2 - 262aefa1-7e54-0280-38de-218222de1300:frag:3:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefa1-7e54-0280-38de-218222de1300:3:0: State to report: FINISHED
2017-10-04 09:46:23,437 [e0 - 262aefa1-7e54-0280-38de-218222de1300:frag:2:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefa1-7e54-0280-38de-218222de1300:2:0: State to report: FINISHED
2017-10-04 09:46:23,443 [e3 - 262aefa1-7e54-0280-38de-218222de1300:frag:1:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefa1-7e54-0280-38de-218222de1300:1:0: State to report: FINISHED
2017-10-04 09:46:23,465 [e1 - 262aefa1-7e54-0280-38de-218222de1300:frag:0:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aefa1-7e54-0280-38de-218222de1300:0:0: State to report: FINISHED
2017-10-04 09:46:23,467 [FABRIC-rpc-event-queue] INFO  query.logger - {"queryId":"262aefa1-7e54-0280-38de-218222de1300","schema":"","queryText":"select\n   'green' as status,\n   nodes.hostname name,\n   nodes.ip_address ip,\n   nodes.fabric_port port,\n   cpu cpu,\n   memory memory \nfrom\n   sys.nodes,\n   (select\n      hostname,\n      fabric_port,\n      sum(cast(cpuTime as float) / cores) cpu \n   from\n      sys.threads \n   group by\n      hostname,\n      fabric_port) cpu,\n   (select\n      hostname,\n      fabric_port,\n      direct_current * 100.0 / direct_max as memory \n   from\n      sys.memory) memory  \nwhere\n   nodes.hostname = cpu.hostname \n   and nodes.fabric_port = cpu.fabric_port  \n   and nodes.hostname = memory.hostname \n   and nodes.fabric_port = memory.fabric_port \norder by\n   name,\n   port","start":1507135582995,"finish":1507135583467,"outcome":"COMPLETED","username":"dylanstark","commandDescription":"execute; query"}
2017-10-04 09:47:24,265 [262aef62-a87e-adbe-d623-586f6da6c600:foreman] WARN  c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal.  Should be fixed once decimal is implemented.
2017-10-04 09:47:24,280 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef62-a87e-adbe-d623-586f6da6c600:2:0
2017-10-04 09:47:24,281 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef62-a87e-adbe-d623-586f6da6c600:0:0
2017-10-04 09:47:24,281 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef62-a87e-adbe-d623-586f6da6c600:1:0
2017-10-04 09:47:24,282 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef62-a87e-adbe-d623-586f6da6c600:3:0
2017-10-04 09:47:24,284 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef62-a87e-adbe-d623-586f6da6c600:4:0
2017-10-04 09:47:24,321 [e4 - 262aef62-a87e-adbe-d623-586f6da6c600:frag:4:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef62-a87e-adbe-d623-586f6da6c600:4:0: State to report: FINISHED
2017-10-04 09:47:24,324 [e3 - 262aef62-a87e-adbe-d623-586f6da6c600:frag:3:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef62-a87e-adbe-d623-586f6da6c600:3:0: State to report: FINISHED
2017-10-04 09:47:24,331 [e0 - 262aef62-a87e-adbe-d623-586f6da6c600:frag:2:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef62-a87e-adbe-d623-586f6da6c600:2:0: State to report: FINISHED
2017-10-04 09:47:24,337 [e2 - 262aef62-a87e-adbe-d623-586f6da6c600:frag:1:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef62-a87e-adbe-d623-586f6da6c600:1:0: State to report: FINISHED
2017-10-04 09:47:24,374 [e1 - 262aef62-a87e-adbe-d623-586f6da6c600:frag:0:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef62-a87e-adbe-d623-586f6da6c600:0:0: State to report: FINISHED
2017-10-04 09:47:24,376 [FABRIC-rpc-event-queue] INFO  query.logger - {"queryId":"262aef62-a87e-adbe-d623-586f6da6c600","schema":"","queryText":"select\n   'green' as status,\n   nodes.hostname name,\n   nodes.ip_address ip,\n   nodes.fabric_port port,\n   cpu cpu,\n   memory memory \nfrom\n   sys.nodes,\n   (select\n      hostname,\n      fabric_port,\n      sum(cast(cpuTime as float) / cores) cpu \n   from\n      sys.threads \n   group by\n      hostname,\n      fabric_port) cpu,\n   (select\n      hostname,\n      fabric_port,\n      direct_current * 100.0 / direct_max as memory \n   from\n      sys.memory) memory  \nwhere\n   nodes.hostname = cpu.hostname \n   and nodes.fabric_port = cpu.fabric_port  \n   and nodes.hostname = memory.hostname \n   and nodes.fabric_port = memory.fabric_port \norder by\n   name,\n   port","start":1507135644107,"finish":1507135644375,"outcome":"COMPLETED","username":"dylanstark","commandDescription":"execute; query"}
2017-10-04 09:48:25,234 [262aef26-5d0c-1a2b-699d-7026d27b4a00:foreman] WARN  c.d.exec.planner.logical.RexToExpr - Converting exact decimal into approximate decimal.  Should be fixed once decimal is implemented.
2017-10-04 09:48:25,252 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef26-5d0c-1a2b-699d-7026d27b4a00:0:0
2017-10-04 09:48:25,252 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef26-5d0c-1a2b-699d-7026d27b4a00:2:0
2017-10-04 09:48:25,253 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef26-5d0c-1a2b-699d-7026d27b4a00:1:0
2017-10-04 09:48:25,254 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef26-5d0c-1a2b-699d-7026d27b4a00:3:0
2017-10-04 09:48:25,259 [FABRIC-rpc-event-queue] INFO  c.d.s.e.rpc.CoordToExecHandlerImpl - Received remote fragment start instruction for 262aef26-5d0c-1a2b-699d-7026d27b4a00:4:0
2017-10-04 09:48:25,305 [e4 - 262aef26-5d0c-1a2b-699d-7026d27b4a00:frag:4:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef26-5d0c-1a2b-699d-7026d27b4a00:4:0: State to report: FINISHED
2017-10-04 09:48:25,310 [e3 - 262aef26-5d0c-1a2b-699d-7026d27b4a00:frag:3:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef26-5d0c-1a2b-699d-7026d27b4a00:3:0: State to report: FINISHED
2017-10-04 09:48:25,429 [e1 - 262aef26-5d0c-1a2b-699d-7026d27b4a00:frag:2:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef26-5d0c-1a2b-699d-7026d27b4a00:2:0: State to report: FINISHED
2017-10-04 09:48:25,433 [e2 - 262aef26-5d0c-1a2b-699d-7026d27b4a00:frag:1:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef26-5d0c-1a2b-699d-7026d27b4a00:1:0: State to report: FINISHED
2017-10-04 09:48:25,458 [e0 - 262aef26-5d0c-1a2b-699d-7026d27b4a00:frag:0:0] INFO  c.d.s.e.f.FragmentStatusReporter - 262aef26-5d0c-1a2b-699d-7026d27b4a00:0:0: State to report: FINISHED
2017-10-04 09:48:25,459 [FABRIC-rpc-event-queue] INFO  query.logger - {"queryId":"262aef26-5d0c-1a2b-699d-7026d27b4a00","schema":"","queryText":"select\n   'green' as status,\n   nodes.hostname name,\n   nodes.ip_address ip,\n   nodes.fabric_port port,\n   cpu cpu,\n   memory memory \nfrom\n   sys.nodes,\n   (select\n      hostname,\n      fabric_port,\n      sum(cast(cpuTime as float) / cores) cpu \n   from\n      sys.threads \n   group by\n      hostname,\n      fabric_port) cpu,\n   (select\n      hostname,\n      fabric_port,\n      direct_current * 100.0 / direct_max as memory \n   from\n      sys.memory) memory  \nwhere\n   nodes.hostname = cpu.hostname \n   and nodes.fabric_port = cpu.fabric_port  \n   and nodes.hostname = memory.hostname \n   and nodes.fabric_port = memory.fabric_port \norder by\n   name,\n   port","start":1507135705104,"finish":1507135705459,"outcome":"COMPLETED","username":"dylanstark","commandDescription":"execute; query"}

It feels like your IP has changed.
If it changes from time to time - DHCP, VPN or other and you are working with files you uploaded into dremio see if following will help.

  1. Set in your /etc/hosts your current IP to some hostname (even “foobar” will do)
  2. use “foobar” in “registration.publish-host” setting in dremio.conf
    This way you can always remap current IP (if/when it changes) to “foobar”