DATA_WRITE ERROR: Failed to spill to disk. Please check space availability

1

I got such error during reflection creation. It seems that Dremio always gets file:///local/0/opt/hadoop-mapr/dremio as spill location.

DATA_WRITE ERROR: Failed to spill to disk. Please check space availability

spill id esort-238af2e6-990f-515c-cab3-90b1e4913600.0.0.8

all spill locations [[file:///local/0/opt/hadoop-mapr/dremio](file:///\\local\0\opt\hadoop-mapr\dremio)]

for sort spilling spill id esort-238af2e6-990f-515c-cab3-90b1e4913600.0.0.8

Caller sort spilling

SqlOperatorImpl EXTERNAL_SORT

Location 0:0:8

Fragment 0:0

I have configured it to a new location in dremio.conf and restarted Dremio but no use.

paths: {
  # the local path for dremio to store data.
  local: "xxxxx/dremio/metadata"

  spilling: ["xxxxx/dremio/metadata/spill"]

  # the distributed path Dremio data including job results, downloads, uploads, etc
  dist: "maprfs://xxxx/data/dremio/diststore"
}
2

I only updated dremio.conf on one coordinator. Do we have to update all executors?

3

@pisces312, yes, you should configure the spill directory on each executor by changing its dremio.conf. Unless you are deploying your cluster using YARN, in which case you do this through the web UI provisioning window.

4

Thanks @ben . So can we say that paths.spilling is not used in coordinator even we set a value in dremio.conf?

5

Spilling occurs during execution on the executor nodes for some operations, and the Dremio service running on each picks up its configuration from its local dremio.conf.

I do not believe any spilling occurs on the coordinator node during query planning, so the coordinator may just ignore that part of the configuration.

6

Got it. Maybe we can make some explanations in docs for spilling configuration.