"No files detected or unable to read file format" on empty dirs in object store

1

I get the error “No files detected or unable to read file format with selected option” when Dremio reaches an empty “directory” within an S3-like object store source. Is there a setting that would allow Dremio to just ignore that directory and continue on?

v4.2.1

2

Some more info:

I added an empty file in the empty leaf directories thinking that it might be enough to get Dremio to continue, but now I get “The top level of your document must either be a single array of maps or a set of white space delimited maps.”

The empty file is a .json file with just this:

[{}]

I also tried with this:

[]

and got the same error.

3

@noah

I am able to promote the same file, see attached

empty.json.zip (561 Bytes)

4

@balaji.ramaswamy
Can dremio ignore a fixed folder name over configuration when query?

5

@koolay If there is a folder under a promoted folder, you can use filter clauses to ignore. Not sure I understand your question completely

6

@balaji.ramaswamy

I want to query the table path of delta lake.
However it does not work because that there is a subfolder within the table path.

7

@koolay

We are not yet there to support DELTA LAKE format, we will be supporting soon

Thanks
Bali