I have an S3 bucket which uses hive partitioning key naming schema where partitioning prefixes all start with an underscore in order to avoid clashes with actual data columns.
Given this example:
I have files within the same bucket “my-bucket”:
When I now navigate to the
data-sourceto a physical dataset Dremio gives me an error:
No files were found.
With 13.1.0 it worked but after upgrading to 14.0.0 it stopped working.
In the release notes for version 3.0 I discovered a fixed issue which says:
Format previews did not work when a directory has ‘hidden’ files (files starting with an underscore in the file name).
Resolved by ignoring period and underscores in files when performing format previews.
Do you think these might be related?
Does anybody else face the same problem and/or know a solution to this problem?