Problem on create large dataset

1

Hi guys,

I am trying to create a dataset from a S3 folder containing a lot of files each of them containing data from a hour, but the request that get the structure of data is not responding.

2

Have you tried re-designing your file structure and using the power of Dremio’s partitioned datasets

https://docs.dremio.com/data-sources/files-and-directories.html

(towards the bottom). We do this at our site on a NAS

3

@mcotter,

Thanks a lot. I will try this.