# Problem on create large dataset

**URL:** <https://community.dremio.com/t/problem-on-create-large-dataset/5635>\
**Category:** Uncategorized\
**Created:** [May 28, 2020, 6:35pm UTC](https://community.dremio.com/t/problem-on-create-large-dataset/5635 "2020-05-28T18:35:01Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![eduardoslopes](https://avatars.discourse-cdn.com/v4/letter/e/a183cd/32.png) [@eduardoslopes](https://community.dremio.com/u/eduardoslopes)\
**Post date:** [May 28, 2020, 6:35pm UTC](https://community.dremio.com/t/problem-on-create-large-dataset/5635/1 "2020-05-28T18:35:02Z")

</div>

Hi guys,

I am trying to create a dataset from a S3 folder containing a lot of files each of them containing data from a hour, but the request that get the structure of data is not responding.

---

<div class="post-metadata">

**Author:** ![mcotter](https://avatars.discourse-cdn.com/v4/letter/m/7993a0/32.png) [@mcotter](https://community.dremio.com/u/mcotter)\
**Post date:** [May 28, 2020, 7:09pm UTC](https://community.dremio.com/t/problem-on-create-large-dataset/5635/2 "2020-05-28T19:09:35Z")

</div>

Have you tried re-designing your file structure and using the power of Dremio’s partitioned datasets

[https://docs.dremio.com/data-sources/files-and-directories.html](https://docs.dremio.com/data-sources/files-and-directories.html)

(towards the bottom). We do this at our site on a NAS

---

<div class="post-metadata">

**Author:** ![eduardoslopes](https://avatars.discourse-cdn.com/v4/letter/e/a183cd/32.png) [@eduardoslopes](https://community.dremio.com/u/eduardoslopes)\
**Post date:** [May 28, 2020, 7:58pm UTC](https://community.dremio.com/t/problem-on-create-large-dataset/5635/3 "2020-05-28T19:58:17Z")

</div>

@mcotter,

Thanks a lot. I will try this.