# Different distributed storage location for cache locations

**URL:** <https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164>\
**Category:** Uncategorized\
**Created:** [May 17, 2018, 7:01pm UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164 "2018-05-17T19:01:25Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![dfleckinger](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/dfleckinger/32/370_2.png) [@dfleckinger](https://community.dremio.com/u/dfleckinger)\
**Post date:** [May 17, 2018, 7:01pm UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/1 "2018-05-17T19:01:25Z")

</div>

Hi,  
Dremio uses paths.dist as the cache location for holding accelerator, CREATE TABLE AS tables, job result, download and upload data, as indicated in [documentation](https://docs.dremio.com/deployment/distributed-storage.html).  
I would like to be able to keep accelerator data on local dremio nodes,  
but use S3 for **downloads**** results ****scratch** and **uploads**.  
Is it possible to specify that in dremio.conf ?

Thanks

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [May 17, 2018, 7:05pm UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/2 "2018-05-17T19:05:58Z")

</div>

Please look here: [http://docs.dremio.com/advanced-administration/configuration-files.html?h=dremio.conf](http://docs.dremio.com/advanced-administration/configuration-files.html?h=dremio.conf)

Not that you can keep acceleration data completely node local, but you can use different location for acceleration data and some others

---

<div class="post-metadata">

**Author:** ![dfleckinger](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/dfleckinger/32/370_2.png) [@dfleckinger](https://community.dremio.com/u/dfleckinger)\
**Post date:** [May 18, 2018, 6:28am UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/3 "2018-05-18T06:28:09Z")

</div>

Indeed. Thanks a lot @yufeldman ! 🙂

---

<div class="post-metadata">

**Author:** ![swarren](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/swarren/32/586_2.png) [@swarren](https://community.dremio.com/u/swarren)\
**Post date:** [May 18, 2018, 6:48am UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/4 "2018-05-18T06:48:58Z")

</div>

By “node local” you mean local storage that is shared among all the nodes right? I think it would be great to use ephemeral storage on AWS servers for the acceleration cache but it’s local to a single node. Or would that work?

---

<div class="post-metadata">

**Author:** ![dfleckinger](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/dfleckinger/32/370_2.png) [@dfleckinger](https://community.dremio.com/u/dfleckinger)\
**Post date:** [May 18, 2018, 7:06am UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/5 "2018-05-18T07:06:10Z")

</div>

I understand by “node local”, as the disk space on local Dremio nodes. Dremio is using pdfs to distribute the storage over the nodes.

---

<div class="post-metadata">

**Author:** ![swarren](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/swarren/32/586_2.png) [@swarren](https://community.dremio.com/u/swarren)\
**Post date:** [May 18, 2018, 1:27pm UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/6 "2018-05-18T13:27:13Z")

</div>

> This storage plugin allows to query local filesystem metadata on remote nodes instances. However, it only allows to open and/or create file on the local filesystem.

I see where this would allow other nodes to read parquet files written locally to a node.

I got this reply when I asked this question…

> [@Is reflection storage intended to be private per node or per cluster?](https://community.dremio.com/t/is-reflection-storage-intended-to-be-private-per-node-or-per-cluster/1161/2):
>
> They should be stored on a shared storage layer, such as S3, HDFS, NAS. You can set the config here: storage area for the accelerator cache. accelerator: ${paths.dist}/accelerator See [https://docs.dremio.com/advanced-administration/configuration-files.html](https://docs.dremio.com/advanced-administration/configuration-files.html) Also, see: [Storing reflection data in S3](https://community.dremio.com/t/storing-reflection-data-in-s3/365) I think we could do a better job documenting this.

But your reply makes sense as well.

---

<div class="post-metadata">

**Author:** ![can](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/can/32/40_2.png) [@can](https://community.dremio.com/u/can)\
**Post date:** [May 18, 2018, 6:52pm UTC](https://community.dremio.com/t/different-distributed-storage-location-for-cache-locations/1164/7 "2018-05-18T18:52:53Z")

</div>

@swarren @dfleckinger hope this helps:

Dremio has 2 modes for distributed storage (reflections, downloads, etc. ):

- Use local node disks. In this mode, Dremio leverages available disk space across nodes to store data and also allow remote reads on this data across the cluster. However, we do not do replication (think HDFS) – which means if a node goes down, the reflection will be marked as invalid and will need to be refreshed.
- Use a shared storage layer (recommended for production): S3, HDFS, ADLS, etc.
