# Dremio server disk management

**URL:** <https://community.dremio.com/t/dremio-server-disk-management/10045>\
**Category:** Uncategorized\
**Created:** [December 14, 2022, 2:59pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045 "2022-12-14T14:59:26Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [December 14, 2022, 2:59pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/1 "2022-12-14T14:59:26Z")

</div>

Hi!

I’ve been learning Dremio disk management and admnistrative tools. In my production environment, the coordinator node full very quicly. I already grow the vm’s sdd disk but the problem persists until I searched about _ **“dremio-admin clean -o”** _ (to purge orphaned metadata). My disk space dows from 100GB to 61GB after run command. **Now on every two day I have to run this command to retrieve disk space, this is not seems fine**.

Analysing the dremio installation directories, I discover db folder that equivalent to all space used on my disk

 ![dremio db folder 60GB](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/0/0eab4b7547da4f23f18578e6c39c8f25bf301427.png)

_ **Catalog is the largest folder in db folder, what we can about it?** _ Can we move to another place, like AWS S3?

![dremio catalog folder 59Gb](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/9/9a9849a315f2266dbf3c53311b004e5d514c4478.png)

**UPDATE:**  
I see a large amount of _.sst_, _.log_ and _LOG.old_ what are these files? Are they really necessary?  
How can I make safe delete of LOG.old.\* files?

Thanks in advance

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 22, 2022, 8:33am UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/2 "2022-12-22T08:33:08Z")

</div>

@Diego

.sst and .log files are transaction log files so please do not remove them

With the current version, most of the metadata has been moved off the local disk to the distributed store like S3, what version of Dremio are you on?

---

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [December 23, 2022, 1:52pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/3 "2022-12-23T13:52:42Z")

</div>

@balaji.ramaswamy we’re in this version on AWS Cloud

 ![dremio_version](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/2/2aa763ae882609e3714f2bdb69340b87c1f61485.png)

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 28, 2022, 3:47pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/4 "2022-12-28T15:47:45Z")

</div>

@Diego I would like to see what is using up all the space in your Catalog, can you please run the below command vi REST, need to change your authorization ID

```auto
curl --location --request GET 'localhost:9047/apiv2/kvstore/report?store=none' \
--header 'Authorization: _dremioirr8hj6qnpc3tfr3omiqvev51c' > kvstore_summary.zip

```

Also, can you please query any Parquet dataset and send me the job profile

---

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [December 28, 2022, 4:58pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/5 "2022-12-28T16:58:18Z")

</div>

@balaji.ramaswamy take these files:  
[kvstore\_summary.zip](https://community.dremio.com/uploads/short-url/a41KwMIwnC1OY4T59NDbtsIkZWZ.zip) (2,0,KB)  
[8a5b7981-fe6e-4602-bca2-191c63a1f38a.zip](https://community.dremio.com/uploads/short-url/fWQXNxKW3NIIqzdrD6Kt2qw6oPJ.zip) (20,0,KB)

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 30, 2022, 1:57pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/6 "2022-12-30T13:57:35Z")

</div>

@Diego The profiles+jobs are using up 60 GB of space. What is the total volume size where the rocksdb is stoed?

---

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [December 30, 2022, 2:51pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/7 "2022-12-30T14:51:03Z")

</div>

@balaji.ramaswamy the disk have up to 100 GB

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [December 31, 2022, 2:38pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/8 "2022-12-31T14:38:35Z")

</div>

@Diego You are using 60% of the total disk space for jobs and profiles, you have 2 options

- Decrease `jobs.max.age_in_days` to something like 14 (default is 30 days)

- Increase disk volume to account for the recover log files needed for rocksdb (say make it 150 GB)

- Can you also make sure `planner.verbose_profile` is false

Thanks  
Bali

---

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [December 31, 2022, 2:57pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/9 "2022-12-31T14:57:18Z")

</div>

@balaji.ramaswamy

- Decrease `jobs.max.age_in_days` to something like 14 (default is 30 days)  
**sure, I will to reduce to 7**

- Increase disk volume to account for the recover log files needed for rocksdb (say make it 150 GB)  
**This will be the last resource**

- Can you also make sure `planner.verbose_profile` is false  
**This is already set to false**

Thanks Bali

---

<div class="post-metadata">

**Author:** ![Diego](https://avatars.discourse-cdn.com/v4/letter/d/f04885/32.png) [@Diego](https://community.dremio.com/u/Diego)\
**Post date:** [January 6, 2023, 6:34pm UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/10 "2023-01-06T18:34:12Z")

</div>

@balaji.ramaswamy when I decrease `jobs.max.age_in_days` to 7 worked fine. Thanks again!

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [January 8, 2023, 4:50am UTC](https://community.dremio.com/t/dremio-server-disk-management/10045/11 "2023-01-08T04:50:39Z")

</div>

@Diego Glad it works, welcome
