Dremio Backup is insanely heavy

Im trying to backup my dremio server with the dremio-admin command but the volume of the backup is extremely heavy. Mainly because of profiles:

-rw-r–r-- 1 root root 47185 May 4 08:18 acceleration_materialization_backup.json
-rw-r–r-- 1 root root 512 May 4 08:18 acceleration_materialization_info.json
-rw-r–r-- 1 root root 133928484864 May 4 09:13 profiles_backup.json
-rw-r–r-- 1 root root 650 May 4 08:18 provisioning_backup.json
-rw-r–r-- 1 root root 376 May 4 08:18 provisioning_info.json
-rw-r–r-- 1 root root 2379 May 4 08:18 roles_backup.json
-rw-r–r-- 1 root root 340 May 4 08:18 roles_info.json

Before doing this backup I stopped my server and did a clean with all options and only 8 days retention for jobs and profiles :

profiles basic rocks store stats
* Estimated Number of Keys: 1157818
* Estimated Live Data Size: 19751883693
* Total SST files size: 19751883693
* Pending Compaction Bytes: 0

jobs basic rocks store stats
* Estimated Number of Keys: 1157823
* Estimated Live Data Size: 556538976
* Total SST files size: 1220500846
* Pending Compaction Bytes: 0

Any idea how can I make this work ? In the current state, I can’t do backup because the process fill a terabyte of filesystem and ultimately fail.

You should try to do cleanup :

Well… Like I said in my message, I did do a clean. With all options.
Here’s the exact command I typed : ./dremio-admin clean -o -c -i -j 8

oups, sorry for that. on my side I run the clean command option by option.

Could you try running compact again?

I also have tried to backup but its take too much time due to huge size for Catalog ,in mine cased coomand itself hangs i have triyed all coomand
./dremio-admin clean -j
./dremio-admin clean -c
./dremio-admin clean ```

nothing  is  working  all  looks hangs for  me  .


This is the same issue as you other Community Post