How to clean up /var/lib/dremio/cm/fs folder (Dremio Software)

1

I’m using Dremio Software, installing on a local CentOS server. And the Dremio folder is taking all of my Namenode space

(base) [root@namenode ~]# du -ah --max-depth 1 /var/lib/dremio/cm/ | sort -rn
504M /var/lib/dremio/cm/db
348G /var/lib/dremio/cm/fs
348G /var/lib/dremio/cm/

How to clean the “fs” folder? Clearly, it’s not the actual “data” itself, since my data are on HDFS, which is another folder.
So what are those? If it’s not data, then I can delete it, right?
ll /var/lib/dremio/cm/fs | head -n 5
total 12328
drwxr-xr-x 2 root root 290816 May 14 09:17 000000
drwxr-xr-x 2 root root 319488 May 14 09:17 000001
drwxr-xr-x 2 root root 282624 May 14 09:17 000002
drwxr-xr-x 2 root root 307200 May 14 09:16 000003

Is there any way to safely clean these? Is there a command like “dremio clean cache” or something?
Or do I have to use rm -rf? Can I just remove the old FS file and keep the recent one?
I’m using Dremio Software 24.0.0

2

try set exec.cachemanager.fs.space_usage_low_threshold_pct 30 in support page

3

Can I just rm -rf it for now? My namenode is nearly full, and the namenode is the brain of our cluster; if it died, then everything would die!

4

@quangbilly79 Before you delete the files, turn off C3 by disabling “Enable local caching when possible” under advanced tab on the source settings and then delete the files. Why are the C3 files filling your NN disks. They should be on your executors local only if they are SSD or NVMe. If they are not then we rather have them turned off. Do you use reflections? If yes then we need to turn off C3 for relfections too

1 Like
5

Tks I found it! The default is 100%, which consume all of my node disk space lol!!!

image
image914×554 33.3 KB