While appending to file: /var/lib/dremio/db/catalog/002798.sst: No space left on device


My servers /var/lib/dremio/db/catalog directory is full and hundreds of GB files in there. Most disk consumed files are xxxxx.log files (002977.log). When I deleted all *.log files and restarted dremio service, all my operations like create user and create spaces, views reverted and no users, spaces and views I created before.

What is this *.log and *.sst files? How should I manage this *.log files to free up space?

Also, Directory “/var/lib/dremio/cm/fs” takes huge space of disk. What information is stored in there and how should we manage it?

Thanks, Dalai


The .log and .sst files are the Dremio metadata database and you should NEVER delete anything from there. What is the size of /var/lib/dremio/

df -h /var/lib/dremio/

“/var/lib/dremio/cm/fs” contains C3 files so Dremio can read of local instead of your distributed store once a dataset is queried once, it improves performance


I wrote this question Disk management and db/catalog files can you give a help there?