While appending to file: /var/lib/dremio/db/catalog/002798.sst: No space left on device

1

Hi,

My servers /var/lib/dremio/db/catalog directory is full and hundreds of GB files in there. Most disk consumed files are xxxxx.log files (002977.log). When I deleted all *.log files and restarted dremio service, all my operations like create user and create spaces, views reverted and no users, spaces and views I created before.

What is this *.log and *.sst files? How should I manage this *.log files to free up space?

Also, Directory “/var/lib/dremio/cm/fs” takes huge space of disk. What information is stored in there and how should we manage it?

Thanks, Dalai

2

@Dalai

The .log and .sst files are the Dremio metadata database and you should NEVER delete anything from there. What is the size of /var/lib/dremio/

df -h /var/lib/dremio/

“/var/lib/dremio/cm/fs” contains C3 files so Dremio can read of local instead of your distributed store once a dataset is queried once, it improves performance

3

@balaji.ramaswamy

I wrote this question Disk management and db/catalog files can you give a help there?