# Cannot start dremio -- rockbd problem -- missing sst files **URL:** \ **Category:** Uncategorized\ **Created:** [December 18, 2018, 1:09pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442 "2018-12-18T13:09:52Z")\ **Posts on this page:** 13\ **Page:** 1
**Author:** ![seraus](https://avatars.discourse-cdn.com/v4/letter/s/f6c823/32.png) [@seraus](https://community.dremio.com/u/seraus)\ **Post date:** [December 18, 2018, 1:09pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/1 "2018-12-18T13:09:52Z")
Hi, we are facing the following issue, when we try to restart dremio with dremio service restart dremio starts for a second then stops The log says (server.out) 2018-12-18 12:36:29,200 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider 2018-12-18 12:36:31,209 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopping LocalKVStoreProvider 2018-12-18 12:36:31,210 [main] INFO c.d.datastore.LocalKVStoreProvider - Stopped LocalKVStoreProvider Failed to complete cleanup. org.rocksdb.RocksDBException: Can’t access /009551.sst: IO error: while stat a file for size: /dremio/db/catalog/009551.sst: No such file or directory Can’t access /009552.sst: IO error: while stat a file for size: /dremio/db/catalog/009552.sst: No such file or directory Can’t access /009537.sst: IO error: while stat a file for size: /dremio/db/catalog/009537.sst: No such file or directory Can’t access /009538.sst: IO error: while stat a file for size: /dremio/db/catalog/009538.sst: No such file or directory Can’t access /009539.sst: IO error: while stat a file for size: /dremio/db/catalog/009539.sst: No such file or directory Can’t access /009540.sst: IO error: while stat a file for size: /dremio/db/catalog/009540.sst: No such file or directory Can’t access /009541.sst: IO error: while stat a file for size: /dremio/db/catalog/009541.sst: No such file or directory Can’t access /009542.sst: IO error: while stat a file for size: /dremio/db/catalog/009542.sst: No such file or directory Can’t access /009543.sst: IO error: while stat a file for size: /dremio/db/catalog/009543.sst: No such file or directory Can’t access /009544.sst: IO error: while stat a file for size: /dremio/db/catalog/009544.sst: No such file or directory Can’t access /009545.sst: IO error: while stat a file for size: /dremio/db/catalog/009545.sst: No such file or directory Can’t access /009546.sst: IO error: while stat a file for size: /dremio/db/catalog/009546.sst: No such file or directory Can’t access /009547.sst: IO error: while stat a file for size: /dremio/db/catalog/009547.sst: No such file or directory Is there any way to force a restart of dremio? I already tried a “dremio-admin clean” but It gave me the same error Thanks ---
**Author:** ![seraus](https://avatars.discourse-cdn.com/v4/letter/s/f6c823/32.png) [@seraus](https://community.dremio.com/u/seraus)\ **Post date:** [December 18, 2018, 2:08pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/2 "2018-12-18T14:08:36Z")
We are using Dremio 2.0.5 on centos 6.10 Thanks ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [December 18, 2018, 5:12pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/3 "2018-12-18T17:12:43Z")
@seraus Any chance you deleted files from db/catalog folder? or is that mount unavailable? Thanks @balaji.ramaswamy ---
**Author:** ![seraus](https://avatars.discourse-cdn.com/v4/letter/s/f6c823/32.png) [@seraus](https://community.dremio.com/u/seraus)\ **Post date:** [January 14, 2019, 2:33pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/4 "2019-01-14T14:33:22Z")
Thanks for your answer balaji.ramaswamy. We overcame the problem by recovering a recent backup. But I’m almost sure that nobody deleted the files manually. Have you ever experienced such an inconsistency due to Dremio malfunction? Thanks ---
**Author:** ![Bruno\_Mata](https://avatars.discourse-cdn.com/v4/letter/b/d2c977/32.png) [@Bruno\_Mata](https://community.dremio.com/u/Bruno_Mata)\ **Post date:** [January 14, 2019, 4:16pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/5 "2019-01-14T16:16:52Z")
Hi, I got the same problem. Trying to solve without backup, if anyone known how please tell us. In my case, no one deleted files directly but restarted my aws EC2 instance with dremio process running. I think the files were corrupted or deleted by the system by staying in a tmp directory. I just don’t known how files was deleted hehehe. Well, any help will be appreciated. Thanks. ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 14, 2019, 5:02pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/6 "2019-01-14T17:02:18Z")
Hi @Bruno_Mata On EC2 instances does the mount on which the db folder resides get reset by server restart? Thanks @balaji.ramaswamy ---
**Author:** ![Bruno\_Mata](https://avatars.discourse-cdn.com/v4/letter/b/d2c977/32.png) [@Bruno\_Mata](https://community.dremio.com/u/Bruno_Mata)\ **Post date:** [January 14, 2019, 6:08pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/7 "2019-01-14T18:08:22Z")
Hi @balaji.ramaswamy thanks for answer. No, the instance have no such config. I think something like a no commited information has lost on reboot. dremio-admin clean command did not work, by the way. Any idea of how reset the metadata state? thanks ---
**Author:** ![Bruno\_Mata](https://avatars.discourse-cdn.com/v4/letter/b/d2c977/32.png) [@Bruno\_Mata](https://community.dremio.com/u/Bruno_Mata)\ **Post date:** [January 14, 2019, 6:24pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/8 "2019-01-14T18:24:24Z")
Maybe this information is relevant, so, other files .sst is in directory. A lot of them. That is why I believe in the hypothesis of a ‘non commited files’. thanks; ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 14, 2019, 6:30pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/9 "2019-01-14T18:30:17Z")
Hi @Bruno_Mata, From 2.1.8 we are reindexing on startup when the previous shutdown in not clean Can you please upgrade to the latest on out website and see if the behavior repeats? Thanks @balaji.ramaswamy ---
**Author:** ![Bruno\_Mata](https://avatars.discourse-cdn.com/v4/letter/b/d2c977/32.png) [@Bruno\_Mata](https://community.dremio.com/u/Bruno_Mata)\ **Post date:** [January 14, 2019, 9:35pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/10 "2019-01-14T21:35:14Z")
Hi @balaji.ramaswamy, Thanks for your help. I did the update and not solve the problem, so, I decide restore a backup (fortunately I found one that someone made). I’ll now setup a periodic automatized backup. thanks; ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [January 14, 2019, 10:13pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/11 "2019-01-14T22:13:39Z")
Hi @Bruno_Mata on your EC2 machine where is the rocksDB now? Thanks @balaji.ramaswamy ---
**Author:** ![Mandrade](https://avatars.discourse-cdn.com/v4/letter/m/85e7bf/32.png) [@Mandrade](https://community.dremio.com/u/Mandrade)\ **Post date:** [April 27, 2020, 7:10pm UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/12 "2020-04-27T19:10:03Z")
Same problem here. Is there any way to solve this problem without having to reinstall Dremio or recover a backup? ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [April 30, 2020, 5:41am UTC](https://community.dremio.com/t/cannot-start-dremio-rockbd-problem-missing-sst-files/2442/13 "2020-04-30T05:41:01Z")
@Mandrade Restoring from backup or if you have a cold copy of the db folder would work, thanks