# Unable to start Dremio in UNIX - KVStore has an older version

**URL:** <https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942>\
**Category:** Uncategorized\
**Created:** [September 27, 2018, 5:42pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942 "2018-09-27T17:42:05Z")\
**Posts on this page:** 12\
**Page:** 1

<div class="post-metadata">

**Author:** ![HAREESH](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/hareesh/32/1601_2.png) [@HAREESH](https://community.dremio.com/u/HAREESH)\
**Post date:** [September 27, 2018, 5:42pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/1 "2018-09-27T17:42:05Z")

</div>

Hi,  
Not able to start Dremio in UNIX server. Can you please help on below error.

**PFB server.out logs:**

Thu Sep 27 13:28:33 EDT 2018 Starting dremio on cspinf-wc-1p  
core file size (blocks, -c) 0  
data seg size (kbytes, -d) unlimited  
scheduling priority (-e) 0  
file size (blocks, -f) unlimited  
pending signals (-i) 63678  
max locked memory (kbytes, -l) 64  
max memory size (kbytes, -m) unlimited  
open files (-n) 1024  
pipe size (512 bytes, -p) 8  
POSIX message queues (bytes, -q) 819200  
real-time priority (-r) 0  
stack size (kbytes, -s) 10240  
cpu time (seconds, -t) unlimited  
max user processes (-u) 63678  
virtual memory (kbytes, -v) unlimited  
file locks (-x) unlimited  
KVStore version is 1.4.9-201802191836310213-7195059  
Catastrophic failure occurred. Exiting. Information follows: Failed to start services, daemon exiting.  
java.lang.IllegalStateException: KVStore has an older version (1.4.9-201802191836310213-7195059) than the server (2.1.6-201809161911340177-edb5b4d-mapr), please run the upgrade tool first  
at com.google.common.base.Preconditions.checkState(Preconditions.java:721)  
at com.dremio.dac.daemon.DremioDaemon$AutoUpgrade.ensureUpgradeSupported(DremioDaemon.java:72)  
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:182)  
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:171)  
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:103)

Any help greatly appreciated.

Thanks,  
hareesh

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [September 27, 2018, 5:47pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/2 "2018-09-27T17:47:48Z")

</div>

> [@HAREESH](#):
>
> please run the upgrade tool first

Upgrade tool is ` .../bin/dremio-admin upgrade`

---

<div class="post-metadata">

**Author:** ![HAREESH](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/hareesh/32/1601_2.png) [@HAREESH](https://community.dremio.com/u/HAREESH)\
**Post date:** [September 27, 2018, 6:18pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/3 "2018-09-27T18:18:41Z")

</div>

Thanks @yufeldman i executed upgrade tool. But while starting dremio getting below error. Can you please help on this.

file locks (-x) unlimited  
KVStore version is 2.1.6-201809161911340177-edb5b4d-mapr  
Catastrophic failure occurred. Exiting. Information follows: Failed to start services, daemon exiting.  
java.lang.RuntimeException: java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: MapR user ticket not available! error = com.mapr.security.MutableInt@19cd7577  
at com.google.common.base.Throwables.propagate(Throwables.java:240)  
at com.dremio.dac.daemon.ZkServer.init(ZkServer.java:111)  
at com.dremio.dac.daemon.ZkServer.start(ZkServer.java:74)  
at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)  
at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:74)  
at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)  
at com.dremio.dac.daemon.DACDaemon.startPreServices(DACDaemon.java:172)  
at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:182)  
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:112)  
Caused by: java.io.IOException: Could not configure server because SASL configuration did not allow the ZooKeeper server to authenticate itself properly: javax.security.auth.login.LoginException: MapR user ticket not available! error = com.mapr.security.MutableInt@19cd7577  
at org.apache.zookeeper.server.ServerCnxnFactory.configureSaslLogin(ServerCnxnFactory.java:205)  
at org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:87)  
at org.apache.zookeeper.server.ZooKeeperServerMain.runFromConfig(ZooKeeperServerMain.java:110)  
at com.dremio.dac.daemon.ZkServer$ZkEmbeddedServer.run(ZkServer.java:142)  
at java.lang.Thread.run(Thread.java:748)

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [September 27, 2018, 6:23pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/4 "2018-09-27T18:23:16Z")

</div>

Make sure you you have specified MAPR\_TICKET in dremio-env  
[https://docs.dremio.com/deployment/yarn-deployment.html?h=dremio-env](https://docs.dremio.com/deployment/yarn-deployment.html?h=dremio-env)

---

<div class="post-metadata">

**Author:** ![HAREESH](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/hareesh/32/1601_2.png) [@HAREESH](https://community.dremio.com/u/HAREESH)\
**Post date:** [September 27, 2018, 7:26pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/5 "2018-09-27T19:26:08Z")

</div>

Thanks alot. Dremio is running now. But tried opening in 9047 port but Dremio UI is not loading. Can you please let me know where we can get the port number in Dremio

**ps -ef | grep dremio**  
dremio 30821 1 6 15:07 ? 00:00:32 /usr/lib/jvm/java-1.8.0-openjdk-1.8.0.181-3.b13.el6\_10.x86\_64/jre/bin/java -Djava.util.logging.config.class=org.slf4j.bridge.SLF4JBridgeHandler -Djava.library.path=/opt/dremio/lib -XX:+PrintGCDetails -XX:+PrintGCDateStamps -Xloggc:/var/log/dremio/server.gc -Ddremio.log.path=/var/log/dremio -Xmx4096m -XX:MaxDirectMemorySize=8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=/var/log/dremio -Dio.netty.maxDirectMemory=0 -DMAPR\_IMPALA\_RA\_THROTTLE -DMAPR\_MAX\_RA\_STREAMS=400 -cp /etc/dremio:/opt/mapr/hadoop/hadoop-2.7.0/etc/hadoop:/opt/dremio/jars/_:/opt/dremio/jars/ext/_:/opt/dremio/jars/3rdparty/\* com.dremio.dac.daemon.DremioDaemon dremio start

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [September 27, 2018, 7:42pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/6 "2018-09-27T19:42:41Z")

</div>

Could you look at the logs: `/var/log/dremio/server.log and /var/log/dremio/server.out`

---

<div class="post-metadata">

**Author:** ![HAREESH](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/hareesh/32/1601_2.png) [@HAREESH](https://community.dremio.com/u/HAREESH)\
**Post date:** [September 28, 2018, 5:42pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/7 "2018-09-28T17:42:53Z")

</div>

PFB attached log files.

[server.zip](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/9d279c86c7a76a77bfc022eccab55f067e594214.zip) (8.9 KB)

---

<div class="post-metadata">

**Author:** ![yufeldman](https://avatars.discourse-cdn.com/v4/letter/y/c77e96/32.png) [@yufeldman](https://community.dremio.com/u/yufeldman)\
**Post date:** [September 28, 2018, 7:43pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/8 "2018-09-28T19:43:52Z")

</div>

Looks like process is stuck somewhere. Try to take jstack of the Dremio process may be?

---

<div class="post-metadata">

**Author:** ![mmohan](https://avatars.discourse-cdn.com/v4/letter/m/f05b48/32.png) [@mmohan](https://community.dremio.com/u/mmohan)\
**Post date:** [September 20, 2019, 1:37pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/9 "2019-09-20T13:37:04Z")

</div>

Hi,  
Is it possible to have multiple version of dremio to be executed at same time?  
When i tried, i am getting error  
Dremio is exiting. Failure while starting services.  
java.lang.IllegalStateException: KVStore has an older version (3.3.2-201908142136370993-d60145d) than the server (4.0.0-201909121834570395-c7a5071), please run the upgrade tool first

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [September 21, 2019, 1:29am UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/10 "2019-09-21T01:29:37Z")

</div>

@mmohan

The current version of Dremio software does not match with the KV store that contains our internal metadata. As the error suggests, it looks like the software is at a higher version so kindly run the “dremio-admin upgrade” command and then start Dremio

---

<div class="post-metadata">

**Author:** ![saimanne](https://avatars.discourse-cdn.com/v4/letter/s/aca169/32.png) [@saimanne](https://community.dremio.com/u/saimanne)\
**Post date:** [February 3, 2023, 10:50pm UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/11 "2023-02-03T22:50:26Z")

</div>

any resolution on this issue ? I have similar issue.

Dremio is exiting. Failure while starting services.  
java.lang.IllegalStateException: KVStore has an older version (21.2.0-202205262146080444-038d6d1b) than the server (22.1.1-202208230402290397-a7010f28), please run the upgrade tool first  
at com.google.common.base.Preconditions.checkState(Preconditions.java:821)  
at com.dremio.dac.daemon.DremioDaemon$AutoUpgrade.ensureUpgradeSupported(DremioDaemon.java:67)  
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:228)  
at com.dremio.dac.cmd.upgrade.Upgrade.run(Upgrade.java:186)  
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:99)

after dremio-admin upgrade , below error is showing.

Dremio is exiting. Failure while starting services.  
com.dremio.services.credentials.SecretCredentialsException: Secret decryption encounters exception  
at com.dremio.services.credentials.LocalSecretsStore.decrypt(LocalSecretsStore.java:128)  
at com.dremio.services.credentials.LocalSecretsStore.doLookup(LocalSecretsStore.java:197)  
at com.dremio.services.credentials.SecretCredentialsProvider.doLookup(SecretCredentialsProvider.java:57)  
at com.dremio.services.credentials.AbstractSimpleCredentialsProvider.lookup(AbstractSimpleCredentialsProvider.java:43)  
at com.dremio.services.credentials.SimpleCredentialsService.lookup(SimpleCredentialsService.java:106)  
at com.dremio.extusr.ldap.LdapUserProvider.lookupSecretFromBindPassword(LdapUserProvider.java:257)  
at com.dremio.extusr.ldap.LdapUserProvider.start(LdapUserProvider.java:239)  
at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)  
at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:88)  
at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)  
at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:196)  
at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:202)  
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:104)  
Suppressed: java.lang.NullPointerException  
at com.dremio.service.orphanagecleaner.OrphanageCleanerService.close(OrphanageCleanerService.java:124)  
at com.dremio.service.SingletonRegistry$AbstractServiceReference.close(SingletonRegistry.java:130)  
at com.dremio.common.AutoCloseables.close(AutoCloseables.java:139)  
at com.dremio.service.ServiceRegistry.close(ServiceRegistry.java:96)  
at com.dremio.service.SingletonRegistry.close(SingletonRegistry.java:90)  
at com.dremio.common.AutoCloseables.close(AutoCloseables.java:139)  
at com.dremio.common.AutoCloseables.close(AutoCloseables.java:76)  
at com.dremio.dac.daemon.DACDaemon.close(DACDaemon.java:296)  
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:103)  
Suppressed: java.lang.NullPointerException  
at com.dremio.provision.service.ProvisioningServiceImpl.close(ProvisioningServiceImpl.java:512)  
… 8 more  
Caused by: javax.crypto.AEADBadTagException: Tag mismatch!  
at com.sun.crypto.provider.GaloisCounterMode.decryptFinal(GaloisCounterMode.java:620)  
at com.sun.crypto.provider.CipherCore.finalNoPadding(CipherCore.java:1116)  
at com.sun.crypto.provider.CipherCore.fillOutputBuffer(CipherCore.java:1053)  
at com.sun.crypto.provider.CipherCore.doFinal(CipherCore.java:853)  
at com.sun.crypto.provider.AESCipher.engineDoFinal(AESCipher.java:446)  
at javax.crypto.Cipher.doFinal(Cipher.java:2168)  
at com.dremio.services.credentials.LocalSecretsStore.decrypt(LocalSecretsStore.java:125)  
… 12 more

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [February 9, 2023, 7:19am UTC](https://community.dremio.com/t/unable-to-start-dremio-in-unix-kvstore-has-an-older-version/1942/12 "2023-02-09T07:19:36Z")

</div>

@saimanne Did you have a new version of Dremio installed?

Did you do `dremio-admin upgrade` then start Dremio?

[https://docs.dremio.com/software/deployment/standalone/standalone-install-upgrade/](https://docs.dremio.com/software/deployment/standalone/standalone-install-upgrade/)