Not able to start production dremio

Hi,

I am not able to start dremio its struck at starting Kv store i also see cpu,ram all looks goods

here is env file

Copyright © 2017-2018 Dremio Corporation

Licensed under the Apache License, Version 2.0 (the “License”);

you may not use this file except in compliance with the License.

You may obtain a copy of the License at

http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software

distributed under the License is distributed on an “AS IS” BASIS,

WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.

See the License for the specific language governing permissions and

limitations under the License.

DREMIO_MAX_HEAP_MEMORY_SIZE_MB=8192
DREMIO_MAX_DIRECT_MEMORY_SIZE_MB=16384,

Dremio environment variables used by Dremio daemon

Directory where Dremio logs are written

Default to $DREMIO_HOME/log

#DREMIO_LOG_DIR=${DREMIO_HOME}/log

Send logs to console and not to log files. The DREMIO_LOG_DIR is ignored if set.

#DREMIO_LOG_TO_CONSOLE=1

Directory where Dremio pidfiles are written

Default to $DREMIO_HOME/run

#DREMIO_PID_DIR=${DREMIO_HOME}/run

Max total memory size (in MB) for the Dremio process

If not set, default to using max heap and max direct.

If both max heap and max direct are set, this is not used

If one is set, the other is calculated as difference

of max memory and the one that is set.

#DREMIO_MAX_MEMORY_SIZE_MB=
#DREMIO_MAX_HEAP_MEMORY_SIZE_MB=8192
#DREMIO_MAX_HEAP_MEMORY_SIZE_MB=70240
#DREMIO_MAX_DIRECT_MEMORY_SIZE_MB=16384

Max heap memory size (in MB) for the Dremio process

Default to 4096 for server

#DREMIO_MAX_HEAP_MEMORY_SIZE_MB=4096
#DREMIO_MAX_DIRECT_MEMORY_SIZE_MB=50240

Max direct memory size (in MB) for the Dremio process

Default to 8192 for server

#DREMIO_MAX_DIRECT_MEMORY_SIZE_MB=8192

Max permanent generation memory size (in MB) for the Dremio process

(Only used for Java 7)

Default to 512 for server

#DREMIO_MAX_PERMGEN_MEMORY_SIZE_MB=512

Garbage collection logging is enabled by default. Set the following

parameter to “no” to disable garbage collection logging.

#DREMIO_GC_LOGS_ENABLED=“yes”

The scheduling priority for the server

Default to 0

DREMIO_NICENESS=0

Number of seconds after which the server is killed forcibly it it hasn’t stopped

Default to 120

#DREMIO_STOP_TIMEOUT=120

Extra Java options - shared between dremio and dremio-admin commands

#DREMIO_JAVA_EXTRA_OPTS=

Extra Java options - client only (dremio-admin command)

#DREMIO_JAVA_CLIENT_EXTRA_OPTS=

Extra Java options - server only (dremio command)

#DREMIO_JAVA_SERVER_EXTRA_OPTS=

@doron here is attached log file

dremio-master-0.zip (2.0 KB)

Hello @doron @balaji.ramaswamy can you please help i dont able to get clue where it was struck i am running production down isse from past 6 hours please help me how to start dremio

Hello @Vikash_Singh,

It seems that Dremio’s master pod is restarting because the log is incomplete here. Please try the following command to capture more of the log kubectl logs dremio-master-0 -f

Sincerely,
@RyanTse

Hello,
here is output did not say any error
D:\VisurInfra\dremioLB\dremio-cloud-tools-master\charts>kubectl logs dremio-master-0 -f
starting dremio
2020-03-17 22:03:26,407 [main] INFO com.dremio.exec.util.GuavaPatcher - Google’s Stopwatch patched for old HBase Guava version.
2020-03-17 22:03:26,414 [main] INFO com.dremio.exec.util.GuavaPatcher - Google’s Closeables patched for old HBase Guava version.
2020-03-17 22:03:26,517 [main] INFO com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 93ms.
Base Configuration:
- jar:file:/opt/dremio/jars/dremio-common-4.1.3-202001022113020736-53142377.jar!/sabot-default.conf

Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/opt/dremio/jars/dremio-services-datastore-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-provision-common-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-jobs-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-coordinator-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-yarn-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-pdfs-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-common-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-sabot-joust-java-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-namespace-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-backend-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-hive3-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-dac-daemon-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-sabot-kernel-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-execselector-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-adls-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-azure-storage-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-configuration-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-common-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-telemetry-api-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-jdbc-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-options-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-telemetry-impl-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-mongo-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-jdbc-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive2-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-elasticsearch-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-resourcescheduler-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-accelerator-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-kernel-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-sabot-logical-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-services-cachemanager-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive3-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-mongo-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-sabot-scheduler-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-function-registry-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-s3-plugin-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-ce-hive2-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-hive-plugin-common-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/dremio-services-users-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf
- jar:file:/opt/dremio/jars/3rdparty/dremio-hive2-exec-shaded-4.1.3-202001022113020736-53142377.jar!/sabot-module.conf

2020-03-17 22:03:26,587 [main] INFO com.dremio.config.DremioConfig - Applying provided system property to config: -Dzookeeper=zk-hs:2181
2020-03-17 22:03:26,588 [main] INFO com.dremio.config.DremioConfig - Applying provided system property to config: -Dservices.coordinator.master.embedded-zookeeper.enabled=false
2020-03-17 22:03:26,588 [main] INFO com.dremio.config.DremioConfig - Applying provided system property to config: -Dservices.executor.enabled=false
2020-03-17 22:03:26,950 [main] INFO c.d.common.scanner.BuildTimeScan - Loaded prescanned packages [com.dremio.storage, com.dremio.datastore, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.telemetry.api, com.dremio.service.jobs, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.datastore, com.dremio.service.namespace, com.dremio.service.users, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.options, com.dremio.telemetry.api, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.provision, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.provision, com.dremio.options, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.datastore, com.dremio.service.jobs, com.dremio.service.namespace, com.dremio.service.users, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.resource, com.dremio.resource.basic, com.dremio.exec.store, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.telemetry.api, com.dremio.services.configuration, com.dremio.services.configuration.ConfigurationStore, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.datastore, com.dremio.service.jobs, com.dremio.service.namespace, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.options, com.dremio.telemetry.api, com.dremio.service.users, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.provision, com.dremio.exec.store, com.dremio.storage, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.options, com.dremio.telemetry.api, com.dremio.service.namespace, com.dremio.datastore, com.dremio.service.users, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.provision, com.dremio.storage, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.options, com.dremio.telemetry.api, com.dremio.service.namespace, com.dremio.datastore, com.dremio.service.users, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.provision, com.dremio.storage, com.dremio.exec.fn.hive, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.options, com.dremio.telemetry.api, com.dremio.service.namespace, com.dremio.datastore, com.dremio.service.users, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.provision, org.apache.hadoop.hive] from locations [jar:file:/opt/dremio/jars/dremio-services-datastore-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-jobs-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-common-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-dac-backend-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-common-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-services-accelerator-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-kernel-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-sabot-logical-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-ce-sabot-scheduler-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json, jar:file:/opt/dremio/jars/dremio-hive-function-registry-4.1.3-202001022113020736-53142377.jar!/META-INF/dremio-module-scan/registry.json]
2020-03-17T22:03:27.259+0000: [GC (Allocation Failure) [PSYoungGen: 524800K->32863K(611840K)] 524800K->32887K(2010112K), 0.0257928 secs] [Times: user=0.12 sys=0.01, real=0.02 secs]
2020-03-17T22:03:27.913+0000: [GC (Allocation Failure) [PSYoungGen: 557663K->51511K(1136640K)] 557687K->51607K(2534912K), 0.0289830 secs] [Times: user=0.08 sys=0.01, real=0.03 secs]
2020-03-17 22:03:28,094 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.datastore, com.dremio.provision, com.dremio.service.jobs, com.dremio.provision.yarn.service, com.dremio.exec.store, com.dremio.joust.geo, com.dremio.service.namespace, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac.cmd, com.dremio.dac.cmd.upgrade, com.dremio.extra.exec.store.dfs, com.dremio.exec.planner.acceleration.substitution, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.plugins.adl.store, com.dremio.plugins.elastic, com.dremio.plugins.azure, com.dremio.services.configuration, com.dremio.services.configuration.ConfigurationStore, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.telemetry.api, com.dremio.exec.store.jdbc, com.dremio.options, com.dremio.telemetry.impl.config, com.dremio.plugins.mongo, com.dremio.exec.store.jdbc, com.dremio.exec.store.hive, com.dremio.extras.plugins.elastic, com.dremio.resource, com.dremio.resource.basic, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.service.cachemanager, com.dremio.exec.store.hive, com.dremio.plugins.mongo, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.fn.hive, com.dremio.plugins.s3.store, com.dremio.exec.store.hive, com.dremio.exec.store.hive.exec, com.dremio.service.users, org.apache.hadoop.hive] in locations [jar:file:/opt/dremio/jars/dremio-provision-common-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-joust-java-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive3-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-dac-daemon-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-kernel-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-execselector-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-adls-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-azure-storage-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-configuration-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-api-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-jdbc-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-options-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-impl-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-mongo-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-jdbc-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-hive2-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-elasticsearch-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-resourcescheduler-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-services-cachemanager-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-hive3-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-s3-plugin-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive2-plugin-launcher-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-common-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-4.1.3-202001022113020736-53142377.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive2-exec-shaded-4.1.3-202001022113020736-53142377.jar!/] took 1141ms
2020-03-17 22:03:28,357 [main] INFO c.d.datastore.LocalKVStoreProvider - Starting LocalKVStoreProvider

@Vikash_Singh,

Could you restart the pods of the Dremio deployment and provide another set of logs? This set of logs doesn’t capture any issues.

Sincerely,
@RyanTse

Hello,

Sorry but i have only that line have been printed in log is any way to enable debug mode and check root cause

dremio-master-0.zip (22 Bytes)

@RyanTse,

here is the log after enabled debug mode

dremio-master-0.zip (2.9 KB)

@RyanTse i have also stopped and started dremio admin

and execting this command
dremio@dremio-admin:/opt/dremio/bin$ ./dremio-admin clean
its strck from 15 min

Hi ,

I have change local: “/tmp/dremio” and restarted pods ,able to start now so its looks like RockDb currupted do we have way to recovered not finding any error too