Dermio kubernetes dremio-master-0 deploy problem

Hello, I am trying to deploy the dremio on my local machine, following this guide:

Other pods is successfuly ready, but the dremio-master-0 is not,

the describe pod shows following infos:

Warning  Unhealthy  26s (x8 over 33s)  kubelet            Startup probe failed: Get "": dial tcp connect: connection refused

the logs shows following errors:

2023-11-15 23:42:15,684 [main] INFO  c.dremio.exec.catalog.PluginsManager - Result of storage plugin startup:
        INFORMATION_SCHEMA: success (0ms). Healthy
        __jobResultsStore: success (127ms). Healthy
        sys: success (0ms). Healthy

2023-11-15 23:42:15,703 [scheduler-1] INFO  c.d.exec.catalog.CatalogServiceImpl - Creating SysFlight source plugin.
2023-11-15 23:42:15,730 [scheduler-1] WARN  c.d.e.catalog.MetadataSynchronizer - Source 'sys' sync failed unexpectedly. Will try again later
java.lang.NullPointerException: Master coordinator is down
        at com.dremio.service.conduit.client.ConduitProviderImpl.getOrCreateChannelToMaster(
        at com.dremio.plugins.sysflight.SysFlightStoragePlugin.getFlightClient(
        at com.dremio.plugins.sysflight.SysFlightStoragePlugin.getFlightTableList(
        at com.dremio.plugins.sysflight.SysFlightStoragePlugin.listDatasetHandles(
        at com.dremio.exec.catalog.MetadataSynchronizer.getDatasetHandleListing(
        at com.dremio.exec.catalog.MetadataSynchronizer.synchronizeDatasets(
        at com.dremio.exec.catalog.MetadataSynchronizer.go(
        at com.dremio.exec.catalog.SourceMetadataManager$RefreshRunner.refreshFull(
        at com.dremio.exec.catalog.SourceMetadataManager$
        at com.dremio.exec.catalog.SourceMetadataManager.refresh(
        at com.dremio.exec.catalog.ManagedStoragePlugin.refresh(
        at com.dremio.exec.catalog.CatalogServiceImpl.refreshSource(
        at com.dremio.exec.catalog.CatalogServiceImpl.lambda$start$0(
        at com.dremio.service.scheduler.LocalSchedulerService$
        at java.util.concurrent.Executors$
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(
        at java.util.concurrent.ScheduledThreadPoolExecutor$
        at java.util.concurrent.ThreadPoolExecutor.runWorker(
        at java.util.concurrent.ThreadPoolExecutor$

Can anyone help?

Hi @zhaoxixiang,

Please answer the following:

  1. What type of scheme and port do you see in dremio-master.yaml?
  2. In values.yaml, was the port changed under the coordinator: or was the TLS enabled?
  3. Can you attach values.yaml along with the full startup log?

Thanks, Bogdan

Thanks for your help, I figured it out myself.

Hi @zhaoxixiang,

Thanks for your help, I figured it out myself.

That’s great! For community’s benefit, can you let us know what the problem was?

Cheers, Bogdan

Sure! I made a silly mistake, fs.s3a.endpoint of aws storage(minio) was not set correctly, so the master can not be reached correctly. My final setup of s3 storage is below:

    extraProperties: |
       <value>must set the right address</value>
1 Like