No default branch set - Nessie

jbaranda · July 14, 2023, 6:34pm

Hi After updating dremio to dremio 24.1 I am no longer able to format files into tables, I get this error: No default branch set. Why could that be? I am working with Parquet, json and XML files stored S3 I am not even using Nessie.

This is the error I get int he profile: om.dremio.exec.store.NoDefaultBranchException) org.projectnessie.error.NessieReferenceNotFoundException: Named reference ‘main’ not found
com.dremio.plugins.NessieClientImpl.getDefaultBranch():149
com.dremio.exec.store.iceberg.nessie.IcebergNessieTableOperations.getDefaultBranch():160
com.dremio.exec.store.iceberg.nessie.IcebergNessieTableOperations.doRefresh():90
org.apache.iceberg.BaseMetastoreTableOperations.refresh():97
org.apache.iceberg.BaseMetastoreTableOperations.current():80
com.dremio.exec.store.iceberg.model.IcebergBaseCommand.getRootPointer():638
com.dremio.exec.store.iceberg.model.IcebergTableLoader.():29
com.dremio.exec.store.iceberg.model.IcebergBaseModel.getIcebergTableLoader():239
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.isRepairNeeded():157
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.lambda$performCheckAndRepairDataset$1():140
com.dremio.common.util.Retryer.call():57
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.performCheckAndRepairDataset():139
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.checkAndRepairDatasetWithoutQueryRetry():124
com.dremio.exec.planner.sql.handlers.refresh.AbstractRefreshPlanBuilder.repairAndSaveDatasetConfigIfNecessary():218
com.dremio.exec.planner.sql.handlers.refresh.FileSystemRefreshIncrementalPlanBuilder.updateDatasetConfigWithIcebergMetadataIfNecessary():182
com.dremio.exec.planner.sql.handlers.MetadataRefreshPlanBuilderFactory.getPlanBuilder():75
com.dremio.exec.planner.sql.handlers.RefreshDatasetHandler.getPlan():66
com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan():59
com.dremio.exec.work.foreman.AttemptManager.plan():565
com.dremio.exec.work.foreman.AttemptManager.lambda$run$4():462
com.dremio.service.commandpool.CommandWrapper.run():70
com.dremio.service.commandpool.ReleasableBoundCommandPool.submit():171
com.dremio.exec.work.foreman.AttemptManager.run():452
com.dremio.exec.work.protector.Foreman.newAttempt():226
com.dremio.exec.work.protector.Foreman.start():168
com.dremio.exec.work.protector.ForemenWorkManager.submit():281
com.dremio.exec.work.protector.ForemenWorkManager$LocalQueryExecutorImpl.submitLocalQuery():582
com.dremio.service.jobs.LocalJobsService.startJob():723
com.dremio.service.jobs.LocalJobsService.lambda$submitJob$1():850
com.dremio.service.commandpool.CommandWrapper.run():70
com.dremio.service.commandpool.ReleasableBoundCommandPool.submit():171
com.dremio.service.jobs.LocalJobsService.submitJob():840
com.dremio.service.jobs.LocalJobsService$JobSubmissionHelper.submitJobToCommandPool():2978
com.dremio.service.jobs.LocalJobsService.runQueryAsJob():957
com.dremio.exec.store.dfs.FileSystemPlugin.runRefreshQuery():394
com.dremio.exec.catalog.DatasetSaverImpl.saveUsingInternalRefreshDatasetQuery():161
com.dremio.exec.catalog.DatasetSaverImpl.save():106
com.dremio.exec.catalog.DatasetSaverImpl.save():138
com.dremio.exec.catalog.MetadataSynchronizer.tryHandleExistingDataset():320
com.dremio.exec.catalog.MetadataSynchronizer.handleExistingDataset():243
com.dremio.exec.catalog.MetadataSynchronizer.synchronizeDatasets():211
com.dremio.exec.catalog.MetadataSynchronizer.go():136
com.dremio.exec.catalog.SourceMetadataManager$RefreshRunner.refreshFull():443
com.dremio.exec.catalog.SourceMetadataManager$BackgroundRefresh.run():557
com.dremio.exec.catalog.SourceMetadataManager.wakeup():265
com.dremio.exec.catalog.SourceMetadataManager.access$300():97
com.dremio.exec.catalog.SourceMetadataManager$WakeupWorker.run():204
com.dremio.service.scheduler.LocalSchedulerService$CancellableTask.run():247
java.util.concurrent.Executors$RunnableAdapter.call():511
java.util.concurrent.FutureTask.run():266
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201():180
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run():293
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():750
Caused By (org.projectnessie.error.NessieReferenceNotFoundException) Named reference ‘main’ not found
org.projectnessie.error.ErrorCode.lambda$asException$1():66
java.util.Optional.map():215
org.projectnessie.error.ErrorCode.asException():66
com.dremio.services.nessie.grpc.client.GrpcExceptionMapper.toNessieException():308
com.dremio.services.nessie.grpc.client.GrpcExceptionMapper.throwDeclaredException():287
com.dremio.services.nessie.grpc.client.GrpcExceptionMapper.handleNessieNotFoundEx():250
com.dremio.services.nessie.grpc.client.impl.GrpcApiImpl.getDefaultBranch():108
com.dremio.plugins.NessieClientImpl.getDefaultBranch():146
com.dremio.exec.store.iceberg.nessie.IcebergNessieTableOperations.getDefaultBranch():160
com.dremio.exec.store.iceberg.nessie.IcebergNessieTableOperations.doRefresh():90
org.apache.iceberg.BaseMetastoreTableOperations.refresh():97
org.apache.iceberg.BaseMetastoreTableOperations.current():80
com.dremio.exec.store.iceberg.model.IcebergBaseCommand.getRootPointer():638
com.dremio.exec.store.iceberg.model.IcebergTableLoader.():29
com.dremio.exec.store.iceberg.model.IcebergBaseModel.getIcebergTableLoader():239
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.isRepairNeeded():157
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.lambda$performCheckAndRepairDataset$1():140
com.dremio.common.util.Retryer.call():57
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.performCheckAndRepairDataset():139
com.dremio.exec.store.dfs.RepairKvstoreFromIcebergMetadata.checkAndRepairDatasetWithoutQueryRetry():124
com.dremio.exec.planner.sql.handlers.refresh.AbstractRefreshPlanBuilder.repairAndSaveDatasetConfigIfNecessary():218
com.dremio.exec.planner.sql.handlers.refresh.FileSystemRefreshIncrementalPlanBuilder.updateDatasetConfigWithIcebergMetadataIfNecessary():182
com.dremio.exec.planner.sql.handlers.MetadataRefreshPlanBuilderFactory.getPlanBuilder():75
com.dremio.exec.planner.sql.handlers.RefreshDatasetHandler.getPlan():66
com.dremio.exec.planner.sql.handlers.commands.HandlerToExec.plan():59
com.dremio.exec.work.foreman.AttemptManager.plan():565
com.dremio.exec.work.foreman.AttemptManager.lambda$run$4():462
com.dremio.service.commandpool.CommandWrapper.run():70
com.dremio.service.commandpool.ReleasableBoundCommandPool.submit():171
com.dremio.exec.work.foreman.AttemptManager.run():452
com.dremio.exec.work.protector.Foreman.newAttempt():226
com.dremio.exec.work.protector.Foreman.start():168
com.dremio.exec.work.protector.ForemenWorkManager.submit():281
com.dremio.exec.work.protector.ForemenWorkManager$LocalQueryExecutorImpl.submitLocalQuery():582
com.dremio.service.jobs.LocalJobsService.startJob():723
com.dremio.service.jobs.LocalJobsService.lambda$submitJob$1():850
com.dremio.service.commandpool.CommandWrapper.run():70
com.dremio.service.commandpool.ReleasableBoundCommandPool.submit():171
com.dremio.service.jobs.LocalJobsService.submitJob():840
com.dremio.service.jobs.LocalJobsService$JobSubmissionHelper.submitJobToCommandPool():2978
com.dremio.service.jobs.LocalJobsService.runQueryAsJob():957
com.dremio.exec.store.dfs.FileSystemPlugin.runRefreshQuery():394
com.dremio.exec.catalog.DatasetSaverImpl.saveUsingInternalRefreshDatasetQuery():161
com.dremio.exec.catalog.DatasetSaverImpl.save():106
com.dremio.exec.catalog.DatasetSaverImpl.save():138
com.dremio.exec.catalog.MetadataSynchronizer.tryHandleExistingDataset():320
com.dremio.exec.catalog.MetadataSynchronizer.handleExistingDataset():243
com.dremio.exec.catalog.MetadataSynchronizer.synchronizeDatasets():211
com.dremio.exec.catalog.MetadataSynchronizer.go():136
com.dremio.exec.catalog.SourceMetadataManager$RefreshRunner.refreshFull():443
com.dremio.exec.catalog.SourceMetadataManager$BackgroundRefresh.run():557
com.dremio.exec.catalog.SourceMetadataManager.wakeup():265
com.dremio.exec.catalog.SourceMetadataManager.access$300():97
com.dremio.exec.catalog.SourceMetadataManager$WakeupWorker.run():204
com.dremio.service.scheduler.LocalSchedulerService$CancellableTask.run():247
java.util.concurrent.Executors$RunnableAdapter.call():511
java.util.concurrent.FutureTask.run():266
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201():180
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run():293
java.util.concurrent.ThreadPoolExecutor.runWorker():1149
java.util.concurrent.ThreadPoolExecutor$Worker.run():624
java.lang.Thread.run():750

balaji.ramaswamy · July 15, 2023, 8:34pm

@jbaranda Is this Community edition or OSS?

jbaranda · July 16, 2023, 5:13am

This is dremio community edition hosted on EKS cluster @balaji.ramaswamy

balaji.ramaswamy · July 22, 2023, 7:03am

@jbaranda Let me check on this and have someone get back to you

Mark_Hoerth · July 22, 2023, 3:21pm

HI @jbaranda,
My apologies that you are hitting an issue. I just installed Dremio 24.1 community edition and did not have any issues using files in my S3 bucket. Please clarify a few points for me:

What version did you upgrade from?
Are all queries displaying this issue, or only some?
I suspect there might be an issue in your KV store database. Could you take a backup and attach it here? If it’s too large to attach, we will work out another way to send it.

Topic		Replies	Views
Cannot create table in Nessie Branch Dremio Cloud	6	1792	March 22, 2022
Dremio-oss, nessie, and s3-compatible storage	2	44	November 6, 2024
How work with branch using DBT + Dremio + Nessie	4	336	October 22, 2024
Postgres Schema for Iceberg Nessie Apache Iceberg	4	109	October 17, 2024
Update and Delete is failing at branch level in dremio	3	17	August 12, 2024

No default branch set - Nessie

Related topics