Iceberg table not registering why?

Apache Iceberg
1

I was using Dremio and nessies – initially nessie was setup with in-memory and i created huge iceberg table [iceberg_tbl1] when nessies rebooted obvious metadata gone and now dremio is not able to access
I configure nessies with postgre now metadata is consistent after reboot
I wanna access or register old iceberg table [iceberg_tbl1] tried all way not working ,
any idea on it.

data and metadata all available on S3 i cross verified too .

2

@rajnish Are you saying the Dremio UI after a reboot is empty? It looks like your KVstore was on a non-persistent disk. Is this a docker image? Can you send your dremio.conf if not a docker image?

3

Hi @balaji.ramaswamy
Thanks for reply,
Dremio was running on my local machine [not on docker]

cat dremio.conf

paths: {

the local path for dremio to store data.

local: ${DREMIO_HOME}“/data”

the distributed path Dremio data including job results, downloads, uploads, etc

#dist: “pdfs://”${paths.local}“/pdfs”
}

services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true,
flight.use_session_service: true
}

dremio is fine , all table is persist after reboot too.
when i configure Nessies [ with default config – in memory DB] and create a iceberg table – this was is not visible after reboot.

for this table, all metadata and data file saved at s3, i verified all is availble at S3, but how i reenable it on Nessie do not find a way.

4

Hi
Updating here,
I think there is no way to re-register table, way around options is there, which I applied and it’s working .
access data folder [contain all .parquet files ] and create table format using Dremio
now recreate table using nessies , using CTAS
it working new iceberg table created.

Thanks,