Nessie does not see iceberg tables after restart

Hi ,
I am using local docker setup with minio and nessie. When i restart the containers, nessie catalog does not see the tables . This is what i see after restart:

I tried to add the table folders but i can’t. How can i solve this?
thanks
tolga

If you are running the Docker Container for Nessie, you have to configure where it will store entries, if not it will by default use the memory store which would be cleared when the server shuts off.

You can find Nessie Configuration for Storage here: Configuration - Project Nessie: Transactional Catalog for Data Lakes with Git-like semantics

Hi Alex , thanks for your help. After setting the db configs, now my metadata is persisted.
Tolga

HI, can u please elaborate the config setup
Facing the same issue here

Hi Shifas,
This is my docker-compose.yml file (only nessie part:)
I am using a local postgre db for catalog data.I created a database called nessie in postgres. ( You can give any name)

nessie:
image: projectnessie/nessie:0.67.0
container_name: nessie
extra_hosts:
- “host.docker.internal:host-gateway”
environment:
NESSIE_VERSION_STORE_TYPE: JDBC
QUARKUS_DATASOURCE_JDBC_URL: “jdbc:postgresql://192.168.0.10:5432/nessie”
quarkus.datasource.username: “postgres”
quarkus.datasource.password: “changeme”
networks:
network:
ipv4_address: 192.168.0.19

ports:
  - 19120:19120

thanks
tolga

1 Like

Have you checked the changes in postgresql, like how is it written on the database for persistence?

Yes i checked it. It creates 2 tables , and for each dml op, it updates these tables.Did you? If you have any insights pls share.

@tolgaevren Are you also losing your tables after a Dremio docker restart

No , table’s data (as parquet files) is in mino objects storage , catalog info is in database . It must be persistent. What happens at your site?

@tolgaevren Sorry the question was directed to @shifas

@shifas When you say you have the same issue, does that mean after a restart your tables no longer exist?

Yes, as mentioned by @tolgaevren, table’s data file exists in the MinIO storage. But catalog info (ie, tables) do not seem to appear on restart