Best Catalog for Apache Iceberg w/ Dremio?

Apache Iceberg
1

Q: Is there a recommended catalog to use with Dremio and Apache Iceberg?

I’m new to Apache Iceberg and Dremio. I’m using MinIO for S3 because we’re doing on-prem, and using docker to run all our services on a single computer (for now).

I ran the Apache Iceberg quick start demo and it includes a REST catalog: apache/iceberg-rest-fixture

Do I understand correctly that Dremio can’t talk to these simple REST catalogs? I swapped it for Nessie but then discovered that some of the Python SQL examples that come with the tabulario/spark-iceberg example, don’t work:

%%sql

ALTER TABLE nyc.taxis RENAME COLUMN fare_amount TO fare

...

AnalysisException: [UNSUPPORTED_FEATURE.TABLE_OPERATION] The feature is not supported: Table `spark_catalog`.`nyc`.`taxis` does not support RENAME COLUMN. Please check the current catalog and namespace to make sure the qualified table name is expected, and also check the catalog implementation which is configured by "spark.sql.catalog".
2

It seems that my spark instance isn’t picking up my configuration changes, so the above error message is related to spark using some default built-in catalog. I’ve tried both setting the spark-defaults.conf and also passing the configuration at runtime, neither is working. But since this is a spark issue and not Dremio, I’ll take my question to the spark community.