I am trying to add a glue catalog as a source in Dremio Community Edition, but I am not having any luck. We are getting a “Source is not currently available.” error in the ui, and looking at the logs on the master/coordinator node, we’re not seeing any relevant logs or errors.
Has anyone else had trouble / success adding a glue data catalog as a source? we are running on eks and have followed these instructions, connecting with EC2 Metadata and have updated the relevant IAM permissions.
@balaji.ramaswamy wondering if you have any insight here? Not sure how to diagnose or resolve this as we’re not getting much in the way of meaningful error messages in the UI and nothing in logs.
@balaji.ramaswamy I was able to add the glue source after realizing our Dremio master was running in a different node group which had a different IAM role than our executors.
However, after adding the Glue source, I’m not able to do much with it. My first test was to create a table using CTAS. I’m getting an error “Database does not exist []”, which I’ve never seen with other sources, including direct links to S3 and Nessie.
Have you or anyone else ever seen this?
@sp2barrels Are you trying to create an iceberg table? Maybe context is not set? Can you please send me the profile?
Thank you Balaji - I think this came down to an issue of mixed terminology, e.g. schema/database. I wasn’t aware that I needed to create what I thought would have been called a schema in Glue prior to running my CTAS statement. I created the database/schema through the AWS console and then my statement ran. Sorry for not updating the ticket and thank you for your response!
@sp2barrels Glad its resolved and you can move forward, please get back to us if you have any further questions