Do you plan on supporting Apache Atlas in Dremio. It would be great to export the virtual layers to atlas and also get metadata from Atlas to describe some tables/columns.
1 Like
Hey @chrisJamesC, thanks for the feedback! For exporting Dremio metadata, we’re initially planning to expose dataset/catalog APIs to allow users to manage datasets and general catalog information. For importing metadata from Atlas: As Dremio adds more data catalog type functionality (dataset/column descriptions, etc.), we’ll also look into syncing/bootstrapping metadata from different systems, including Atlas. Probably, this will initially be done through the same dataset/catalog API, followed by first class integrations.
Would be great to get your feedback and how this would fit into your use-case needs (any details on your end-to-end flow appreciated).
Is there any updates regarding this thread?