Atlas Integration

Do you plan on supporting Apache Atlas in Dremio. It would be great to export the virtual layers to atlas and also get metadata from Atlas to describe some tables/columns.

2 Likes

Hey @chrisJamesC, thanks for the feedback! For exporting Dremio metadata, we’re initially planning to expose dataset/catalog APIs to allow users to manage datasets and general catalog information. For importing metadata from Atlas: As Dremio adds more data catalog type functionality (dataset/column descriptions, etc.), we’ll also look into syncing/bootstrapping metadata from different systems, including Atlas. Probably, this will initially be done through the same dataset/catalog API, followed by first class integrations.

Would be great to get your feedback and how this would fit into your use-case needs (any details on your end-to-end flow appreciated).

Is there any updates regarding this thread?