Thanks @balaji.ramaswamy, I would really appreciate if you could reply to this as well as we are trying to take a decision to use dremio’s enterprise or community version
We have an ETL usecase where we need to extract XML files, Transform, do some aggregations to the data and then load the data to a data warehouse. And we are using Nifi, Kafka and Spark for this, but we also have to adjust our architecture so that the end users can use it as a central data repository hub. Below are the data governance features we need to integrate to our existing ETL architecture
End Users should be able to govern the data assets through a UI, add their own data sources, data orchestration
Data catalog, data discovery, searching
Data governance, data lineage, data privacy ( dynamic masking based on users/group permissions)
Data Quality checks etc
End Users should be able to define custom spark jobs through Web UI - (Apache Zeppelin)
Are these features provided by Dremio’s enterprise version? which version is suitable? is Dremio a suitable platform for the central data repository usecase? I have already gone through Dremio’s features but need to verify in order to come to a conclusion.