I have few questions on dremio and trying to compare with presto.
In presto, we normally create tables in hive, and have presto to run queries by referring to the metadata created in hive. I’m trying to understand the same in dremio, and have below questions.
- Does dremio support meta data management or do we need to have hive similar to presto.
- If dremio supports meta data management, then how to create tables in Dremio. Using docker am able to create a data lake. But not sure how to create tables. Do we need to create PDS and VDS from data lake.
- Is this the only way to create tables (using PDS and VDS)?
- I have partitioned data in ADLS based on specific column and want to create tables accordingly. Not sure how to create table with partition column in dremio.
- How to refresh dremio table whenever a new partition is added in ADLS. As per our use case, we create at least 100 partitions every day to tables. Want to have this reflected in dremio table in real-time.