How to create dataset for daily data report

In my company use the ETL workflow process(in Hive) to make the data for daily BI report,if I want to switch to dremio how to do it?
The workflow run daily and Hive tables are partitioned by day. The daily partitions of every tables are about 5-8GB.
4-5 hive tables have to join together.

Hi @zhang_maosen,

If I understand it right, your current ETL workflow is on Hive. You should be able to add the Hive meta store as source to the Dremio UI and able to run queries through Dremio. I am pasting the link to adding a Hive source below

Once you add the Hive source and run some workloads, you can also create reflections that can accelerate your query further

Kindly let me know if you have any other questions


Thank u for your reply
I could desc more about my problem:
for example:
My Hive table T1 and T2, both are partitioned by column ‘day’. everyday I will insert new day partition into T1 and T2.
Then join T1 and T2 and insert into T3 like this:
insert overwrite into T3 as select T1.*, T2.* from T1 join T2 on where current_day() and

then I create a dataset for T3 like this:
select * from (select T1., T2. from T1 join T2 on where = nest_0
But this query is very slowly and reflection create fail

Hi @zhang_maosen

What error do you get when you try to create a reflection?