Dremio and HDFS

[Monika_Goel]

Hi Team,

I am reading data from HDFS cluster and doing ETL on top of it in dremio.

Can you please help me to understand how queries are executed in dremio:
Does queries run on dremio cluster or HDFS cluster?
If they run on dremio cluster then how does dremio able to access data from HDFS, does it copy all HDFS data on dremio node?

Thanks in advance.

[yufeldman]

@Monika_Goel

Does queries run on dremio cluster or HDFS cluster?

Queries run on the Dremio executors, data is read from HDFS

@Monika_Goel

If they run on dremio cluster then how does dremio able to access data from HDFS, does it copy all HDFS data on dremio node?

Data is accessed directly from HDFS - streaming - during data reading (scan phase). Data is not stored in Dremio per say.