How join works with two different data sources

I am trying to join tables from two different data sources. i want to understand the backed process like how Dremio joins data from datasource1.table1 and merge with datasource2.table2

is Dremio transferring data from one source to another?


Based on the query Dremio prepares the plan to read the erquired data/columns/rows from the both sources and provides the results.Dremio do not transfer any data from one source to other.


you mean that every time when we a query Dremio fetches the data through network?

Yes we read data for each query. Dremio has a feature called refelctions which stores the data in parquet format, using reflections the query accelerates and finishes in quick time.


Thanks @Venugopal_Menda !
i need some more clarification on joins with filters.

when i am joining two tables from different data sources by applying filter on data, i want to understand where the data will get filtered. i mean will the dremio read complete data and then filter it or is there any mechanism, that will filter the data and then fetch it?

Hello @madhusudhan,

Depending on the types of sources you are joining, sometimes Dremio will push down the filter into the source (pre-filter) and sometimes it will grab many records from the source and filter in Dremio’s execution engine. Which source would you be joining across?

ben, i am joining data from S3 and postgresql.