Reflections and stale data

1

Hi,
Is this an expected behaivour?
I have a source table (postgress) . table has 3 columns trxid,customerid,amount
I create a raw reflection for this table including only 2 columns : trxid,customerid

Then i enter a new record into postgress table. I dont refresh the reflection.
case 1 : when i select all columns from the table , dremio retrieves data from postgress because reflection does not include all columns. The result set includes the new row. This is expected.
case 2: when i select only the columns that the reflection has , dremio retrieves data from the reflection. the result set does not include the new row. Is this expected behaviour?

thanks
tolga

2

@tolgaevren, yes this is expected. Dremio redirected your query to use the reflection instead of sending it to Postgres, and the last time the reflection was refreshed this new row did not exist.

If you refresh the reflection, this new row should appear.

3

Thanks Jacob,
Than the two sql has different resultset , this is little bit confusing. But if it’s the expected behaviour of Dremio , its ok.
tolga

4

Learn about what reflections are and how to use them here: Accelerating Queries with Reflections | Dremio Documentation

5

Hi Lenoy ,
Thanks for yur help , i understood the reflections.
thx
tolga