I ve read Refresh Reflections | Dremio Documentation this part:
Both the anchor table and the reflection must be partitioned, and the partition transforms that they use must be compatible.
When a reflection is defined on a view that joins two or more anchor tables…
I already use the benefits of incremental reflection of vds that involves single anchor AWS GLUE table with partitions == reflection partitions. And it works fine - my reflection refreshes are much faster and only involve partititions changes in GLUE.
But can i have same benefits in case I join this vds(or anchor table directly) to other vds:
see vds_joined:
select * from vds_partitioned join vds_other on ...
where vds_partitioned has reflection updated using incremental refresh
vds_other - has reflection using full refresh
parent vds_joined - has reflection with partition == vds_partitioned.partition
when vds_partitioned is incrementally refreshed for changed partition and vds_other reflection was not changed since prev vds_joined refresh, i can imagine incremental refresh can happen for vds_joined where it will load only changed partition data from vds_partitioned and inner join on full data of vds_other and save incremental results.
But seems it doesn’t work now this way and full refresh is done.
Can this be achived somehow?
vds_other is based on couple of RDBMS tables and reflection is not partitioned.
If vds_other reflection would be partitioned(how?) would it help?
Build Info:
- Build: 25.1.6-202501021803480419-127757a4
- Edition: AWS Edition (activated)