# Incremental reflections with union **URL:** \ **Category:** Uncategorized\ **Created:** [June 11, 2018, 2:46pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285 "2018-06-11T14:46:13Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![muv](https://avatars.discourse-cdn.com/v4/letter/m/e9a140/32.png) [@muv](https://community.dremio.com/u/muv)\ **Post date:** [June 11, 2018, 2:46pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/1 "2018-06-11T14:46:13Z")
Do I understand it right that, having an incremental refresh policy on a physical dataset, Dremio will attempt to refresh reflections for derived datasets also incrementally? If this is the case, then does Dremio support incremental reflections for derived datasets **with unions**? The reason I’m asking is that, according to Dremio documentation, incremental reflections don’t work for derived datasets with joins. I plan to have a derived dataset with a code like this: ``` SELECT f1, f2 FROM my_datasource UNION ALL SELECT f2, f1 FROM my_datasource ``` where my\_datasource is a physical dataset with a raw reflection with incremental refresh policy. ---
**Author:** ![bjbhjsj](https://avatars.discourse-cdn.com/v4/letter/b/a9adbd/32.png) [@bjbhjsj](https://community.dremio.com/u/bjbhjsj)\ **Post date:** [July 12, 2018, 3:22am UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/2 "2018-07-12T03:22:14Z")
Hi,friend, did you had resoved the problem in dremio 2.0? ---
**Author:** ![muv](https://avatars.discourse-cdn.com/v4/letter/m/e9a140/32.png) [@muv](https://community.dremio.com/u/muv)\ **Post date:** [July 12, 2018, 8:18am UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/3 "2018-07-12T08:18:28Z")
No. I raised it separately as a support ticket and was told by Dremio support that this would not currently work. We had already version higher than 2.0 and I did not receive any updates on this since then. ---
**Author:** ![dealercrm](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/dealercrm/32/627_2.png) [@dealercrm](https://community.dremio.com/u/dealercrm)\ **Post date:** [July 18, 2018, 10:22pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/4 "2018-07-18T22:22:11Z")
I found that it bypasses the incremental reflection of the physical source and simply does a full refresh with a direct query to the source. This definitely would be a huge improvement if it was fixed in the next release. There are numerous reasons why this is important. For me, the top of the priority list is getting very large data sets into Dremio when they fail or they are too lengthy with a full update. There is really no other way to split up the extract into Dremio from that I can tell. ---
**Author:** ![can](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/can/32/40_2.png) [@can](https://community.dremio.com/u/can)\ **Post date:** [July 19, 2018, 1:33am UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/5 "2018-07-19T01:33:51Z")
@muv @bjbhjsj @dealercrm we currently support incremental update for datasets that don’t include joins or unions – using full update is the expected behavior otherwise. Incremental update enhancement for such patterns is on our roadmap – but not currently slated towards any immediate release. If you are an enterprise edition customer, we should work with your account executive so that we can better understand impact of this on your use cases and discuss priorities. ---
**Author:** ![akikax](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@akikax](https://community.dremio.com/u/akikax)\ **Post date:** [September 18, 2018, 12:07pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/6 "2018-09-18T12:07:39Z")
Just came across the same scenario, I am wondering about my options available to workaround this limitation? ---
**Author:** ![akikax](https://avatars.discourse-cdn.com/v4/letter/a/ee7513/32.png) [@akikax](https://community.dremio.com/u/akikax)\ **Post date:** [September 20, 2018, 9:19pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/7 "2018-09-20T21:19:43Z")
Any ideas on this? The only way I have been able to work around this is by dumping the very last union into a scratch table and creating a reflection on it (but this is far from elegant) ---
**Author:** ![anthony](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/anthony/32/670_2.png) [@anthony](https://community.dremio.com/u/anthony)\ **Post date:** [September 21, 2018, 2:25pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/8 "2018-09-21T14:25:40Z")
@[akikax](https://community.dremio.com/u/akikax) As mentioned [earlier here](https://community.dremio.com/t/incremental-reflections-with-union/1285/5), the approach you mentioned is probably best (else would require full update). The ability for incremental update on a VDS level is something being tracked for the future. ---
**Author:** ![kprifogle](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/kprifogle/32/1719_2.png) [@kprifogle](https://community.dremio.com/u/kprifogle)\ **Post date:** [December 4, 2019, 3:47pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/9 "2019-12-04T15:47:24Z")
Does this mean that if you have incremental refresh turned on then you cannot use UNION queries and expect it to use the reflections currently? ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [December 4, 2019, 11:21pm UTC](https://community.dremio.com/t/incremental-reflections-with-union/1285/10 "2019-12-04T23:21:30Z")
@kprifogle [Incremental Refresh](https://docs.dremio.com/acceleration/updating-reflections.html#full-and-incremental-refresh) has restrictions and if your VDS has one of them, it would fall back to full refresh Thanks