# Daily delta load instead of daily full load for source files **URL:** \ **Category:** Uncategorized\ **Created:** [February 4, 2024, 2:58pm UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467 "2024-02-04T14:58:42Z")\ **Posts on this page:** 7\ **Page:** 1
**Author:** ![crazyisjen](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/crazyisjen/32/4552_2.png) [@crazyisjen](https://community.dremio.com/u/crazyisjen)\ **Post date:** [February 4, 2024, 2:58pm UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/1 "2024-02-04T14:58:42Z")
dremio ver: 19.0 As I have to append and update the daily delta change records to previous snapshot of table instead of full historical load. However dremio not able to perform insert select operation. For workaround there is a “incremental load” option for PDS (just insert but not support update) If the current design have 3 layers and I choose the incremental load for fast performance and build reflection for both vds #1 & vds #2 PDS \> VDS#1 (remove the duplicates of old records for same id) \> VDS#2 (agg.) will both reflections run on a incremental basis or full basis? will vds#2 inherit the incremental load? will the incremental just apply to vds#1? Would there some workaround to apply incremental load since takes tons of time to do the full load. Many thanks for solving my pain point! ---
**Author:** ![crazyisjen](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/crazyisjen/32/4552_2.png) [@crazyisjen](https://community.dremio.com/u/crazyisjen)\ **Post date:** [February 7, 2024, 8:07am UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/2 "2024-02-07T08:07:58Z")
hi any idea ~ thanks a lot ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [February 14, 2024, 11:58pm UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/3 "2024-02-14T23:58:10Z")
@crazyisjen Have you thought about Iceberg table format so you can do DML? ---
**Author:** ![crazyisjen](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/crazyisjen/32/4552_2.png) [@crazyisjen](https://community.dremio.com/u/crazyisjen)\ **Post date:** [February 16, 2024, 9:58am UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/4 "2024-02-16T09:58:14Z")
Yes I can try the Iceberg table, can ver.19 can support DML? as I tried seems cannot run insert/update statement to the table When detecting the source file data is changed, will there a way to have a SCD2 capture (store historical data ) automatically after setting the business key and changing column in the configure page? If the S3 file was not convert into iceberg table, can I still perform DML? Thanks a lot! ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [February 20, 2024, 11:00pm UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/5 "2024-02-20T23:00:19Z")
@crazyisjen V19 does not support DML, best is to be on the latest 24.x Any reason you are on 19.x? ---
**Author:** ![crazyisjen](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/crazyisjen/32/4552_2.png) [@crazyisjen](https://community.dremio.com/u/crazyisjen)\ **Post date:** [February 22, 2024, 6:53am UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/6 "2024-02-22T06:53:57Z")
okay let me try to upgrade to 24.x is that need to convert the s3 files into iceberg table first before perform update/ insert? If for PDS (import from database source table), is it also can perform DML? Any aspect of limitation of using iceberg table thanks ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [March 3, 2024, 12:43am UTC](https://community.dremio.com/t/daily-delta-load-instead-of-daily-full-load-for-source-files/11467/7 "2024-03-03T00:43:06Z")
@crazyisjen First upgrade and then convert