Concurrent DML operation on Iceberg table

1

Hi Team,

We are posting dremio jobs through Dremio’s SQL API. We are trigerring subsequent SQL jobs that update an Iceberg table, however some of the jobs are failing with the below error:

Concurrent DML operation has updated the table, please retry. (Attached screenshot for your reference)

Please let us know if we can issue multiple DML commands on an Iceberg table concurrently.

Iceberg_ConcurrencyUpdate_Failed
Iceberg_ConcurrencyUpdate_Failed2689×340 25.6 KB

Thanks
Sumi

2

Hi @Sumithra

Looks like the commit is happening at the same time, if you introduce a delay between the SQL statements, does it still happen?

Thanks
Bali

3

Hi @balaji.ramaswamy ,

Yes, we tried introducing delay between the SQL statements, but still some updates coincides with the commit of other update statements and is failing some of the updates. Please let us know if there is a way around issuing multiple update statements on the same iceberg table simultaneously.

Thanks
Sumi

4

@Sumithra Currently no, is this because you have updates from different users ?