Hi ,
I write a python spark code which insert records into a iceberg table and it runs ok. at the end of the program i select the iceberg table to see if the records are inserted. Everything is fine.
But , when i select the same table in dremio sql editor , sql does not return the inserted records.
only way to see the records is to go to the data set menu → table → settings → format. In this page , data is retrieved again and i can see the inserted records.
Is there a cache or etc that i need to check in order to see the inserted records?
thanks
tolga
@tolgaevren Which catalog is the Iceberg table created on. It seems like it is a file system source. So first step is tpo promote the folder to a dataset and then you should be able to query
Hi Balaji,
Here is what i am doing:
I created an iceberg file in dremio . I am using local minio object storage. I can insert , select this table without any problem in Dremio sql panel. When i check the object storage , i see the parquet files are created for my updates inserts etc.
In python i write a jdbc program that directly connects to iceberg.( I am using iceberg jdbc driver)
I am trying to achive to update an ceberg table from independent resources.
In python code also i have no problem insertng selecting records.
Also the records that i insert in dremio sql screen appears in pyrhon jdbc code select .
But the records that i insert inside python iceberg jdbc , does not directly appears in the result set of dremio sql screen. If i goto the table format ( table is already in iceberg format) only then the result set is updated. What part am i missing here?
tolga
Hi Balaji,
I found the answer. When i run
ALTER TABLE cust10 REFRESH METADATA
than i see the updated data.
tolga
@tolgaevren Yes, that will work