Primary Key or avoid duplicate records

Hi there!!! :slight_smile:

We are loading some files inside a folder, and setting the format in the folder. Everythings is fine, until someone add a new file in the folder with same records…(with other filename)
How to avoid duplicate records ??
How to define a primary key??
There is some way, to validate that new records wasn’t loading before??

Thanks in advance


Hi @AlejandroJMelo

We currently do not support primary key enforcement in Dremio. You can try and clean the data using SQL after it is promoted in Dremio