Create Historical Data in Dremio

Hi, i’m new on Dremio and need some help.
I have a dataset but the information changes every day, I need a query wich I can “collect” the data every day and save in a new VDS, only need maximum 4 days of historical data.

data_information day1
union all
data_information day2
union all
data_information day3
union all
data_information day4

Exist any way to create a new VDS like that?

@caz_chelo What is the data set partitioned on?

@caz_chelo The way I’ve done this is to place each day files in its own folder structure, when I deal with csv/txt

In S3 for example, imagine the ‘folder structure’ is as follows - I hold data going back a year by day.



I then place a PDS on root

Select * from the PDS will give extra column names of dir*

I create a VDS selecting from the PDS and code a conversion to date of the ‘dir’ column to a new column called FileDate

I can then select from that where where the FileDate is > ‘today - 4’

I can then retain data OLDER than 4 days but mask it out.