Partition on S3 and creating physical table

1

I have a folder partitioned in S3 in the following format/folder.
BucketName SubFolder1 SubFolder2 SubFolder3 folderYear foldermonth parquet file.
At what level should click the folder icon to covert it to a physical file so we can start querying on it. Ideally we would love to roll up at the Subfolder3 level sp that we can query it with year and month without having to explicitly join each file. Would it work?.

2

@Siladitya

If you promote at subfolder3 you will see 2 columns in the dataset

dir0 showing values “year=2020”
dir1 showing values “month=06”

Thanks
Bali

1 Like