Hello Dremioers!
I’m just starting my first steps with Dremio and very excited about its potential. With its specifications, this product has no match in the market.
I completed the University courses on virtual labs and just trying to replay everything on a local install. I have MinIO as storage and created sample data there. The issue I’m facing is that when I try to promote a folder that contains a lot of JSON files, it only sees one file even though all files are present when I browse the folder in Dremio. It could be something silly I’m missing, but I’m completely blank here. The only way to have Dremio read those files at the time of promotion is to compress them, which I don’t think we should do for all files in a lake.
Cheers,
Yasser
@yalmasri When you mean that it only “sees” one file, do you mean that the query results seem to be coming from just one file?
During promotion you will be prompted with sample data, which is in my case only one record. The image below is from MinIO, where it shows multiple files under folder sample_data_1:
While here I’m showing the folder before promotion:
Finally, here the problem occurs, only one file is presented:
What did I do wrong?
@yalmasri Hit run button after you promote, dod you see all rows? Send us the job profile of the run
That worked indeed! How clumsy of me, I saw one row and got intimidated to continue one more step. Like you said; hitting the run button returned all rows in the folder.
Balaji, you are the one!
@yalmasri Yay ! Glad it works now, welcome !
