Once a directory has been converted into a dataset, is there a way to display the list all the files in this directory ?
Either with the UI or with a special field available in the dataset, both would be great !
Hive provides the “INPUT__FILE__NAME” field for instance.
There is currently no way to get a list of all the files once a folder has been promoted. You can select individual files using SQL but in the UI we don’t have a way to do that yet.
If you promoted source.folder and folder contains file1, you should still be able to run select * from source.folder.file1. even if folder is a dataset.