Extract value from source file to add new column

Hi, I have five files each for separate day, the file naming convention is such that it contains the date.
I need to union these files and add a column to include the date extracted from the file name.
Need help on how can I do this in Dremio.


Hello @sajjan.jindal,

You could try creating a partitioned dataset. Place each file in its own directory, and give the directory a name that replicates the filename. Then place all of these directories in a container directory, and promote this container directory to a physical dataset. If you name the directories appropriately, Dremio should be able to give a “union” view of all the files. See: https://docs.dremio.com/data-sources/files-and-directories.html#partitioned-datasets

Hi Ben,

Thanks for the details.
I have created a directory and added five text delimited files with same structure in this directory. After mounting the source, when I try to convert the directory to a file “directory pointing to a directory with a table icon”, I am getting the error ’ Something went wrong’ on dataset settings. how can I check what is the issue?


Hi @sajjan.jindal, you should be able to find more information in Dremio’s server.log. Try following this log as you reproduce the error.