Hi, I have five files each for separate day, the file naming convention is such that it contains the date.
I need to union these files and add a column to include the date extracted from the file name.
Need help on how can I do this in Dremio.
Regards,
SJ
Hello @sajjan.jindal,
You could try creating a partitioned dataset. Place each file in its own directory, and give the directory a name that replicates the filename. Then place all of these directories in a container directory, and promote this container directory to a physical dataset. If you name the directories appropriately, Dremio should be able to give a “union” view of all the files. See: https://docs.dremio.com/data-sources/files-and-directories.html#partitioned-datasets
Hi Ben,
Thanks for the details.
I have created a directory and added five text delimited files with same structure in this directory. After mounting the source, when I try to convert the directory to a file “directory pointing to a directory with a table icon”, I am getting the error ’ Something went wrong’ on dataset settings. how can I check what is the issue?
Regards,
Sajjan.
Hi @sajjan.jindal, you should be able to find more information in Dremio’s server.log. Try following this log as you reproduce the error.