Create dataset from glob or regex pattern

Continuing the discussion from Exclude certain subfolders when creating a dataset:

Is there any way to promote a dataset by using a glob pattern to include/exclude specific files/folders?

If not, a couple of solutions that might work could be:

  • Option to specify an include or exclude regex pattern when promoting a dataset
  • Option to have Dremio respect a .git_ignore file

I have several different scenarios where I am unable to separate mixed file types without investing an unreasonable amount of time and effort. This limitation is the single factor preventing my company from being able to use Dremio as our real-time analytics and ETL solution.