Dremio Table Format as Parquet Dynamically

Dear Team,

I’m new bee to the Dremio and in exist system, a normal table will be created on daily basis. Once this process is done, manually we will change the table in Parquet format, set auto reflection of data on every 1 hour in settings and save it in our project folder with Custom name.

Hence we do the said process manually, we are facing some issues in automate the reports .

Kindly give us a solution to do this manual work by scripting or any other best methods.

Thank You,

WR,
Sathish R

@sathhishcse

Once the data is available in the datalake, you can do the below

Let Dremio discover the data, this can automatically done by the background refresh job, see the source settings, under the metadata tab (or) you can run a command “alter PDS refresh metadata”, documentation for both below


Next, you would want to refresh the reflection, for this you can use the API provided

Then you are ready to run your dashboards

Thanks
Bali

Hi Balaji,

We have a mandatory manual daily exercise in Dremio which is used to enable the reports. I have documented our activity and request you to guide us to automate the same using any scripts.

WR,

Sathish R

(Attachment Daily Dremio Activity.docx is missing)

Attachment as Zip file.

Daily Dremio Activity.zip (117 KB)