How to restart the failed reflection automatically

1

Hi, Can anyone help me on this and also I want to see which reflection is failing again and again and its count.

2

Dremio actually not do this automatically, you can develop a custom code using Dremio API to make it.
For example we develop a DAGs using apache airflow, to refresh reflection, and check availability, and notify errors

3

Thank you for the information, Do you have any code with you so that i can take a reference from it

4

@saurabh99

On community edition we give up after 3 times. In Enterprise edition you can customize. I would also be interested in finding out why it failed 3 times. Was the source down? did it run out of memory?and fix the underlying cause

Also the default refresh is every hour. So if you do not want it every hour, you should decrease the frequency so we do not use up the 3 attempts too soon

Thanks
@balaji.ramaswamy

5

When i go on reflection settings i did not found any option of schedule.

6

Hi i find this option on my physical data not on reflection , every time my reflection fail i have to go and manually try to run that, can we set them automatically up …

image

7

VDS not have schedule option, because VDS are always automatically refreshes when parents PDS or VDS are updated.
As @balaji.ramaswamy said only in enterprise edition you can increase the 3 times retry for fail reflection.

But the important ting here is why your reflection fail?
For example in my case dremio are not connect to production database, it’s connect to alternate database
the source are down because are down 3h all days (backup-restore contingence database)
so I program a dag with airflow to run refresh VDS after restore is complete, then check reflection availability each hour, and notify if some are wrong

8

Hi balaji, i can do this? thank you

9

@LucGth

layout.refresh.max.attempts

10

hi, where where should i put / change this parameter? thanks.

11

@sszelenko

Click on admin-support key-enter the above key in the support key box-click show-it should 3 in it,

Increase to the number you want

Thanks
Bali

1 Like
12

Hello.
Would you be able to share or point me to some place that I can search for how to do this custom code using Dremio API to refresh failed reflections?

Thanks

13

@martinocando

http://docs.dremio.com/rest-api/catalog/post-catalog-id-refresh.html

14

Brilliant. Thanks for the information!

15

Welcome @martinocando