Reflections can be updated every 30 minutes ?
In the tool, the minimum I can set is hours.
I need to refresh my data every 30 minutes.
I don’t want to go straight to the database, that’s why I need to use reflections.
I think it’s fully managed in the Enterprise Edition.
If you are using OSS version, you may pass through API in order to refresh your reflection, then call the job through your scheduler.
If it can help you, there some Python script to refresh a reflection.
you’ll have to change user/password,dremioServer,port and pathReflection
import json
import requests
import config
username = "user"
password = "psw"
headers = {'content-type':'application/json'}
dremioServer = 'http://yourServer:9047'
loginData = {'userName': username, 'password': password}
response = requests.post(dremioServer+'/apiv2/login', headers=headers, data=json.dumps(loginData))
data = json.loads(response.text)
token = data['token']
headers = {'content-type':'application/json', 'authorization':'_dremio{authToken}'.format(authToken=token)}
def apiGet(endpoint):
return json.loads(requests.get('{server}/api/v3/{endpoint}'.format(server=dremioServer, endpoint=endpoint), headers=headers).text)
def apiPost(endpoint, body=None):
text = requests.post('{server}/api/v3/{endpoint}'.format(server=dremioServer, endpoint=endpoint), headers=headers, data=json.dumps(body)).text
if (text):
return json.loads(text)
else:
return None
def apiPut(endpoint, body=None):
return requests.put('{server}/api/v3/{endpoint}'.format(server=dremioServer, endpoint=endpoint), headers=headers, data=json.dumps(body)).text
def apiDelete(endpoint):
return requests.delete('{server}/api/v3/{endpoint}'.format(server=dremioServer, endpoint=endpoint), headers=headers)
def refreshReflection(pathReflection):
# Should be call like
# refreshReflection('/DOCKER_VOLUMS/5M')
try:
idToRefresh = apiGet('catalog/by-path'+pathReflection)['id']
apiPost('catalog/{id}/refresh'.format(id=idToRefresh))
except Exception as e:
print(f'Error reflection: {pathReflection} ')
pathReflection = '/DOCKER_VOLUMS/TEST/RefreshReflection'
idToRefresh = apiGet('catalog/by-path'+pathReflection)['id']
print(f"ID to refresh is: {idToRefresh}")
apiPost('catalog/{id}/refresh'.format(id=idToRefresh))
Script will return you the ID
And you can check the reflection history:
if you want to schedule it, just run that through schedule every 30min.
Hope it helps.
Y.
1 Like
@jlcornejo On the PDS on which the VDS reflection is build you can change the frequency
thanks! 
I will try this option
There’s a support key to enable subhour refresh policies:
accelerator.enable.subhour.policies
Go to Admin Settings → Support … Add the key, and enable it.
After that, you will be able to select minutes as unit
1 Like