How to add acceleration settings at dataset level

1

I am currently using the below API to set accelerationRefreshPolicy for a dataset.

PUT - /api/v3/catalog/{id}

accelerationRefreshPolicy at dataset level only have these 4 properties (https://docs.dremio.com/rest-api/catalog/dataset.html#accelerationrefreshpolicy-parameter)

{
  "refreshPeriodMs": Number,
  "gracePeriodMs": Number,
  "method": String ["FULL", "INCREMENTAL"],
  "refreshField": String [optional]
}

However, how do I add the following acceleration settings for a dataset.

  1. accelerationNeverExpire
  2. accelerationNeverRefresh

I have checked the entire documentation but could not find, how to set accelerationNeverExpire, accelerationNeverRefresh at dataset level.

For more details check How to add dcceleration settings at dataset level

2

@ben Can you please hep me with this. Have stuck at this since a few days now

3

@nehal,

My apologies for the delayed response.

This looks like a bug, because you can set this in the UI on a particular physical dataset, but not via the v3 API we publish. Furthermore, you can set it on the overall source with v3 API.

This is confusing for users. We will have to fix it.

Meanwhile, you can use the older v2 API endpoint to make the change. Here’s how I would update a dataset called hdfs.hr.employees to never refresh and never expire (using cur)l:

curl --request PUT \
  --url http://{your dremio host}:9047/apiv2/dataset/hdfs.hr.employees/acceleration/settings \
  --header 'authorization: _dremio{token goes here}' \
  --header 'content-type: application/json' \
  --data '{
  "method": "FULL",
  "accelerationRefreshPeriod": 3600000,
  "accelerationGracePeriod": 10800000,
  "accelerationNeverExpire": true,
  "accelerationNeverRefresh": true
}'
1 Like
4

@ben Thank you.
We have done the same thing as of now, using settings API of v2 . Got to know about this API by monitoring the Dremio network calls