This POST call updates all reflections from the physical dataset, right?
I can’t use it because I have several virtual datasets with reflection that come from the same physical dataset, and I need to programmatically update only the reflection of one virtual dataset.
Or can I use this POST in some way with a virtual dataset?
Tks!
In short, I would need to programmatically reproduce for a single reflection what Dremio does scheduled for the reflections auto refresh.
Currently that is not possible. One way is to disable and enable the reflection on the VDS. Looks like you can pass enabled: false
{
“entityType”: “reflection” [immutable, generated by Dremio],
“id”: String [immutable, generated by Dremio],
“tag”: String [immutable, generated by Dremio],
“name”: String,
“enabled”: Boolean,
“createdAt”: String (RFC3339 date) [immutable, generated by Dremio],
“updatedAt”: String (RFC3339 date) [immutable, generated by Dremio],
“type”: String [“AGGREGATION”, “RAW”] [immutable after creation],
“datasetId”: String [immutable after creation],
“currentSizeBytes”: Number [immutable, generated by Dremio],
“totalSizeBytes”: Number [immutable, generated by Dremio],
“status”: Reflection Status [immutable, generated by Dremio],
“dimensionFields”: [Reflection Field With Granularity] [optional, only valid for AGGREGATION],
“measureFields”: [Reflection Field] [optional, only valid for AGGREGATION],
“displayFields”: [Reflection Field] [optional, only valid for RAW],
“distributionFields”: [Reflection Field] [optional],
“partitionFields”: [Reflection Field] [optional],
“sortFields”: [Reflection Field] [optional],
“partitionDistributionStrategy”: String [“CONSOLIDATED”, “STRIPED”],
}
http://docs.dremio.com/rest-api/reflections/put-reflection.html
Hello @balaji.ramaswamy ,
this old thread raised a problem we would like to improve on: trigger a refresh of a reflection using the API without having to do a “disable” of the reflection. As @joelhansen it’s a problem because users are still posting queries when the reflection is being refreshed.
What are the plans to implement such API triggered reflection refresh without disabling the last available version of it ?
Is there a workaround to this current limitation (we use v24) ?
Thanks & Rgs,
Alexandre
@allCag The feature to refresh at VDS level is coming up. At present either background refresh or API catalog refresh on opne of the PDS on which the VDS is defined or disable/enable (This option however will lose job history)
Let me find out on timelines on all the new features coming for reflection management 2.0
Hi @balaji.ramaswamy ,
that’s great to hear that this feature will come !
Could you update on the timeline for the delivery of this Reflection Management v2.0 ?
Rgs,
Alexandre
@allCag VDS level refresh is slated for this year but no exact ETA, the release notes should have it, please check here in a few months and hopefully will have a version/date
Is there a comprehensive doc on Reflection management? Please do post a link here if possible. Thanks!