How to Refresh Reflection for a Specific VDS in Dremio

1

Hi team,

I’m using Dremio Community Edition v25.x. I have a setup where two VDS views (let’s call them VDS_1 and VDS_2) are created from the same table (test_table). Then I created a final view (VDS_FINAL) as a UNION ALL of VDS_1 and VDS_2. I have reflections enabled on : VDS_1 and VDS_2.

From my understanding, reflection refresh is only possible at the physical table level. So, refreshing the reflection on test_table should automatically refresh the reflections on VDS_1 and VDS_2.

However, is there a way to refresh the reflection only for a specific VDS (for example, just VDS_2) without refreshing the base table?

Thanks in advance!

2

@JoiceJacob VDS refresh feature is in 25.x

ALTER VIEW <view_name>
  [ AT { REF[ERENCE] | BRANCH } <reference_name> ]
  { { ALTER | MODIFY | CHANGE } COLUMN <old_column_name> <new_column_name> <data_type>
  | ADD COLUMNS ( <column_name1> <data_type> [,... ] )
  | DROP COLUMN <column_name>
  | { ADD | DROP } PARTITION FIELD { <column_name> | <partition_transform> }
  | CREATE EXTERNAL REFLECTION <reflection_name> USING <view_name>
  | CREATE AGGREGATE REFLECTION <reflection_name> USING  
    { DIMENSIONS ( <column_name1>, <column_name2>, ... )
    | MEASURES ( <column_name1> ( <aggregation_type>, <column_name2> <aggregation_type> , ... ) )
    | DIMENSIONS ( <column_name1>, <column_name2>, ... ) MEASURES ( <column_name1> ( <aggregation_type>, <column_name2> <aggregation_type> , ... ) ) }
    [ PARTITION BY ( <column_name1>, <column_name2>, ... ) ]
    [ LOCALSORT BY ( <column_name1>, <column_name2>, ... ) ]
  | CREATE RAW REFLECTION <reflection_name> USING DISPLAY ( <column_name1>, <column_name2>, ...)
    [ PARTITION BY ( <column_name1>, <column_name2>, ... ) ]
    [ LOCALSORT BY ( <column_name1>, <column_name2>, ... ) ]
  | DROP REFLECTION <reflection_name>
  | REFRESH REFLECTIONS
  | ROUTE REFLECTIONS TO { DEFAULT ENGINE | ENGINE { <engine_name> | <engine_uuid> } }
3

Hi @balaji.ramaswamy

Thank you for your response.
Based on the document you shared:

  1. I understand that it’s possible to refresh a specific VDS reflection using the Reflection API—please correct me if I’m mistaken.
  2. Does the Reflection API (particularly for refreshing a specific VDS) work only in the Enterprise Edition, or is it also supported in the Community Edition?

Thanks in Advance !

4

Hi @balaji.ramaswamy

Can you please give an update on above points ?

Thanks in Advance

5

Refreshing a reflection from an anchor view is an EE feature. It’s really just a shortcut for finding all the physical datasets within the view and triggering a refresh from those physical datasets. The basic idea is that we want all reflections downstream from those physical datasets to be in a consistent state.