Hi, Is there a way to auto delete the reflections? We have set the refresh interval as 1 hour and expiry as 1 hour, so a new reflection is created every 1 hour. But they are not deleted even after expiry. What we understood is there is a grace period after which the expired ones are taken for deletion. Currently it is maintaining about 5 reflections. Can we change that grace period? Our queries are pretty fast and usually run within seconds to minutes. So we do not want such a huge grace period. It is unnecessarily taking space. Thanks in advance.
Dremio version: 24.1.0
The default grace period for removal of deprecated reflections is about 4 hours. You should be able to configure it using support key: reflection.deletion.grace_seconds
You can track reflections and materializations using system tables: sys.reflections and sys.materializations
For example:
select * from sys.reflections where reflection_id = '5951c9a6-cbbf-41d7-aa33-95615163c110'
select * from sys.materializations where materialization_id = 'f0c27f88-3d3b-453b-9e97-772965a7e4b5'
Hi @lenoyjacob, Thanks for your response. We tried adding the reflection.deletion.grace_seconds key to the dremio.conf file. But it throws an error that “not a recognized parameter”. Where should we be adding this key? Is there a specific format? Thanks.
@akshaymehtadremio This is a Dremio support key (does not go into dremio.conf), click on settings-support and enter “reflection.deletion.grace_seconds” in the box provided on the right side that says “support key” then click on “show”, change the value and click save
@balaji.ramaswamy, Thanks for your response. We tried your solution and it worked. We are good to close the request.
Thanks a lot for the update @akshaymehtadremio
