Can I proactively refresh metadata?

I sometimes modify the database and hope to be able to query the newly added tables from the database in real time. Can I actively refresh the metadata instead of refreshing it every other time through the metadata refresh strategy?
Or can dremio trigger some events when I add tables to the database?
@balaji.ramaswamy

1 Like

You can try this command

ALTER PDS Alfresco.public.alf_node REFRESH METADATA FORCE UPDATE

https://docs.dremio.com/software/advanced-administration/metadata-caching/#performing-manual-refreshes

@unni If your data is on HDFS or Hive, FORCE UPDATE is not required as the default without anything called LAZY UPDATE, Dremio will skip the refresh when there are no changes to the promoted folder while in FORCE UPDATE, it would not do the timestamp check. If data is on S3, the 2 commands are the same

@unni @balaji.ramaswamy Thanks, I will try this way.

@balaji.ramaswamy @unni
This method seems to only refresh the dataset, while I need to refresh the source. I looked at the source code and it seems that the method for refreshing the source (refreshFull in SourceMetadataManager) is not exposed and cannot be called through the restful api.

@bigfacewo That is correct but we are working on a feature to do on demand forget and/or refresh of all promoted datasets, will be available in the coming versions