# Newly appended data on hdfs is not refreshing in physical dataset **URL:** \ **Category:** Uncategorized\ **Created:** [August 13, 2020, 2:32pm UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091 "2020-08-13T14:32:21Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![ankitaKamat](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@ankitaKamat](https://community.dremio.com/u/ankitaKamat)\ **Post date:** [August 13, 2020, 2:32pm UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/1 "2020-08-13T14:32:21Z")
Is there a direct or automated way to automatically refresh the newly appended data in hdfs folder path to be visible in physical dataset, When i fire a sql query “alter table \<\> refresh metadata;” it refreshes the data… but i want a direct or automated way… Is there any property or any API to do this? PS- I am using community version ---
**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\ **Post date:** [August 14, 2020, 7:31am UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/2 "2020-08-14T07:31:50Z")
@ankitaKamat You can set the background refresh to a frequency that will suit your data freshness needs [Caching Source Metadata](https://docs.dremio.com/advanced-administration/metadata-caching.html) Thanks Bali ---
**Author:** ![ankitaKamat](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@ankitaKamat](https://community.dremio.com/u/ankitaKamat)\ **Post date:** [August 14, 2020, 8:02am UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/3 "2020-08-14T08:02:32Z")
@balaji.ramaswamy My data is on HDFS,so i am using HDFS data source… The above caching source metadata does not support to HDFS data source… it is specified in the link which you mentioned in the reply Is there any other option? ---
**Author:** ![ben](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/ben/32/1367_2.png) [@ben](https://community.dremio.com/u/ben)\ **Post date:** [August 14, 2020, 9:32pm UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/4 "2020-08-14T21:32:04Z")
@ankitaKamat, You could use the [SQL API](https://docs.dremio.com/rest-api/sql/post-sql.html) to submit the “alter table \<\> refresh metadata” command. And, as @balaji.ramaswamy mentioned, you can adjust the [background refresh interval for your HDFS source in Dremio](https://docs.dremio.com/data-sources/hdfs.html#metadata). ---
**Author:** ![ankitaKamat](https://avatars.discourse-cdn.com/v4/letter/a/9d8465/32.png) [@ankitaKamat](https://community.dremio.com/u/ankitaKamat)\ **Post date:** [August 17, 2020, 10:08am UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/5 "2020-08-17T10:08:18Z")
@ben, thanks for you response… but i am unable to find metadata option in my HDFS source, is it because i am using community edition? ---
**Author:** ![ben](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/ben/32/1367_2.png) [@ben](https://community.dremio.com/u/ben)\ **Post date:** [August 21, 2020, 5:17am UTC](https://community.dremio.com/t/newly-appended-data-on-hdfs-is-not-refreshing-in-physical-dataset/6091/6 "2020-08-21T05:17:50Z")
@ankitaKamat, you should have this feature with Community Edition. If you are not and Admin in Dremio, then you will not be able to see it: ![Screen Shot 2020-08-20 at 10.16.30 PM](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/2X/d/dcc6c6542bad717ffccbc51df0ab8c61662334b1.png)