# How to setting metadata in dremio system

**URL:** <https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482>\
**Category:** Uncategorized\
**Created:** [November 22, 2017, 7:32am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482 "2017-11-22T07:32:21Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![JoyJava](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/joyjava/32/293_2.png) [@JoyJava](https://community.dremio.com/u/JoyJava)\
**Post date:** [November 22, 2017, 7:32am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/1 "2017-11-22T07:32:21Z")

</div>

Our system has its own metadata, how can we set our metadata to dremio’s metadata?  
Can you provide related documents or interfaces?

THX

---

<div class="post-metadata">

**Author:** ![kelly](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/kelly/32/11_2.png) [@kelly](https://community.dremio.com/u/kelly)\
**Post date:** [November 22, 2017, 12:55pm UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/2 "2017-11-22T12:55:06Z")

</div>

Can you say a little more about the kind of metadata you have and how it should be used by Dremio?

---

<div class="post-metadata">

**Author:** ![JoyJava](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/joyjava/32/293_2.png) [@JoyJava](https://community.dremio.com/u/JoyJava)\
**Post date:** [November 24, 2017, 8:31am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/3 "2017-11-24T08:31:37Z")

</div>

We want to be able to use the latest metadata in real time, instead of relying on dremio’s timer to refresh the metadata, is there any way to do this?

---

<div class="post-metadata">

**Author:** ![can](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/can/32/40_2.png) [@can](https://community.dremio.com/u/can)\
**Post date:** [November 27, 2017, 6:16pm UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/4 "2017-11-27T18:16:43Z")

</div>

@JoyJava you may want to check out Metadata Caching settings on the source settings screen. These options are set individually for each source. The following should get you closer to real-time:

- Dataset Discovery - Fetch interval to 1 minute.
- Dataset Details - Fetch mode to “As needed”.
- Dataset Details - Expire after to 1 minute.

Note that this is not usually recommended due to query time performance considerations.

![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/cdcd3cb3c04f071888c477132de83947dae0e2fa.png)

---

<div class="post-metadata">

**Author:** ![Isabella\_Zong](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/isabella_zong/32/386_2.png) [@Isabella\_Zong](https://community.dremio.com/u/Isabella_Zong)\
**Post date:** [January 22, 2018, 7:47am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/5 "2018-01-22T07:47:52Z")

</div>

@can which version supports this metadata caching?or is this real-time solution only supported in Enterprise Edition.  
my setting page displayed as following:  
 ![dremio_dataset_setting](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/5d6f049f6643a5e31ab7307bed8ffe6a2a82b00a.png)  
Dateset Discovery option is not displayed.

---

<div class="post-metadata">

**Author:** ![JoyJava](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/joyjava/32/293_2.png) [@JoyJava](https://community.dremio.com/u/JoyJava)\
**Post date:** [January 22, 2018, 7:57am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/6 "2018-01-22T07:57:02Z")

</div>

> [@Isabella\_Zong](#):
>
> @can which version supports this metadata caching?or is this real-time solution only supported in Enterprise Edition.
> 
> my setting page displayed as following:
> 
> Dateset Discovery option is not displayed.

We’ve got it, and we found this statement in the source code,  
**SQL is as follows: ALTER TABLE “es-test”. “Xxxx”. “Default” REFRESH METADATA**

Hope to be useful to you

---

<div class="post-metadata">

**Author:** ![dfleckinger](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/dfleckinger/32/370_2.png) [@dfleckinger](https://community.dremio.com/u/dfleckinger)\
**Post date:** [January 22, 2018, 4:12pm UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/7 "2018-01-22T16:12:26Z")

</div>

Is there also any way to set the refresh delay or expiration of a reflection / acceleration ?

eg i have : a source data table, let’s say country where the refresh is set to every day  
another source data table, events, where the refresh is set to be every 2 hours.

I have an acceleration dataset that is calculation from country and events, and I want it to be refreshed only once per day, not more. Currently I don’t think there is any setting to define the reflection/acceleration refresh delay, isn’t it ?

---

<div class="post-metadata">

**Author:** ![can](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/can/32/40_2.png) [@can](https://community.dremio.com/u/can)\
**Post date:** [January 22, 2018, 8:03pm UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/8 "2018-01-22T20:03:13Z")

</div>

@Isabella_Zong all editions of Dremio support metadata caching. As @JoyJava mentioned, you can use the `ALTER TABLE X REFRESH METADATA` command to force a refresh. Also, Dataset Discovery option does not apply for file-system sources such as HDFS, MapR-FS or NAS – as we retrieve file/folder listings for these source types in the UI live.

@dfleckinger you can configure Reflection refresh and expiration by going to settings for a physical dataset or data source. Note that settings on the physical dataset will overwrite the source-wide settings.  
 ![image](https://us1.discourse-cdn.com/flex020/uploads/dremio/original/1X/39aed51071b3a144d0f52a4002a7042feed3505b.png)

For joins, Dremio uses the smaller refresh interval as the refresh interval for that reflection.

It is important to note that reflection refresh is independent of dataset metadata refresh. Reflection refresh jobs will use whatever metadata is available for that dataset at the time of reflection refresh.

---

<div class="post-metadata">

**Author:** ![Isabella\_Zong](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/isabella_zong/32/386_2.png) [@Isabella\_Zong](https://community.dremio.com/u/Isabella_Zong)\
**Post date:** [January 23, 2018, 2:48am UTC](https://community.dremio.com/t/how-to-setting-metadata-in-dremio-system/482/9 "2018-01-23T02:48:53Z")

</div>

@can, @JoyJava  
thanks for your kindly help!  
i used ALTER PDS “dataset path” REFRESH METADATA command to refresh physical dataset, it works all right.  
i’m not sure if it has the same meaning as ALTER TABLE X REFRESH METADATA command.
