# Discover newly added table **URL:** \ **Category:** Uncategorized\ **Created:** [August 5, 2020, 9:34am UTC](https://community.dremio.com/t/discover-newly-added-table/6048 "2020-08-05T09:34:27Z")\ **Posts on this page:** 11\ **Page:** 1
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 5, 2020, 9:34am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/1 "2020-08-05T09:34:27Z")
Hi, wanted to ask how to shorten a time between creating of new table in source and being able to query it in dremio. For example, SF connector (ARP) does not allow to define discover refresh rate, and it takes quite long time before it is available for querying. Additionally, can the discovery be triggered manually? Lastly, the best option would be to discover the table every time someone tries to query it but dremio does not know about it yet. Thanks ---
**Author:** ![Rakesh\_Malugu](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/rakesh_malugu/32/2384_2.png) [@Rakesh\_Malugu](https://community.dremio.com/u/Rakesh_Malugu)\ **Post date:** [August 5, 2020, 9:45am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/2 "2020-08-05T09:45:24Z")
Hello @froxCZ If you know the path of the newly created table then the below will help to see the table in Dremio immediately, Alter PDS {tablename with path} refresh metadata; Thanks, @Rakesh_Malugu ---
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 5, 2020, 10:13am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/3 "2020-08-05T10:13:23Z")
@Rakesh_Malugu Hi! That would be awesome. I tried it, but get different error. I dont understand why… The SF dataset definitely exists. I can run query like `select * from SF."METRICS3_STG".PUBLIC.TEST;` fine, but this fails: `alter PDS SF refresh metadata;` VALIDATION ERROR: Unable to refresh dataset. (com.dremio.connector.metadata.DatasetNotFoundException) null com.dremio.exec.catalog.SourceMetadataManager.refreshDataset():570 com.dremio.exec.catalog.ManagedStoragePlugin.refreshDataset():756 com.dremio.exec.catalog.CatalogImpl.refreshDataset():606 com.dremio.exec.catalog.SourceAccessChecker.refreshDataset():231 com.dremio.exec.catalog.DelegatingCatalog.refreshDataset():209 com.dremio.exec.planner.sql.handlers.direct.RefreshTableHandler.toResult():59 com.dremio.exec.planner.sql.handlers.commands.DirectWriterCommand.plan():99 com.dremio.exec.work.foreman.AttemptManager.plan():427 com.dremio.exec.work.foreman.AttemptManager.lambda$run$1():330 com.dremio.service.commandpool.CommandWrapper.run():62 com.dremio.context.RequestContext.run():95 com.dremio.common.concurrent.ContextMigratingExecutorService.lambda$decorate$3():199 com.dremio.common.concurrent.ContextMigratingExecutorService$ComparableRunnable.run():180 java.util.concurrent.ThreadPoolExecutor.runWorker():1149 java.util.concurrent.ThreadPoolExecutor$Worker.run():624 java.lang.Thread.run():748 Would u know why? ---
**Author:** ![Rakesh\_Malugu](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/rakesh_malugu/32/2384_2.png) [@Rakesh\_Malugu](https://community.dremio.com/u/Rakesh_Malugu)\ **Post date:** [August 5, 2020, 10:26am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/4 "2020-08-05T10:26:57Z")
Hello @froxCZ I mean, alter pds SF.“METRICS3\_STG”.PUBLIC.TEST refresh metadata; can you try this? Thanks, @Rakesh_Malugu ---
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 5, 2020, 11:55am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/6 "2020-08-05T11:55:47Z")
@Rakesh_Malugu I can run that on already discovered table. But if I run it on table that dremio does not know about yet, it fails with the exception I posted. ---
**Author:** ![waynekoepcke](https://avatars.discourse-cdn.com/v4/letter/w/f17d59/32.png) [@waynekoepcke](https://community.dremio.com/u/waynekoepcke)\ **Post date:** [August 5, 2020, 5:00pm UTC](https://community.dremio.com/t/discover-newly-added-table/6048/7 "2020-08-05T17:00:56Z")
Hi, Datasets can be created over the REST API. Is that an option here? Create the dataset when you create the table using the API. ---
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 5, 2020, 5:24pm UTC](https://community.dremio.com/t/discover-newly-added-table/6048/8 "2020-08-05T17:24:30Z")
Will try. But found only documented way to create VDS, not PDS [https://docs.dremio.com/rest-api/catalog/dataset.html](https://docs.dremio.com/rest-api/catalog/dataset.html) Can u tell me how to create pds? ---
**Author:** ![waynekoepcke](https://avatars.discourse-cdn.com/v4/letter/w/f17d59/32.png) [@waynekoepcke](https://community.dremio.com/u/waynekoepcke)\ **Post date:** [August 5, 2020, 9:29pm UTC](https://community.dremio.com/t/discover-newly-added-table/6048/9 "2020-08-05T21:29:07Z")
I use this for creating physical datasets [https://docs.dremio.com/rest-api/catalog/post-catalog.html](https://docs.dremio.com/rest-api/catalog/post-catalog.html) I’m not sure what your underlying datasource is, I have only created datasets on parquet files. An example request body to that API would look like this {“path”: [“employee”], “entityType”: “dataset”, “type”: “PHYSICAL\_DATASET”, “id”: “employee”, “format”: {“type”: “Parquet”}} ---
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 6, 2020, 10:06am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/10 "2020-08-06T10:06:22Z")
Did not get that to work… > Physical Datasets can only be created by promoting other entities. Not even sure it can work. Seems creating PDS is designed only for parquet, csv or other files… not for discovering tables from other DB sources. @Rakesh_Malugu what else can we try? ---
**Author:** ![Rakesh\_Malugu](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/rakesh_malugu/32/2384_2.png) [@Rakesh\_Malugu](https://community.dremio.com/u/Rakesh_Malugu)\ **Post date:** [August 6, 2020, 11:01am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/11 "2020-08-06T11:01:34Z")
Hello @froxCZ The error means that the table didn’t found. (com.dremio.connector.metadata.DatasetNotFoundException) null You will get the same error when you try to run the query on a non-existing table. If you have already a PDS in the newly created table location, then I would recommend you copy the PDS path and just change the table name to newly created one and run the alter PDS command. Thanks, @Rakesh_Malugu ---
**Author:** ![froxCZ](https://avatars.discourse-cdn.com/v4/letter/f/ee7513/32.png) [@froxCZ](https://community.dremio.com/u/froxCZ)\ **Post date:** [August 6, 2020, 11:13am UTC](https://community.dremio.com/t/discover-newly-added-table/6048/12 "2020-08-06T11:13:35Z")
@Rakesh_Malugu I’m not sure you understand my issue. Wehn I create new table in Snowflake, Dremio does know about it. When I run the ALTER PDS statement, Dremio says that table does not exist, even though the table exists in Snowflake already. Dremio is not able to discover it, until periodical Discover names refresh is ran ☹ How can I tell Dremio that table X exists in Snowflake and to register it as PDS?