Hi,
i’m playing with the Catalog API to promote/delete parquet datasets.
In the $scratch directory, I’ve promoted a test folder to dataset, then removed the dataset using the API.
Folder name is test.
When calling the API endpoint /api/v3/catalog/by-path/, I get different id in response depending on the fact that I put quotes around folder name
with ‘$scratch/“test”’ (with double quotes)
{u’children’: [{u’id’: u’dremio:/$scratch/test/“1_0_0.parquet”’,
u’path’: [u’$scratch’, u’test’, u’“1_0_0.parquet”’],
u’type’: u’FILE’}],
u’entityType’: u’folder’,
u’id’: u’dremio:/$scratch/“test”’,
u’path’: [u’$scratch’, u’“test”’]}
with ‘$scratch/test’ (without double quotes)
{u’children’: [{u’id’: u’dremio:/$scratch/test/“1_0_0.parquet”’,
u’path’: [u’$scratch’, u’test’, u’“1_0_0.parquet”’],
u’type’: u’FILE’}],
u’entityType’: u’folder’,
u’id’: u’2cf9a662-8905-4abc-b0ad-2b872776eca1’,
u’path’: [u’$scratch’, u’test’],
u’tag’: u’0’}
Does it mean we should always use double quotes around folder names when querying catalog by paths ?
The first id returned works when I try to promote it as a dataset, but the second id does not enable to promote the dataset.
Documentation (https://docs.dremio.com/rest-api/catalog/get-catalog-path.html) does not provide information whether we should use quotes or not.