First I get the id of the file using the GET call (‘api/v3/catalog/by-path/’)
The Id returned from the above API call is below and is stored in a variable called catalog_id:
“id”:“dremio:/Data Lake/SOME-BUCKET/SampleDremio/Testfile/“Testfile1.tsv””
From the same API, i also get the path which is stored in a variable called catalog_path :
“path”:[“Data Lake”,“SOME-BUCKET”,“SampleDremio”,“Testfile”,"“Testfile1.tsv”"]
To promote this file to a dataset, I am using the POST call (‘api/v3/catalog/{id}’). The payload for the POST call looks as :
payload = {
“entityType”: “dataset”,
“id”: “dremio%3A%2FData%20Lake%2Fj%2FSampleDremio%2FTestfile%2F"Testfile1.tsv”",
“path”: catalog_path,
“type”: “PHYSICAL_DATASET”,
“format”: {
“type”: “Text”,
“fieldDelimiter”: “\t”,
“lineDelimiter”: “\n”,
“escape”: “”",
“skipFirstLine”: True,
“extractHeader”: True,
“trimHeader”: True,
“autoGenerateColumnNames”: True
}
}
Everytime i run this python script and it returns:
{“errorMessage”:“Something went wrong”,“moreInfo”:“HTTP 404 Not Found”}
If I change it to the folder instead of the file, it gets promoted but not sure why it is nt working at the file level.
ID - “dremio:/BRDF Storage Account/curated/TSA/ClaimsData2007-2009/TsaClaimsData2007-2009Dataset.parquet”
post url - {server_name}/api/v3/catalog/dremio%3A%2FBRDF%20Storage%20Account%2Fcurated%2FTSA%2FClaimsData2007-2009%2FTsaClaimsData2007-2009Dataset.parquet