Unable to list children of FOLDER by API

1

I wanted to use the dremio api to create a PDS for each file of a folder inside my data source (S3).

This is what I did:

  1. GET /catalog
    • response:
{'data': [{'id': 'YYYYYYYYYY',
   'path': ['@DremioAdmin'],
   'tag': '0',
   'type': 'CONTAINER',
   'containerType': 'HOME'},
  {'id': 'ZZZZZZZZZZ',
   'path': ['Space'],
   'tag': '0',
   'type': 'CONTAINER',
   'containerType': 'SPACE'},
  {'id': 'XXXXXXXXXX',
   'path': ['DremioS3Source'],
   'tag': '0',
   'type': 'CONTAINER',
   'containerType': 'SOURCE'}]}
  1. grab source id (XXXXXXXXXX)
  2. GET /catalog/XXXXXXXXXX
    • response (only the ‘children’ values):
{...
'children':[
{'id': 'dremio:/DremioS3Source/"JJJJJJJJJJ"',
  'path': ['DremioS3Source', '"JJJJJJJJJJ"'],
  'type': 'CONTAINER',
  'containerType': 'FOLDER'},
 {'id': 'dremio:/DremioS3Source/"KKKKKKKKKK"',
  'path': ['DremioS3Source','"KKKKKKKKKK"'],
  'type': 'CONTAINER',
  'containerType': 'FOLDER'}]
...}
  1. grab folder id (‘dremio:/DremioS3Source/“KKKKKKKKKK”’)
  2. GET /catalog/‘dremio:/DremioS3Source/“KKKKKKKKKK”’
    • response:
{'errorMessage': 'Something went wrong', 'moreInfo': 'HTTP 404 Not Found'}

So the problem is, i grab this really weird id from the ‘children’ of the data source (not like the other hashes), and when I request the using the catalog, nothing shows up.

Why is this happening?
Are there any workarounds to it?

(BTW, I thought it was worth mentioning that this data source is not inside an S3 bucket, but on the root directory of my S3, so each folder inside the source is actually a bucket).

2

@tonio

You most likely need to URL encode the ID to make it URL safe. For example, dremio:/DremioS3Source/"KKKKKKKKKK" becomes dremio%3A%2FDremioS3Source%2F%22KKKKKKKKKK%22

1 Like
3

Exactly that, thank you!