Hi everyone,
I’m interested in contributing to Dremio OSS and wanted to get some feedback before starting work on a new feature.
I’d like to propose a read-only API to export the catalog configuration (such as sources, spaces, folders, and virtual datasets) as JSON. The primary use cases are backup, migration between environments, and version controlling catalog definitions.
Before investing time in the implementation, I’d like to know if this is something the maintainers would consider for Dremio OSS, or if there are any existing solutions or design considerations I should be aware of.
Thanks!
Dremio Cloner came to mind when I read your post.
Sorry for the delay and thanks for pointing this out! I wasn’t aware of Dremio Cloner, and after looking into it, it does cover many of the same use cases I had in mind.
From what I understand, though, Cloner achieves the export by orchestrating multiple existing APIs and recursively traversing the catalog from the client side. My idea is slightly different: having a native read-only API in Dremio OSS that can recursively export the catalog configuration in a single operation, which could also simplify tools like Cloner.
I’ll explore Cloner further, but I’d be interested to know if the maintainers/community see value in having such a native export API in OSS.