Ability to import/export Dremio Datasets etc for use in a CI/CD process

1

I am aware that the question has been asked before, so looking for an update.

Has the situation changed, and if not how can I actually achieve it?

2

@spireite Not yet, the only crude way is to download information_schema.views

3

Thanks, I was so hoping that this would have been addressed by now.

4

@spireite, maybe DBT Dremio adapter could help for that, it works fine with DBT and you can create VDS, etc… if it is what you mean.

5

Hi @spireite !
We use gitlab-ci for CD dremio from git sources.
My command developed gradle plugin for this task.
Our basic principles:

  1. Git repository archetype repeats Dremio structure. Directory src/main/dremio contains directories-equivalents of Dremio spaces. Every space contains sql-files with VDS definitions.
  2. There is diff task to calculate the difference: absent on server; present on the server, but the absent definition; the content is different on the server and in the git repository.
  3. Append only mode by default. But there is a force rewrite mode.
  4. One communication protocol for the plugin - REST (no JDBC etc.)
  5. Fail-safe principle - deploy everything is possible. Errors are logged at the end of the process.
  6. Reporting. We use the JUnit format. Space is a test suite. VDS deployment is a test.
6

Interesting, thanks very much @tfedyanin