Migrating all data and configs when upgrading Dremio version

Hi everyone!!

I want to migrate my deployment from the current version 4.8 to the latest one 13.0.
My question is: how can I migrate everything (data, users, etc) to this new deployment? I dont want to lose any kind of existing data/configuration.

By looking into the documentation there’s an option to do a backup/restore but it says it doesnt work on different Dremio versions.

And here it also says that I can stop the project, deploy a new instance and attach again to the new one. But it was not clear to me if this will ensure I will have all the existing data/users/configuration there.

Hello tiibra,
When I tested the upgrade, after I attached the new instance to the existing project, the Dremio environment was upgraded and all users, sources etc were intact.
Hope this helps.

Hi Michael,

Thanks for your answer. I will try this way and see how it goes.

Confirm there are backups being taken of the EBS for the dremio 4.8 instance, if there are you should be able to stop the project in the UI, then provided you are using the same account to create the new instance, Dremio will find the S3 with almost everything. What it doesn’t get is the user uploaded datasets (uploaded via the UI) and any data info since the last successful backup. To get those you need to play with the dremio admin backup tool on the running dremio 4.8 instance and extract them from what the backup contains. There’s also a tool on github that has worked but has somewhat difficult setup and may not be stable (last tested ~4.6) called dremio cloner it is worth a try if the first method fails. As another backup method, you can also query the sys schema into Excel (if not using https) or use the API some to get most of the contents programmatically using Python using similar methods as that tool.

Good points - note that the Dremio cloner has a new home - see GitHub - deane-dremio/dremio-cloner