Restore functionality broken?

1

Running this command:

sudo -u dremio /opt/dremio/bin/dremio-admin restore --backupdir /tmp/backups -v

Returns
java.io.FileNotFoundException: File /tmp/backups does not exist.
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatusInternal(DistributedFileSystem.java:901)
at org.apache.hadoop.hdfs.DistributedFileSystem.access$600(DistributedFileSystem.java:112)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:961)
at org.apache.hadoop.hdfs.DistributedFileSystem$22.doCall(DistributedFileSystem.java:958)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.listStatus(DistributedFileSystem.java:958)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1537)
at org.apache.hadoop.fs.FileSystem.listStatus(FileSystem.java:1580)
at com.dremio.dac.util.BackupRestoreUtil.scanInfoFiles(BackupRestoreUtil.java:190)
at com.dremio.dac.util.BackupRestoreUtil.validateBackupDir(BackupRestoreUtil.java:229)
at com.dremio.dac.cmd.Restore.main(Restore.java:81)
verify failed java.io.FileNotFoundException: File /tmp/backups does not exist.

Any ideas?

2

So I tried every possible combination and I am almost 100% sure that the restore option does not work when pointing to a local backup folder.

I was able to work around the issue by uploading my local backup folder into hdfs and pointing the restoring option to it.

Wilson

3

BTW, this issue should be easy to replicate and confirm… Let me know if you find yourself in the same situation so we can formally raise the bug

4

Hi @akikax

Are you trying to restore from 2.1 to 3.0? Backup and Restore should be on the same version

Thanks
@balaji.ramaswamy

5

Nahh, from 2.1 to 2.1, same version (I read the docos), you should be able to easily replicate the issue. Same behaviour with 3.0 to 3.0

6

Hi @akikax

Works as expected for me, can you please send me output of below 2 commands?

ls -ld /tmp/backups
ls -ltrh /tmp/backups

Also can you please try this

Change dremio.conf to write local to your home directory say (~/dremiodb), like below

local: “/dremiodb”
mkdir -p ~/dremiodb/db

Try the below command,

sudo -u dremio /opt/dremio/bin/dremio-admin restore --backupdir /tmp/backups -v

7

@akikax, @balaji.ramaswamy, I’ve encountered the same issue, but was able to get it work with local filesystem using the file:// schema prefix in full path. Seems it is a minor bug in implementation which leads to use HDFS by default.

8

Thanks for the update @savermyas

I was aware of that, did not strike to me that you might be on a Hadoop edge node and we default to hdfs://. Great find !