I used HDFS as the storage of Dremio before, Now we try to connect Dremio to Alluxio cluster (HDFS as Alluxio’s under storage system), but we found that there is no support for “alluxio://” protocol (in dremio.conf) in the existing documents. How can I do this?
@meiosischen Has to be HDFS, does Alluxio provide a way to map it as a HDFS file scheme?
you can use “Alluxio S3 API” as if s3 storage
It may be possible to use the Alluxio “Transparent URI” feature of the enterprise version of Alluxio. This will allow all hdfs:// references to be redirected to the Alluxio shim filesystem (alluxio://).
Also, I have tested Dremio with Alluxio using Alluxio’s S3 API. In Dremio, I configured a new data source using the “Amazon S3 Source” data source type, then I selected the “Enable compatibility mode” checkbox. Finally, I specified several user supplied connection properties in Dremio, like this:
Hope this helps,