Unable to create reflections with Dremio Community Edition

21

Hi Balaji,

We have installed the Dremio on the Oracle Cloud instance and I have configured the dist with S3, but when I am running the reflections I am getting following exception.

(java.util.UnknownFormatConversionException) Conversion = ‘Unknown format (pdfs) conversion for path /opt/dremio/data/pdfs/accelerator/8b5e4a94-6fd2-4048-a55f-ce857e238780/110efb81-8844-413e-9def-df165898286a_0 Error Message : No File System scheme matches’

below is the config file.
paths: {
#the local path for dremio to store data.
#local: ${DREMIO_HOME}“/data”
#local: “/storage/opt/dremio/data/”

the distributed path Dremio data including job results, downloads, uploads, etc

#dist: “pdfs://”${paths.local}“/pdfs”
dist: “dremioS3:///#bucketname#/#folder#/”

}

services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true,
flight.use_session_service: true
}
registration.publish-host: “0.0.0.0”

22

@SAIDULU Did you restart Dremio after the change?

23

HI @balaji.ramaswamy

Yes, I have done it but no luck.

I did a search in the community and disabled the support keys in the settings then it worked.

support_keys_disabled
support_keys_disabled719×552 36.9 KB

Thanks
Saidulu

24

@SAIDULU YEs that will work as now you are putting your reflections on local. This is not recommended/supported architecture anymore. Are you able to read a file from the S3 location you have given from AWS CLI?

25

@balaji.ramaswamy

I have not configured to put the reflections on locally. I have commented line in the config file but still, it didn’t work so had to disable the properties. Will there be any issue if we disable those three options.?

And yes I am able to read/access the S3 from CLI

paths: {

the local path for dremio to store data.

local: ${DREMIO_HOME}“/data”

local: “/storage/opt/dremio/data/”

the distributed path Dremio data including job results, downloads, uploads, etc

#dist: “pdfs://”${paths.local}“/pdfs”
dist: “dremioS3:///MyBucket/MyFolder/”

}

Thanks
Saidulu

26

@SAIDULU

Run any job in Dremio and splease send us the job profile. That should give us some indication, if possible send us the full dremio.conf