Unable to create reflections with Dremio Community Edition

We are unable to create reflections using:
Build
21.1.1-202204292111390812-57b1832f
Edition
Community Edition
the /opt/dremio/data/pdfs/accelerator folder is empty and we’re seeing the following error in the jobs listing:
UnknownFormatConversionException: Conversion = ‘Unknown format (pdfs) conversion for path /opt/dremio/data/pdfs/accelerator/c55d9ef6-a693-4681-9c54-92f965a925ab/4493ca78-07d0-43e9-b460-b0999e8883c4_0/metadata/ded7085b-3236-4b07-b73d-53c204662d1b.avro Error Message : No File System scheme matches’

I have the same problem

disable dremio.iceberg.enabled
or use s3 or hdfs… as paths.dist

I don’t see an option for disabling dremio.iceberg

I tried setting paths.dist and that didn’t work. I don’t see an option to set dremio.iceberg.enabled: false. I’m still unable to create even simple raw reflections.

UnknownFormatConversionException: Conversion = ‘Unknown format (pdfs) conversion for path /opt/dremio/data/pdfs/accelerator/0ce84a34-10cd-4625-8809-12d7e213e101/c31427de-6e48-4216-9536-3997634e8665_0/metadata/1f9e55dd-2281-4942-a3de-84b684819eab.avro Error Message : No File System scheme matches’

https://docs.dremio.com/software/advanced-administration/support-settings/

Yes, see more here : https://docs.dremio.com/software/release-notes/210-release/#2-pdfs-is-not-supported-for-distributed-storage-in-versions-2100-and-above

Did they give any reason ?

I don’t see these options in Community Edition

I’m trying to save reflections to the local drive and not a distributed file system. Is this a problem?

yes, this is not possible since version 21.

you can also fix the configuration like this :

alter system set dremio.iceberg.enabled = true/false;

alter system set dremio.execution.support_unlimited_splits = true/false;

alter system set dremio.iceberg.ctas.enabled = true/false;
1 Like

I figured it out and it works! Thanks!

@summersmd

From 21.x PDFS is no longer supported so dist:// has to be pointed to a S3 or ADLS or HDFS or GCS