Unable to create reflections with Dremio Community Edition

We are unable to create reflections using:
Build
21.1.1-202204292111390812-57b1832f
Edition
Community Edition
the /opt/dremio/data/pdfs/accelerator folder is empty and we’re seeing the following error in the jobs listing:
UnknownFormatConversionException: Conversion = ‘Unknown format (pdfs) conversion for path /opt/dremio/data/pdfs/accelerator/c55d9ef6-a693-4681-9c54-92f965a925ab/4493ca78-07d0-43e9-b460-b0999e8883c4_0/metadata/ded7085b-3236-4b07-b73d-53c204662d1b.avro Error Message : No File System scheme matches’

I have the same problem

disable dremio.iceberg.enabled
or use s3 or hdfs… as paths.dist

1 Like

I don’t see an option for disabling dremio.iceberg

I tried setting paths.dist and that didn’t work. I don’t see an option to set dremio.iceberg.enabled: false. I’m still unable to create even simple raw reflections.

UnknownFormatConversionException: Conversion = ‘Unknown format (pdfs) conversion for path /opt/dremio/data/pdfs/accelerator/0ce84a34-10cd-4625-8809-12d7e213e101/c31427de-6e48-4216-9536-3997634e8665_0/metadata/1f9e55dd-2281-4942-a3de-84b684819eab.avro Error Message : No File System scheme matches’

https://docs.dremio.com/software/advanced-administration/support-settings/

Yes, see more here : https://docs.dremio.com/software/release-notes/210-release/#2-pdfs-is-not-supported-for-distributed-storage-in-versions-2100-and-above

Did they give any reason ?

I don’t see these options in Community Edition

I’m trying to save reflections to the local drive and not a distributed file system. Is this a problem?

yes, this is not possible since version 21.

you can also fix the configuration like this :

alter system set dremio.iceberg.enabled = true/false;

alter system set dremio.execution.support_unlimited_splits = true/false;

alter system set dremio.iceberg.ctas.enabled = true/false;
1 Like

I figured it out and it works! Thanks!

@summersmd

From 21.x PDFS is no longer supported so dist:// has to be pointed to a S3 or ADLS or HDFS or GCS

Disabling iceberg worked for me.

the best approach is use NAS, if you disable iceberg, basically are truncate list of current and future benefits of dremio

paths.dist=file:///opt/dremio/data/pdfs

test ok