Exception with Non-Canonical Paths

Dremio Cloud
1

Hey everyone!
I am using a docker compose file to create a Data Lakehouse model. In this file I am using Dremio as one of the services of the docker-compose file, with the following implementation.

services:
  geodremio:
    image: dremio/dremio-oss:25.1
    container_name: geodremio
    networks:
      iceberg_net:
    ports:
      - 7000:9047
      - 7001:31010
      - 7002:32010
    volumes:
      - ./dremio/data:/opt/dremio/data

While it works, for the most part, I get the following exception

java.io.IOException: Cannot create non-canonical path /opt/dremio/data/pdfs/node_history/metrics/20241016T081409Z.csv
2024-10-16 11:14:09 at com.dremio.exec.store.dfs.PseudoDistributedFileSystem.create(PseudoDistributedFileSystem.java:389)
2024-10-16 11:14:09 at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1233)
2024-10-16 11:14:09 at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1210)
2024-10-16 11:14:09 at org.apache.hadoop.fs.FileSystem.create(FileSystem.java:1091)
2024-10-16 11:14:09 at com.dremio.exec.hadoop.HadoopFileSystem.create(HadoopFileSystem.java:258)
2024-10-16 11:14:09 at com.dremio.io.file.FilterFileSystem.create(FilterFileSystem.java:70)
2024-10-16 11:14:09 at com.dremio.exec.store.dfs.LoggedFileSystem.create(LoggedFileSystem.java:104)
2024-10-16 11:14:09 at com.dremio.services.nodemetrics.persistence.NodeMetricsStorage.write(NodeMetricsStorage.java:57)
2024-10-16 11:14:09 at com.dremio.services.nodemetrics.persistence.NodeMetricsWriter.writeMetrics(NodeMetricsWriter.java:143)
2024-10-16 11:14:09 at com.dremio.services.nodemetrics.persistence.NodeMetricsWriter.run(NodeMetricsWriter.java:112)
2024-10-16 11:14:09 at com.dremio.service.scheduler.SimpleLocalSchedulerService$CancellableTask.run(SimpleLocalSchedulerService.java:88)
2024-10-16 11:14:09 at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
2024-10-16 11:14:09 at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
2024-10-16 11:14:09 at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:304)
2024-10-16 11:14:09 at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
2024-10-16 11:14:09 at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
2024-10-16 11:14:09 at java.base/java.lang.Thread.run(Thread.java:829)

2

@Makdak26 I assume your Dremio still works fine, we have a ticket to remove this spam of messages. Dremio no longer supports pdfs and this will not appear if you have a distributed storage defined

3

Hi @balaji.ramaswamy, thank you for your message! Indeed Dremio works without issues. Good to know that this will be removed in the future. Currently I have connected a distributed storage to Dremio (MinIO), along with other sources, but directly through Dremio’s platform. I will also take a look at defining it through the configuration files.