Hi Team,
I have created a new Dremio Image with latest tag and included SnowFlake plugin and driver. After deploying the latest image in AKS, snowflake option is visible, but unable to connect to snowflake. Getting below error. Please help me fix this issue. I have tried all possible ways to place the file connector jar in. Still getting the error.
====Docker file=====
FROM dremio/dremio-oss:latest
USER root
To copy multiple files, change the below two lines to the following:
COPY /opt/dremio/jars/
RUN chown 1000:1000 /opt/dremio/jars/ /opt/dremio/jars/
#COPY ./dremio-snowflake-plugin.jar ./snowflake-jdbc-3.13.5.jar /opt/dremio/jars/
COPY ./dremio-snowflake-plugin.jar /opt/dremio/jars/
RUN chown 1000:1000 /opt/dremio/jars/dremio-snowflake-plugin.jar
For Dremio ARP connectors, you may need to copy file(s) to /opt/dremio/jars/3rdparty/ as well. Uncomment the following lines as appropriate:
(1) For a single file, uncomment the below two lines.
COPY /opt/dremio/jars/3rdparty/
#COPY ./snowflake-jdbc-3.13.8.jar /opt/dremio/jars/3rdparty/
#RUN chown 1000:1000 /opt/dremio/jars/3rdparty/snowflake-jdbc-3.13.8.jar
COPY ./snowflake-jdbc-3.13.8.jar /opt/dremio/jars/3rdparty/
RUN chown 1000:1000 /opt/dremio/jars/3rdparty/snowflake-jdbc-3.13.8.jar
(2) Or, to copy multiple files, uncomment the below two lines.
COPY /opt/dremio/jars/3rdparty/
RUN chown 1000:1000 /opt/dremio/jars/3rdparty/ /opt/dremio/jars/3rdparty/
USER dremio
===== Files inside the master pod====
dremio@dremio-master-0:/opt/dremio$ cd jars
dremio@dremio-master-0:/opt/dremio/jars$ ls -ltr | grep snow
-rwxr-xr-x 1 1000 1000 13957 Sep 23 06:53 dremio-snowflake-plugin.jar
dremio@dremio-master-0:/opt/dremio/jars$ cd 3rdparty/
dremio@dremio-master-0:/opt/dremio/jars/3rdparty$ ls -ltr | grep snow
-rwxr-xr-x 1 1000 1000 28380605 Sep 14 00:11 snowflake-jdbc-3.13.8.jar
dremio@dremio-master-0:/opt/dremio/jars/3rdparty$
But the driver is not loading - c.d.common.scanner.BuildTimeScan - Loaded prescanned packages
===== Error =============
2021-09-23 07:55:20,890 [start-snowflake] ERROR c.d.e.s.jdbc.JdbcSchemaFetcherImpl - Connection is not valid.
java.sql.SQLException: Cannot create JDBC driver of class ‘net.snowflake.client.jdbc.SnowflakeDriver’ for connect URL ‘jdbc:snowflake://pt22669.switzerland-north.azure.snowflakecomputing.com/warehouse_name=COMPUTE_WH’
at org.apache.commons.dbcp2.BasicDataSource.createConnectionFactory(BasicDataSource.java:2178)
at org.apache.commons.dbcp2.BasicDataSource.createDataSource(BasicDataSource.java:2050)
at org.apache.commons.dbcp2.BasicDataSource.getConnection(BasicDataSource.java:1533)
at com.dremio.exec.store.jdbc.CloseableDataSource$DatasourceWrapper.getConnection(CloseableDataSource.java:87)
at com.dremio.exec.store.jdbc.JdbcSchemaFetcherImpl.getState(JdbcSchemaFetcherImpl.java:604)
at com.dremio.exec.store.jdbc.JdbcStoragePlugin.getState(JdbcStoragePlugin.java:334)
at com.dremio.exec.catalog.ManagedStoragePlugin.setLocals(ManagedStoragePlugin.java:906)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:547)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:614)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.sql.SQLException: No suitable driver
at org.apache.commons.dbcp2.BasicDataSource.createConnectionFactory(BasicDataSource.java:2169)
… 12 common frames omitted
2021-09-23 07:55:20,890 [start-snowflake] WARN c.d.e.catalog.ManagedStoragePlugin - Error starting new source: snowflake
java.lang.Exception: Unavailable:
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:552)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:614)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
Regards,
Ramesh