Following the instructions How to Create an ARP Connector | Dremio
The ARP jar is not being loaded by dremio on restart. There are no errors, and the log produced where it shows all of the jars loaded, it doesnt show the jar is being loaded.
sabot-module.conf says this: dremio.classpath.scanning.packages += “com.dremio.exec.store.jdbc”
Here is the log of the server along with screenshot of the jars that should have been bootstrapped.
2021-06-04 07:31:51,340 [main] INFO c.d.common.scanner.ClassPathScanner - Scanning packages [com.dremio.plugins.s3.store, com.dremio.plugins.adl.store, com.dremio.exec.store.hive, com.dremio.plugins.azure, com.dremio.exec.fn.hive, com.dremio.extras.plugins.elastic, com.dremio.exec.store.hive, com.dremio.exec.store.hive.exec, com.dremio.service.accelerator, com.dremio.service.reflection, com.dremio.service.voting, com.dremio.exec.store.jdbc, com.dremio.exec.store.jdbc, com.dremio.plugins.mongo, com.dremio.plugins.mongo, com.dremio.exec.store.dfs, com.dremio.exec.store, com.dremio.joust.geo, com.dremio.extra.exec.store.dfs, com.dremio.exec.planner.acceleration.substitution, com.dremio.sabot.task.slicing.SlicingTaskPool, com.dremio.exec.store.hive, com.dremio.service.cachemanager, com.dremio.exec.store.mock, com.dremio.common.logical, com.dremio.exec.store.dfs, com.dremio.exec.server.options, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac.support.BasicSupportService, com.dremio.services.configuration, com.dremio.services.configuration.ConfigurationStore, com.dremio.dac, com.dremio.dac.support.SupportService, com.dremio.dac.cmd, com.dremio.dac.cmd.upgrade, com.dremio.service.jobs, com.dremio.provision, com.dremio.plugins.elastic, com.dremio.datastore, com.dremio.exec.store.dfs, com.dremio.exec.ExecConstants, com.dremio.exec.catalog, com.dremio.exec.compile, com.dremio.exec.expr, com.dremio.exec.physical, com.dremio.exec.planner.physical, com.dremio.exec.server.options, com.dremio.exec.store, com.dremio.exec.store.dfs.implicit.ImplicitFilesystemColumnFinder, com.dremio.exec.rpc.user.security, com.dremio.sabot, com.dremio.sabot.op.aggregate.vectorized, com.dremio.sabot.rpc.user, com.dremio.service.execselector.ExecutorSelectionService, com.dremio.service.jobtelemetry.server.store, com.dremio.service.namespace, com.dremio.options, com.dremio.resource, com.dremio.resource.basic, com.dremio.telemetry.api, com.dremio.telemetry.utils, com.dremio.service.users, com.dremio.provision.yarn.service, org.apache.hadoop.hive] in locations [jar:file:/opt/dremio/jars/dremio-s3-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-adls-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-hive3-plugin-launcher-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-azure-storage-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-elasticsearch-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive2-plugin-launcher-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-hive-plugin-common-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-hive3-plugin-launcher-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-jdbc-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-jdbc-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-mongo-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-mongo-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-nas-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-pdfs-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-joust-java-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-sabot-kernel-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-hive2-plugin-launcher-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-ce-services-cachemanager-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-client-base-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-configuration-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-coordinator-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-dac-daemon-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-provision-common-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-elasticsearch-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-hdfs-plugin-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-execselector-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-namespace-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-options-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-resourcescheduler-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-api-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-telemetry-utils-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-services-users-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/dremio-yarn-4.5.0-202006180205030236-1a5d28da.jar!/, jar:file:/opt/dremio/jars/3rdparty/dremio-hive2-exec-shaded-4.5.0-202006180205030236-1a5d28da.jar!/] took 1240ms
@desi : As per the instructions below, did you try copying them when Dremio is not running?
First, make sure Dremio is not running, then, from the **target **directory, copy the .jar file and place it inside the /$DREMIO_HOME/jars directory. Additionally, move the JDBC driver to the /$DREMIO_HOME/jars/3rdparty directory. Start and log in in Dremio.
Yes these instructions were exactly followed.
Okay. I’m not sure what could be the issue. May be @balaji.ramaswamy can help here.
@Kyle_Porter any thoughts? ARP jar not being loaded
@desi Let me ask the engineer who wrote this connector
If there is no error, then it’s a scanning issue. If the sabot configuration is present, then have you ensured that the permissions on the files match the other JAR files for Dremio?
Have you tested the SQLite connector to see if it is loaded as a sanity check?
The permissions are the same. Sabot config is present in the jar as indicated. I did not compile deploy sqlite per se, but modified it as shown in examples. As a sanity test, I did drop in the salesforce jar on Dremio hub, and got an exception that sqldialect no class def found. Irrespective of the error, it means Dremio is scanning but not loading and not throwing an exception. I will try sqlite later today.
I am using sun jdk 1.8. I think it could be javac compiler options that may be missing? Like backwards compatibility?
This is the exception in deploying the Dremio hub jar mentioned above - Dremio Salesforce and java.lang.NoClassDefFoundError: com/dremio/exec/store/jdbc/AbstractDremioSqlDialect
Can you confirm the SQLite source works properly? For the exception you reference, are you saying that’s the error you’re getting?