Hi everyone,
I am trying to follow the tutorial on how to develop a custom ARP module (the one based on SQLite).
My initial intent was to customize it for a currently unsupported dbms from dremio, but after building it with maven and following the procedures in order to add the jars to the docker container even if the new source was visible, as soon as i was clicking on “Save” on the new source i was getting everytime “500 - Request failed”, no matter on the correctness of the parameters i am setting.
So i tried to leave untouched the arp tutorial code that is on SQLite and trying to add as source a sqlite database but again, the same error.
Furthermore, i found a plethora of ARP implementations from a CDATA repository (jerodj-cdata (jerodj-cdata) / Repositories · GitHub) and even with them i get the same error.
The only log i see on the docker log console while triggering a “Save” request is:
172.17.0.1 - - [20/Mar/2024:20:45:07 +0000] “PUT /apiv2/source/a/?nocache=1710967507103 HTTP/1.1” 500 0 “http://localhost:9047/” “Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/123.0.0.0 Safari/537.36”
Just to mention, with the other “native” modules (MySQL, PostgreSQL, etc.) i have no problems on adding new sources, both running on local machine or remotely.
Dremio build informations:
Any idea?
Thanks in advance!
I believe there was some refactoring with the CredentialService code in v24.3.x so , using your reference to sqllite as an example , can you try changing
import com.dremio.security.CredentialsService;
to
com.dremio.services.credentials.CredentialsService
in SqliteConf.java .
Then rebuild
1 Like
i tried but now i am facing problems while rebuilding:
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Dremio SQLite Community Connector 21.1.1-202204292111390812-57b1832f
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] — maven-clean-plugin:2.4.1:clean (default-clean) @ dremio-sqlite-plugin —
[INFO]
[INFO] — maven-resources-plugin:2.5:resources (default-resources) @ dremio-sqlite-plugin —
[debug] execute contextualize
[INFO] Using ‘UTF-8’ encoding to copy filtered resources.
[INFO] Copying 3 resources
[INFO]
[INFO] — maven-compiler-plugin:3.0:compile (default-compile) @ dremio-sqlite-plugin —
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /root/dremio-sqllite-connector/target/classes
[INFO] -------------------------------------------------------------
[ERROR] COMPILATION ERROR :
[INFO] -------------------------------------------------------------
[ERROR] /root/dremio-sqllite-connector/src/main/java/com/dremio/exec/store/jdbc/conf/SqliteConf.java:[41,8] com.dremio.exec.store.jdbc.conf.SqliteConf is not abstract and does not override abstract method buildPluginConfig(com.dremio.exec.store.jdbc.JdbcPluginConfig.Builder,com.dremio.security.CredentialsService,com.dremio.options.OptionManager) in com.dremio.exec.store.jdbc.conf.JdbcConf
[ERROR] /root/dremio-sqllite-connector/src/main/java/com/dremio/exec/store/jdbc/conf/SqliteConf.java:[84,3] method does not override or implement a method from a supertype
[INFO] 2 errors
[INFO] -------------------------------------------------------------
[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 14.708s
[INFO] Finished at: Thu Mar 21 14:05:49 CET 2024
[INFO] Final Memory: 85M/205M
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project dremio-sqlite-plugin: Compilation failure: Compilation failure:
[ERROR] /root/dremio-sqllite-connector/src/main/java/com/dremio/exec/store/jdbc/conf/SqliteConf.java:[41,8] com.dremio.exec.store.jdbc.conf.SqliteConf is not abstract and does not override abstract method buildPluginConfig(com.dremio.exec.store.jdbc.JdbcPluginConfig.Builder,com.dremio.security.CredentialsService,com.dremio.options.OptionManager) in com.dremio.exec.store.jdbc.conf.JdbcConf
[ERROR] /root/dremio-sqllite-connector/src/main/java/com/dremio/exec/store/jdbc/conf/SqliteConf.java:[84,3] method does not override or implement a method from a supertype
[ERROR] → [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
You get this error because the CredentialService change is specific to v24.3.x and it looks like the version of dremio you are trying to build against is v21. To overcome this amend the version referenced in your pom.xml to match the community version you are using.
In this case you are using v24.3.2 CE so we can amend the following entries in the pom.xml from
<groupId>com.dremio.plugin</groupId>
<version>21.1.1-202204292111390812-57b1832f</version>
<artifactId>dremio-sqlite-plugin</artifactId>
<name>Dremio SQLite Community Connector</name>
<properties>
<version.dremio>21.1.1-202204292111390812-57b1832f</version.dremio>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
to
<groupId>com.dremio.plugin</groupId>
<version>24.3.2-202401241821100032-d2d8a497</version>
<artifactId>dremio-sqlite-plugin</artifactId>
<name>Dremio SQLite Community Connector</name>
<properties>
<version.dremio>24.3.2-202401241821100032-d2d8a497</version.dremio>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
</properties>
and then rebuild.
it works! thank you so much.
