Writing a generic JDBC connector that supports ANSI SQLs

[patricker]

@olumo I don’t have a definitive answer to your question, but I have spent a fair amount of time diving into the closed-source innards of the ARP JDBC adapter source code.

Go grab a program like JD-GUI and start digging around the custom ARP adapter that Dremio implemented for MS SQL (which is how they are able to set all the custom character ENCODING stuff), which they defined through Java code instead of through a YAML file.

Of course, you can only use the code you see as a guide, license wise it’s off limits from copy/paste or modification.

The code does not pass many unused classes around. I wanted to run some pre/post query commands, but was locked out because of how it’s coded; and since Dremio will not open source the ARP JDBC driver, there isn’t any path forward for many use cases.

I really like the concept behind the ARP adapter, I just wish it was a lot more flexible (like if it was open source…) Some of my previous posts on this topic:

Confusing ARP JDBC Error Message

I’ve been seeing a JDBC error recently, “Query cancelled by user”. I’ve seen it from both MS SQL and Snowflake. In my query I’m joining several datasets together from different platforms, so Dremio is running parts of the query in parallel, distributing the data across the provisioned cluster, and doing joins. When I look at the log, it looks like what actually happens is that a previous Hash Distribution step fails, and so Dremio properly tells other steps happening in parallel to cancel, bu…

MSSQL Connector Incorrectly Plans/Casts Boolean Operations

If a MS SQL Table contains a bit type column, and you try to query it using column_name = 1 then the query planner translates this to, WHERE [column_name]. This is invalid syntax for MS SQL, so it throws the error: An expression of non-boolean type specified in a context where a condition is expected If I try use Dremi’s internal functions to cast it to integer, CAST(column_name AS int) = 1 I get the error: Cast function cannot convert value of type BOOLEAN to type INTEGER I found that …

Enhancement Request - ARP JDBC

We’ve had a lot of success using the ARP JDBC plugin framework. We’d like to be able to make pre/post SQL calls whenever a connection is retrieved/returned to the connection pool; if we had to override JdbcConf.newPlugin and provide an extended version of JdbcStoragePlugin that would be fine. We’d also like to have access to the querying user at this scope, so we could set session parameters on the database. These querying user details are outside the scope of dataset security, that we could ac…