Java 11 JDBC support

Hi,
I added your JDBC driver into my project (with version 4.2.2-202004211133290458-b550b6fa) and it doesn’t work with Java 11. It says

    java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.<init>(long, int) not available
	at cdjd.io.netty.util.internal.PlatformDependent.directBuffer(PlatformDependent.java:474) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.NettyArrowBuf.getDirectBuffer(NettyArrowBuf.java:257) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffer(NettyArrowBuf.java:236) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.NettyArrowBuf.nioBuffers(NettyArrowBuf.java:220) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.NettyArrowBuf.setBytes(NettyArrowBuf.java:378) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.MutableWrappedByteBuf.setBytes(MutableWrappedByteBuf.java:373) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.ExpandableByteBuf.setBytes(ExpandableByteBuf.java:27) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.buffer.AbstractByteBuf.writeBytes(AbstractByteBuf.java:1133) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.socket.nio.NioSocketChannel.doReadBytes(NioSocketChannel.java:350) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.nio.AbstractNioByteChannel$NioByteUnsafe.read(AbstractNioByteChannel.java:148) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:714) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:650) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:576) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:493) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:989) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at cdjd.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) ~[dremio-jdbc-driver-4.2.1-202004111451200819-0c3ecaea.jar:4.2.1-202004111451200819-0c3ecaea]
	at java.base/java.lang.Thread.run(Thread.java:834) ~[na:na]

2020-04-23 10:33:53.360  INFO 18393 --- [       Client-1] c.com.dremio.sabot.rpc.user.UserClient   : [USER]: Channel closed null <--> null (user client)
2020-04-23 10:33:54.371 ERROR 18393 --- [nio-8080-exec-1] com.zaxxer.hikari.pool.HikariPool        : HikariPool-1 - Exception during pool initialization.

java.sql.SQLException: Failure in connecting to Dremio: cdjd.com.dremio.exec.rpc.RpcException: HANDSHAKE_COMMUNICATION : [USER]: Channel closed null <--> null (user client)

When I tried to run it with Java 8, it works great. Do you plan to support the Java 11 or is there a workaround on how to make it work with 11?

Thanks a lot!

2 Likes

@jdufek

We neither support Java 11 nor do we have a workaround. We are looking to add support soon

Thanks
@balaji.ramaswamy

1 Like

Hi Team/@balaji.ramaswamy
I am also getting Same Error while connecting Openjdk 1.9 with Dremio using jdbc Jar

java.lang.UnsupportedOperationException: sun.misc.Unsafe or java.nio.DirectByteBuffer.(long, int) not available

Do Dremio Support Java Version greater than 1.8 ?

Regards,
Akash

@akash, @jdufek,

Try adding the following parameter to your Java 11 runtime when you run the Dremio JDBC driver:

-Dcdjd.io.netty.tryReflectionSetAccessible=true

Hey all,

Is there a public issue tracker or some way to vote on getting Java 11 support for the Java JDBC driver? Not being able to run on newer versions than (seemingly) Java 8 is a pretty big issue for the driver. I know at my company we’re actively migrating towards Java 11, and this is a considerable blocker.

Interesting, while this didn’t work for me, the option:
-Dio.netty.tryReflectionSetAccessible=true
worked to get the JDBC running on Java 11!

@cygnus, to be clear, the parameter I showed you only gets the JDBC Driver client running, not Dremio server.

I’m having the same problem here is my logs:
[Client-1] ERROR cdjd.com.dremio.exec.rpc.BasicClient - Failed to establish connection
java.util.concurrent.ExecutionException: cdjd.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: dremiolab.allstate.com/10.194.21.26:31010
at cdjd.io.netty.util.concurrent.AbstractFuture.get(AbstractFuture.java:54)
at cdjd.com.dremio.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionEstablishmentListener.operationComplete(BasicClient.java:285)
at cdjd.com.dremio.exec.rpc.BasicClient$ConnectionMultiListener$ConnectionEstablishmentListener.operationComplete(BasicClient.java:273)
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:511)
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:504)
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:483)
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:424)
at cdjd.io.netty.util.concurrent.DefaultPromise.tryFailure(DefaultPromise.java:121)
at cdjd.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.fulfillConnectPromise(AbstractNioChannel.java:327)
at cdjd.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:343)
at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:633)
at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeysOptimized(NioEventLoop.java:580)
at cdjd.io.netty.channel.nio.NioEventLoop.processSelectedKeys(NioEventLoop.java:497)
at cdjd.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:459)
at cdjd.io.netty.util.concurrent.SingleThreadEventExecutor$5.run(SingleThreadEventExecutor.java:886)
at java.lang.Thread.run(Thread.java:748)
Caused by: cdjd.io.netty.channel.AbstractChannel$AnnotatedConnectException: Connection refused: no further information: dremiolab.allstate.com/10.194.21.26:31010
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at cdjd.io.netty.channel.socket.nio.NioSocketChannel.doFinishConnect(NioSocketChannel.java:325)
at cdjd.io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:340)
… 6 more
Caused by: java.net.ConnectException: Connection refused: no further information
… 10 more

@mark_smart

Is your server on JDK 11? or is it the client?

Thanks
Bali

Hi Bali,

Basically I’m trying to hit what is in effect a Dremio Lab that my company’s Dremio support team has provided - we don’t have an actual instance of Dremio in place as yet. I can hit the Lab via the APIs in Postman without a problem but I need to confirm that I can hit the Lab via the jdbc as I am the SME of a spring boot microservices app that is migrating from Hive to Dremio. So I have a simple java (JDK 1.8) test application to check connection to the lab via the jdbc client driver. I know there are issues with the new versions of the driver and new recent versions of the JDK. I have tried setting the runtime option using -Dcdjd.io.netty.tryReflectionSetAccessible=true but I still get the same problem with a refusal to connect from the Lab side.

@mark_smart

I am a little confused, are you saying you are hitting this error with your client on JDK11 or JDK8 and server on JDK8?

From the client what happens if you do

telnet 10.194.21.26 31010
I assume the above IP is the coordinator’s IP?

Sorry for the confusion.
I am testing connection to my company’s Dremio Lab via jdbc with a simple java (jkd 1.8) test application. I cannot confirm what version of java is used for the server.

I am on Windows 10 and telnet is disabled by our support team.

However, in powershell I can connect to the Lab:

ComputerName : dremiolab.allstate.com
RemoteAddress : 10.194.21.26
InterfaceAlias : _Common_enterprise_split_na - vpn.allstate.com
SourceAddress : 10.90.220.49
PingSucceeded : True
PingReplyDetails (RTT) : 119 ms
TraceRoute : 172.23.9.72
172.23.9.65
10.64.4.90
10.10.1.106
10.194.1.65
10.194.21.26

This is my java test program:

// Requires JDK 1.8 in your classpath
// Download the Dremio jar from Index of jdbc-driver/3.0.6-201812082352540436-1f684f9/
// and save to the same location as your program
// For logging download slf4j-api and slf4j-simple (select same versions) jars from Artifactory
// Use this command to configure netty:
// set JAVA_TOOL_OPTIONS: -Dcdjd.io.netty.tryReflectionSetAccessible=true
// Compile and run the program, from the location you saved it, in a command prompt with the command:
// javac DremioJdbcTest.java && java -cp .;* DremioJdbcTest
import java.sql.*;
import java.util.Properties;

public class DremioJdbcTest {

public static void main(String[] args) {
    final String DB_URL = "jdbc:dremio:direct=dremiolab.allstate.com:31010;";
    final String USER = "";
    final String PWD = "";
    Properties props = new Properties();
    props.setProperty("userName",USER);
    props.setProperty("password",PWD);
	
    Connection conn = null;
    Statement stmt = null;
    try {

        System.out.println("Connecting to database...");
        conn = DriverManager.getConnection(DB_URL, props);

        System.out.println("Creating statement...");
        stmt = conn.createStatement();
        String sql;
        sql = "SELECT user";
        ResultSet rs = stmt.executeQuery(sql);

        while (rs.next()) {
            System.out.print(rs.getString("user"));

        }
        rs.close();
        stmt.close();
        conn.close();
    } catch (SQLException se) {
        se.printStackTrace();
    }
}

}

Creds have been removed.