Dremio JDBC driver error with Spark

When I try and access data using spark and 4.1.0 dremio jdbc driver, I get this error:

2019-Dec-14 00:01:09 AM [Client-1] INFO cdjd.com.dremio.sabot.rpc.user.UserClient - [USER]: Channel closed null ↔ null (user client)
2019-Dec-14 00:01:09 AM [ForkJoinPool-3-worker-3] INFO com.kalibri.etl.utilities.SnowflakeDatasource - Reading data from KLDW_TEST.reference.kl_hotel_co
2019-Dec-14 00:01:12 AM [ForkJoinPool-3-worker-3] INFO cdjd.com.dremio.common.config.SabotConfig - Configuration and plugin file(s) identified in 7ms.
Base Configuration:
- jar:file:/Users/mfarmer/git/kalibri-etl/etl2.0/lib/dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar!/sabot-default.conf
Intermediate Configuration and Plugin files, in order of precedence:
- jar:file:/Users/mfarmer/git/kalibri-etl/etl2.0/lib/dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar!/sabot-module.conf
2019-Dec-14 00:01:12 AM [ForkJoinPool-3-worker-3] WARN cdjd.com.dremio.telemetry.api.metrics.Metrics - Removing old metric since name matched newly registered metric. Metric name: USER-send-durations-ms
2019-Dec-14 00:01:12 AM [Client-1] INFO cdjd.com.dremio.sabot.rpc.user.UserClient - [USER]: Channel closed null ↔ null (user client)
2019-Dec-14 00:01:12 AM [Client-1] INFO cdjd.com.dremio.sabot.rpc.user.QueryResultHandler - User Error Occurred [ErrorId: 5b2fae24-c801-437d-ae4b-f6a1b13abd80]
cdjd.com.dremio.common.exceptions.UserException: Connection null ↔ null (user client) closed unexpectedly. SabotNode down?
at cdjd.com.dremio.common.exceptions.UserException$Builder.build(UserException.java:776) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.com.dremio.sabot.rpc.user.QueryResultHandler$ChannelClosedHandler$1.operationComplete(QueryResultHandler.java:373) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:500) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:493) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:472) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:413) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:538) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:527) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]
at cdjd.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:98) [dremio-jdbc-driver-4.1.0-201912030136020081-49feeb75.jar:4.1.0-201912030136020081-49feeb75]

On the coordinator node, in queries,json I see this

{“queryId”:“220b97a6-a6da-e37e-831e-28c432929700”,“queryText”:“SELECT * FROM kldw.reference.channel WHERE 1=0”,“start”:1576298584671,“finish”:1576298585163,“outcome”:“COMPLETED”,“use
rname”:“mfarmer”}
{“queryId”:“220b97a6-335d-0547-ba9f-36d071e01600”,“queryText”:“SELECT * FROM kldw.reference.kl_ta_exp_xref WHERE 1=0”,“start”:1576298585231,“finish”:1576298585449,“outcome”:“CANCELED
“,“outcomeReason”:“Query cancelled by user ‘mfarmer’”,“username”:“mfarmer”}
{“queryId”:“220b97a5-9f20-81eb-bcf8-137fde102600”,“queryText”:“SELECT * FROM kldw.reference.channel WHERE 1=0”,“start”:1576298585232,“finish”:1576298585845,“outcome”:“CANCELED”,“outc
omeReason”:“Query cancelled by user ‘mfarmer’”,“username”:“mfarmer”}
{“queryId”:“220b9745-4284-1c71-21e8-5b1140a8c800”,“queryText”:“SELECT * FROM kldw.reference.kl_ref_rules WHERE 1=0”,“start”:1576298682791,“finish”:1576298682814,“outcome”:“COMPLETED”
,“username”:“mfarmer”}
{“queryId”:“220b9744-a8c5-d7fd-e140-dda0a4e0b800”,“queryText”:“SELECT * FROM kldw.reference.kl_ta WHERE 1=0”,“start”:1576298682790,“finish”:1576298682815,“outcome”:“COMPLETED”,“usern
ame”:“mfarmer”}
{“queryId”:“220b9744-fda7-c5e8-0497-1bebb2839500”,“queryText”:“SELECT * FROM kldw.reference.pfp_add_commission_exp WHERE 1=0”,“start”:1576298682815,“finish”:1576298682873,“outcome”:”
COMPLETED”,“username”:“mfarmer”}
{“queryId”:“220b9744-93d6-3b43-7a7f-cd825cf0b400”,“queryText”:“SELECT * FROM kldw.reference.property WHERE 1=0”,“start”:1576298682819,“finish”:1576298682879,“outcome”:“COMPLETED”,“us
ername”:“mfarmer”}
{“queryId”:“220b9744-c8ae-9d5d-64b8-89aa3e4c1400”,“queryText”:“SELECT * FROM kldw.reference.kl_usali WHERE 1=0”,“start”:1576298682815,“finish”:1576298682889,“outcome”:“COMPLETED”,"us

  • a mixture of cancelled and complete queries. All queries from spark FAIL

It appears to be perhaps a CLIENT issue, rather than a dremio coordinator issue ? Any thoughts on tracking down the issue ?

I can confirm that the JDBC driver works as expected from Aqua Data Studio - so this is a spark/jdbc issue