I have configured KeyCloak as an external token provider by following these documents:
- External Token Providers | Dremio Documentation
- https://support.dremio.com/hc/en-us/articles/29109477930651-Configuring-KeyCloak-as-an-SSO
All configurations seem to be fine. I got proper jwt token like the following:
{“access_token”:“eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJENEdlTDRoTVZlTWVJTFg5a2FxaTJtaVIyTGVoTUhGQlNwNFJvaTRFbjA0In0.eyJleHAiOjE3NTI3NjU5MDAsImlhdCI6MTc1Mjc2NTg0MCwianRpIjoiNTJkYzQ0ODQtMjBkOC00ZjVhLThkNTYtY2E5YjZmMWJjMTQyIiwiaXNzIjoiaHR0cHM6Ly9rZXljbG9hay5hcnJvd2ZsaWdodHNxbC5jb20vcmVhbG1zL21hc3RlciIsImF1ZCI6ImRyZW1pby1rZXljbG9hay1pZHAtdGVzdCIsInN1YiI6ImUwY2NhMTg1LTEyZDEtNDNiYS1iNmQ5LTZmMDA4NGExZjBmYyIsInR5cCI6IkJlYXJlciIsImF6cCI6ImRyZW1pby1rZXljbG9hay1pZHAtdGVzdCIsInNlc3Npb25fc3RhdGUiOiIzZDIwNDY2OC0wZDA2LTQyOTQtOGZiMS1mYzdhMWUzNjZkN2EiLCJzY29wZSI6Im9wZW5pZCBwcm9maWxlIiwic2lkIjoiM2QyMDQ2NjgtMGQwNi00Mjk0LThmYjEtZmM3YTFlMzY2ZDdhIiwibmFtZSI6IlRlc3QgT25lIiwicHJlZmVycmVkX3VzZXJuYW1lIjoicWcuZ2MudGVzdEBnbWFpbC5jb20iLCJnaXZlbl9uYW1lIjoiVGVzdCIsImZhbWlseV9uYW1lIjoiT25lIn0.r4qHI5gZ9UbQrZmkCzXaa2XO2gJx0eCKbPFEwpaA6zaph7IMATVoWxbzvOQm6jTYvYHoWdjxVIbP7blwJ_JDUqk7Vm2g0wk7jVX8GZTC_nVg2gMvlf4GeIG5tCU24EQRKoBtuFIXvJNpFC5iu_ZpxlBS7vtR_NXHidZichVUgIuda1Di8YzpeVIlyD-YxRQF4T-CUe4fIvXoS8rS1K7AFdVSvw_Yx7xtmus4EdfMmsV3VRsNjQe19Jl_0BpkElGfBU15TiMN2Nu8JxK9tNWpKfmdwYeHeO3dY7WnV6IB5JWjAesDLoXeRMuTEHpI0QodXSumeETHPlVJ-bkQP8QdPw”,“expires_in”:60,“refresh_expires_in”:1800,“refresh_token”:“eyJhbGciOiJIUzUxMiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJkMDY1ZWQzNy0xN2Y2LTQ0NGUtOGU1My1iYzRkNWZkMmNjN2UifQ.eyJleHAiOjE3NTI3Njc2NDAsImlhdCI6MTc1Mjc2NTg0MCwianRpIjoiZDVmMjJiZjAtNzNmZi00NjM5LWJjMDItOGJmNGVlNTgyMzBmIiwiaXNzIjoiaHR0cHM6Ly9rZXljbG9hay5hcnJvd2ZsaWdodHNxbC5jb20vcmVhbG1zL21hc3RlciIsImF1ZCI6Imh0dHBzOi8va2V5Y2xvYWsuYXJyb3dmbGlnaHRzcWwuY29tL3JlYWxtcy9tYXN0ZXIiLCJzdWIiOiJlMGNjYTE4NS0xMmQxLTQzYmEtYjZkOS02ZjAwODRhMWYwZmMiLCJ0eXAiOiJSZWZyZXNoIiwiYXpwIjoiZHJlbWlvLWtleWNsb2FrLWlkcC10ZXN0Iiwic2Vzc2lvbl9zdGF0ZSI6IjNkMjA0NjY4LTBkMDYtNDI5NC04ZmIxLWZjN2ExZTM2NmQ3YSIsInNjb3BlIjoib3BlbmlkIHByb2ZpbGUiLCJzaWQiOiIzZDIwNDY2OC0wZDA2LTQyOTQtOGZiMS1mYzdhMWUzNjZkN2EifQ.bZjgEYjMKQTDznwZI7GPZzoooAcbZYP4dpWcwCVCtaklPaDmTPIsQW7QaquCXHF7h5YVg9prVL1W4VVCMsVMhA”,“token_type”:“Bearer”,“id_token”:“eyJhbGciOiJSUzI1NiIsInR5cCIgOiAiSldUIiwia2lkIiA6ICJENEdlTDRoTVZlTWVJTFg5a2FxaTJtaVIyTGVoTUhGQlNwNFJvaTRFbjA0In0.eyJleHAiOjE3NTI3NjU5MDAsImlhdCI6MTc1Mjc2NTg0MCwiYXV0aF90aW1lIjowLCJqdGkiOiI3ZjNjNzE2OS0zNmU1LTRmMTYtYmVjNy1kOTkxZTQ0MDExMjYiLCJpc3MiOiJodHRwczovL2tleWNsb2FrLmFycm93ZmxpZ2h0c3FsLmNvbS9yZWFsbXMvbWFzdGVyIiwiYXVkIjoiZHJlbWlvLWtleWNsb2FrLWlkcC10ZXN0Iiwic3ViIjoiZTBjY2ExODUtMTJkMS00M2JhLWI2ZDktNmYwMDg0YTFmMGZjIiwidHlwIjoiSUQiLCJhenAiOiJkcmVtaW8ta2V5Y2xvYWstaWRwLXRlc3QiLCJzZXNzaW9uX3N0YXRlIjoiM2QyMDQ2NjgtMGQwNi00Mjk0LThmYjEtZmM3YTFlMzY2ZDdhIiwiYXRfaGFzaCI6IkVoekpQRVp1cUEtLXYwZHJiUUtNMUEiLCJzaWQiOiIzZDIwNDY2OC0wZDA2LTQyOTQtOGZiMS1mYzdhMWUzNjZkN2EiLCJuYW1lIjoiVGVzdCBPbmUiLCJwcmVmZXJyZWRfdXNlcm5hbWUiOiJxZy5nYy50ZXN0QGdtYWlsLmNvbSIsImdpdmVuX25hbWUiOiJUZXN0IiwiZmFtaWx5X25hbWUiOiJPbmUifQ.t5CCipVtxu8vuq-7XBcpu-JVW8EFCxCeqMJ8EW-ErRMSLCwBPAUv4AFrIy_gOdLVPJqh8lOUhEeMOryAknrhy_9ul_OYK7Fuhw_PLvDgWI1rklds6FApeyB3_Cr6hY7Vrah4_q-5RY8JKaf_ywv4ZKo-pKeNtYrGP_xfF17Ua75Q9lKkLC8dHgc3Vt1YJU5-fQiG0bi1Gdz3AjGX9Xn49smGEmbhuFHQcnBK1IMI6Brx-wd1LUarWYgh7b_csDpgc9foTpm-eRm0jpSK33fJlNUUxQ5XtTEyauEfmlfMgRflaIILZWxt5mFyra-3qi0FmZuFpoBIra8ihGo22sZuAQ”,“not-before-policy”:0,“session_state”:“3d204668-0d06-4294-8fb1-fc7a1e366d7a”,“scope”:“openid profile”}
It is decoded as the following:
Payload:
{
“sub”: “e0cca185-12d1-43ba-b6d9-6f0084a1f0fc”,
“iss”: “https://keycloak.arrowflightsql.com/realms/master”,
“typ”: “Bearer”,
“preferred_username”: “qg.gc.test@gmail.com”,
“given_name”: “Test”,
“sid”: “3d204668-0d06-4294-8fb1-fc7a1e366d7a”,
“aud”: “dremio-keycloak-idp-test”,
“azp”: “dremio-keycloak-idp-test”,
“scope”: “openid profile”,
“name”: “Test One”,
“exp”: 1752765900,
“session_state”: “3d204668-0d06-4294-8fb1-fc7a1e366d7a”,
“iat”: 1752765840,
“family_name”: “One”,
“jti”: “52dc4484-20d8-4f5a-8d56-ca9b6f1bc142”
}
It has correct “aud”, “preferred_username”, “iss”, etc.
I also configured External Token Provider:
Audience: dremio-keycloak-idp-test
User Claim Mapping: preferred_username
Issuer URL: https://keycloak.arrowflightsql.com/realms/master
JWKS URL: https://keycloak.arrowflightsql.com/realms/master/protocol/openid-connect/certs
But when I use jwt token in the following sample test program, I keep getting error:
org.apache.arrow.flight.FlightRuntimeException: UNKNOWN:
at org.apache.arrow.flight.CallStatus.toRuntimeException(CallStatus.java:121)
at org.apache.arrow.flight.grpc.StatusUtils.fromGrpcRuntimeException(StatusUtils.java:161)
at org.apache.arrow.flight.FlightClient.getInfo(FlightClient.java:302)
at org.apache.arrow.flight.sql.FlightSqlClient.execute(FlightSqlClient.java:136)
at org.apache.arrow.flight.sql.FlightSqlClient.execute(FlightSqlClient.java:116)
location = Location.forGrpcTls(“data.dremio.cloud”, 443);
private static void runFlightCall(Location location,
BufferAllocator allocator,
String token) throws Exception {
//String sqlQuery = “SELECT * FROM "Samples"."samples.dremio.com"."NYC-taxi-trips" LIMIT 5”;
String sqlQuery = “SELECT * from demo1.folder_y.names;”;
try (FlightClient client = FlightClient.builder(allocator, location)
.build()) {
CredentialCallOption credentialCallOption = new CredentialCallOption((callHeaders) -> {
callHeaders.insert("authorization", "Bearer " + token);
});
FlightSqlClient sqlClient = new FlightSqlClient(client);
FlightInfo flightInfo = sqlClient.execute(sqlQuery, credentialCallOption);
List<FlightEndpoint> endpoints = flightInfo.getEndpoints();
System.out.println("Query result from default project");
for (FlightEndpoint endpoint : endpoints) {
try (FlightStream stream = sqlClient.getStream(endpoint.getTicket(), credentialCallOption)) {
if (stream != null) {
while (stream.next()) {
VectorSchemaRoot root = stream.getRoot();
System.out.println(root.contentToTSVString());
}
}
} catch (Exception e) {
e.printStackTrace();
throw e;
}
}
}
}
Using arrow flight jdbc is not working either with the following connection string:
“jdbc:arrow-flight-sql://sql.dremio.cloud:443?”
+ “useEncryption=true”
+ “&disableCertificateVerification=true”
+ “&token=” + token;
Could anyone provide guidence on what I miss?
Thanks!