Inconsistent token exchange behavior between enterprise and cloud

1

I tested both Dremio Cloud and Dremio Enterprise (v26.x), and observed inconsistent behavior in token exchange:

  1. Dremio Cloud allows the same token to be exchanged multiple times within its validity window.
  2. Dremio Enterprise, however, treats token exchange as a one-time operation. Reusing the same token for exchange results in the following error:
    {"error":"invalid_request","error_description":"Invalid token"}

This behavior in Dremio Enterprise makes implementing SSO significantly more difficult and leads to a poor user experience, which may hinder adoption.

Could you please clarify the reason for this inconsistency between the two offerings?

2

Hi Dan,

Thank you for trying out the OAuth Token Exchange and for your question. To better understand the issue, could you please provide more information about your use case?

Additionally, it would be helpful if you could share the following details for both scenarios:

  1. grant_type and subject_token_type

If you are using an external token, please check its validity and ensure that the subject user exists within Dremio. Also, if you have access to the enterprise environment, could you check the logs for any relevant error logs when attempting the token exchange?

Thank you!

3

Hello,

Thank you for the prompt response. I am testing a standard SSO use case with the following setup:

  1. One source database (non-Dremio) and one target database (Dremio).
  2. The source database is integrated with an identity provider (Keycloak) for authentication, and Keycloak is registered in Dremio as an External Token Provider.
  3. Users authenticate against the source database (OCID) using the IDP.
  4. Users run cross-database queries between the source database and Dremio, using token exchange to authenticate with Dremio.

I tested the token exchange flow against both Dremio Cloud and Dremio Enterprise using the following simple Java function:

java

CopyEdit

private static String exchangeTokenWithDremio(String incomingToken) throws Exception {
    Map<String, String> params = Map.of(
        "grant_type", "urn:ietf:params:oauth:grant-type:token-exchange",
        "subject_token_type", "urn:ietf:params:oauth:token-type:jwt",
        "subject_token", incomingToken,
        "scope", "dremio.all"
    );

    String form = params.entrySet().stream()
        .map(e -> URLEncoder.encode(e.getKey(), StandardCharsets.UTF_8) + "=" + URLEncoder.encode(e.getValue(), StandardCharsets.UTF_8))
        .reduce((a, b) -> a + "&" + b)
        .orElse("");

    HttpRequest request = HttpRequest.newBuilder()
        .uri(URI.create(dremioTokenExchangeUrl))
        .header("Content-Type", "application/x-www-form-urlencoded")
        .POST(HttpRequest.BodyPublishers.ofString(form))
        .build();

    HttpClient client = HttpClient.newHttpClient();
    HttpResponse<String> response = client.send(request, HttpResponse.BodyHandlers.ofString());

    System.out.println("Token exchange response:");
    System.out.println(response.body());

    String accessToken = response.body()
        .split("\"access_token\":\"")[1]
        .split("\"")[0];

    System.out.println(accessToken);
    return accessToken;
}

The token exchange works and authentication is successful in both environments. However, with Dremio Enterprise (v26.x), I’m encountering an issue where the same incoming token can only be exchanged once. Reusing the same token results in the following error:

json

CopyEdit

{"error":"invalid_request","error_description":"Invalid token"}

This is not aligned with the standard behavior expected from OAuth 2.0 token exchange, nor is it consistent with Dremio Cloud, which allows multiple exchanges within the token’s validity window.

Dremio Cloud behaves as expected, while Dremio Enterprise’s handling of token exchange significantly complicates SSO integration and negatively impacts the user experience.

Could you please clarify the reason for this inconsistency and advise on how to proceed?

Thank you.

4

I have verified the token’s validity and confirmed that the subject user exists in Dremio.
I also reviewed the logs for any relevant errors during the token exchange attempt, but found nothing in the Kubernetes logs.

Do you expect Dremio Cloud and Dremio Enterprise to behave the same in terms of token exchange by design?
Maybe there be an issue with my implementation? I could used expired token to do exchange?

5

Hi Dan,
Thanks for sharing the information on the flows.

We tested the external token exchange behavior on Enterprise and found that we could obtain a new Dremio access token by exchanging the same external token, as long as it is valid and not expired. This behavior is consistent for both Dremio Cloud and Enterprise Edition (EE).

We recommend checking whether the external token is already expired or near expiry before attempting an exchange. If you are working with external tokens that have a short expiration time (60 seconds) or if the token is already expired before you invoke the exchange request, you should first obtain a new token from your underlying Identity Provider (IDP) using a refresh token, if available. Alternatively, consider updating the IDP configuration to increase the expiration time for tokens.
Note: Dremio will only issue the new Access token based on expiry time left in the external token, so if your IDP tokens are short-lived, increasing the expiry to a reasonable value will help.

6

Hi @ajulka9 , Thanks for following up. I think the problem was that my IDP token is short-lived (60 sec). Also AI fooled me by stating that Dremio Enterprise treats token exchange as a one-time operation :sweat_smile:

Sorry for the confusion.

Now I am good. Thank you

7

Glad the issue is resolved for you.

Thanks,
aj