Dremio S3 ignores nonProxyHosts settings

1

Hi there,

TLDR: Dremio S3 ignores -Dhttp.nonProxyHosts and -Dhttps.nonProxyHosts start parameters. How to fix this?

I have the following problem:
I am deploying Dremio on an on premise Kubernetes cluster that is located behind a forward proxy/company firewall.
I set up the proxy settings via

  extraStartParams: >-
    -Dhttp.proxyHost=proxy.hostname
    -Dhttp.proxyPort=8080
    -Dhttps.proxyHost=proxy.hostname
    -Dhttps.proxyPort=8080
    -Dhttp.nonProxyHosts=on.premise.storage.hostname|*.cluster.local|localhost
    -Dhttps.nonProxyHosts=on.premise.storage.hostname|*.cluster.local|localhost

and with distributed storage on aws S3 which works fine.

However, we have to work with on premise object storage (Minio) located at https://on.premise.storage.hostname:9020 on the internal network, which the forward proxy does not resolve.
Usually, I would expect, that setting nonProxyHosts for the Java runtime would take care of resolving requests to that hostname internally, however, the S3 plugin does not respect these settings and uses the given proxy instead, hence cant find the storage endpoint.

I’d be happy for any advice on how to resolve this.

Thanks!

2

Hey there–

This is new to me so apologies if my advice doesn’t solve this. I found another person here that is trying to work with similar settings. Have you attempted to set the proxy settings with the DREMIO_JAVA_EXTRA_OPTS setting in the dremio-env file in your conf folder?

How to connect Dremio with GCS storage using proxy - #7 by Pawel has an example.

Dan