How to access a public S3 bucket through corporate firewall

1

Tried testing the samples.dremio.com public bucket…

It shows up in Sources, but it comes up empty. The error message is below…

How do I add a HTTP_PROXY setting into the Dremio configuration to get through the corporate firewall?

Caused by: org.apache.http.conn.ConnectTimeoutException: Connect to samples.dremio.com.s3.amazonaws.com:80 [samples.dremio.com.s3.amazonaws.com/52.218.193.138] failed: connect timed out
at org.apache.http.impl.conn.DefaultHttpClientConnectionOperator.connect(DefaultHttpClientConnectionOperator.java:150) ~[httpclient-4.5.2.jar:4.5.2]
at org.apache.http.impl.conn.PoolingHttpClientConnectionManager.connect(PoolingHttpClientConnectionManager.java:353) ~[httpclient-4.5.2.jar:4.5.2]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_121]

2

Hello @david.lee,

Dremio can be configured to run with additional JVM flags. You can use this property to set the http.proxyHost and http.proxyPort options.

From your installation, go to /conf and edit the dremio-env file.

At the bottom of this file is a DREMIO_JAVA_EXTRA_OPTS variable which is commented out with the # character at the front. Uncomment this, and add -Dhttp.proxyHost=[your proxy host] and -Dhttp.proxyPort=[your proxy port]

3

Ok this worked after some tweeking…

DREMIO_JAVA_EXTRA_OPTS="-Dhttp.proxyHost=[proxy host] -Dhttp.proxyPort=[port]"

4

Perfect! Thanks for the update.

5

I did exactly what @david.lee suggested, but I am seeing the following error:

Unable to execute HTTP request: Connect to samples.dremio.com.s3.amazonaws.com:80 [samples.dremio.com.s3.amazonaws.com/52.218.244.242] failed: Connection refused: connect

@jduong could this be caused by something other than the proxy?