It 's failed to start dremio with version 4.9, and the distributed storage is minio.
Some errors:
2020-10-29 08:14:35,109 [start-__home] WARN c.d.e.catalog.ManagedStoragePlugin - Error starting new source: __home
com.google.common.util.concurrent.UncheckedExecutionException: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to 127.0.0.1:8123 [/127.0.0.1] failed: Connection refused (Connection refused)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2051)
at com.google.common.cache.LocalCache.get(LocalCache.java:3953)
at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3976)
at com.google.common.cache.LocalCache$LocalLoadingCache.get(LocalCache.java:4960)
at com.dremio.exec.store.dfs.FileSystemPlugin.newFileSystem(FileSystemPlugin.java:317)
at com.dremio.exec.store.dfs.FileSystemPlugin.createFS(FileSystemPlugin.java:305)
at com.dremio.exec.store.dfs.FileSystemPlugin.createFS(FileSystemPlugin.java:301)
at com.dremio.exec.store.dfs.FileSystemPlugin.createFS(FileSystemPlugin.java:297)
at com.dremio.exec.store.dfs.FileSystemPlugin.start(FileSystemPlugin.java:614)
at com.dremio.dac.homefiles.HomeFileSystemStoragePlugin.start(HomeFileSystemStoragePlugin.java:100)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:523)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: com.amazonaws.SdkClientException: Unable to execute HTTP request: Connect to 127.0.0.1:8123 [/127.0.0.1] failed: Connection refused (Connection refused)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleRetryableException(AmazonHttpClient.java:1201)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1147)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)
at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)
at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)
at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5054)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:5000)
at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4994)
at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:993)
at com.amazonaws.services.s3.AmazonS3Client.listBuckets(AmazonS3Client.java:999)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.dremio.plugins.s3.store.S3FileSystem.lambda$registerReference$0(S3FileSystem.java:199)
at com.sun.proxy.$Proxy67.listBuckets(Unknown Source)
at com.dremio.plugins.s3.store.S3FileSystem.getContainerCreators(S3FileSystem.java:293)
at com.dremio.plugins.util.ContainerFileSystem.refreshFileSystems(ContainerFileSystem.java:91)
at com.dremio.plugins.util.ContainerFileSystem.initialize(ContainerFileSystem.java:159)
at com.dremio.exec.store.dfs.FileSystemPlugin$1.lambda$load$0(FileSystemPlugin.java:203)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
at com.dremio.exec.store.dfs.FileSystemPlugin$1.load(FileSystemPlugin.java:208)
at com.dremio.exec.store.dfs.FileSystemPlugin$1.load(FileSystemPlugin.java:185)
at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3529)
at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2278)
at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2155)
at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2045)
... 15 common frames omitted
dremio.conf file
paths: {
# the local path for dremio to store data.
local: ${DREMIO_HOME}"/data"
# the distributed path Dremio data including job results, downloads, uploads, etc
dist: "dremioS3:///dremio/sub"
#dist: "pdfs://"${paths.local}"/pdfs"
}
services: {
coordinator.enabled: true,
coordinator.master.enabled: true,
executor.enabled: true
}
core-site.xml file
<property>
<name>fs.s3a.endpoint</name>
<description>Endpoint can either be an IP or a hostname, where Minio server is running . However the endpoint value cannot contain the http(s) prefix. E.g. 175.1.2.3:9000 is a valid endpoint. </description>
<value>172.17.0.1:9000</value> // this is endpoint of local minio
</property>
I am confused that what is the port 8123.