Could not connect to pg_dwh, check your JDBC connection information and credentials

Dremio Cloud
1

I’m trying out Dremio Cloud and connecting to Posgres Database. I get the error Could not connect to pg_dwh, check your JDBC connection information and credentials. I know the credentials are correct because on able to connect using Dremio Software and PGAdmin from my local computer. I just want to know if am doing something wrong or it’s a bug/limitation in Dremio Cloud.

Thank you very much in advance for your help

2

Hi there!

Welcome to the Dremio Cloud community!

Have you looked over our docs page here for Postgres to see if there was anything you may have missed?
https://docs.dremio.com/cloud/sources/external/postgres/

Cindy

3

Yea that is what I used as a guide and followed everything. But it’s still not working

4

Hello,

I’ve reached out to you over DM for more details to troubleshoot this issue. Looking forward to your response.

Thanks,
Payal

5

Hello Dan,
Thanks for sharing the requested information. I looked into the logs for your project ID and could see the below error messages:

message: "Exception while creating source."
stack_trace: "com.dremio.common.exceptions.UserException: Could not connect to pg_dwh, check your JDBC connection information and credentials
.
.
.Caused by: java.util.concurrent.ExecutionException: com.dremio.exec.store.jdbc.RemoteFetcherUnavailableException: Preview engine unavailable
	at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
	at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
	at com.dremio.exec.catalog.ManagedStoragePlugin.replacePlugin(ManagedStoragePlugin.java:1056)
	at com.dremio.exec.catalog.ManagedStoragePlugin.createOrUpdateSource(ManagedStoragePlugin.java:370)
	... 76 common frames omitted
Caused by: com.dremio.exec.store.jdbc.RemoteFetcherUnavailableException: Preview engine unavailable
	at com.dremio.exec.store.jdbc.RemoteFetcherNameProvider.getAddress(RemoteFetcherNameProvider.java:71)
	at com.dremio.exec.store.jdbc.RemoteFetcherNameProvider.lambda$new$0(RemoteFetcherNameProvider.java:60)
	at com.dremio.exec.store.jdbc.RemoteFetcherNameProvider.get(RemoteFetcherNameProvider.java:109)
	at com.dremio.exec.store.jdbc.RemoteFetcherNameProvider.get(RemoteFetcherNameProvider.java:36)
	at com.dremio.exec.store.jdbc.JdbcSchemaFetcherProxy.remoteName(JdbcSchemaFetcherProxy.java:89)
	at com.dremio.exec.store.jdbc.JdbcSchemaFetcherProxy.getState(JdbcSchemaFetcherProxy.java:100)
	at com.dremio.exec.store.jdbc.JdbcStoragePlugin.getState(JdbcStoragePlugin.java:333)
	at com.dremio.exec.catalog.ManagedStoragePlugin.setLocals(ManagedStoragePlugin.java:959)
	at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$2(ManagedStoragePlugin.java:573)
	at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$4(ManagedStoragePlugin.java:640)
	at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	... 1 common frames omitted

This error indicates that your Dremio Cloud preview engine is not enabled due to which you are facing issues while adding source.

In order to resolve this issue, please keep the Preview engine enabled while adding the postgress database source.

To enable to the preview engine, you can follow below steps:

  1. On the Project Settings page, click Engines in the sidebar menu. The list of engines in this project are displayed.
  2. Enable the engine by using the toggle in the Enabled column.
  3. Confirm that you want to enable the engine.

For more details on managing engines in Dremio Cloud, please refer to this document - Dremio

Hope this information helps. Let me know if the issue still persists.

Thanks,
Payal

6

Hi Gupta,
Thanks for the reply. However I’m still getting the same error. The preview engine was already enabled. I even added the following line to the Engine Routing to preview

query_type() IN ('JDBC', 'ODBC', 'UI Run', 'Flight')

Below is the screen shot of showing my enabled Preview enable.

image
image1440×280 19.1 KB

Thanks again for your help so far

Regards,

Dan

7

Hi,

Thanks for being so patient as we looked into this issue for you. Upon further investigation of your organization, it looks like your preview engine is failing to scale up, hence the source add failure. We are seeing an error regarding your subnet being invalid/not existing.

Here is how to navigate to the error in the Dremio Cloud UI:

  1. Navigate to Settings → Project Settings → Engines
  2. Select the “preview” engine
  3. Select the “events” tab
    Screen Shot 2022-08-24 at 1.06.02 PM
    Screen Shot 2022-08-24 at 1.06.02 PM920×466 27.2 KB
  4. Under Description on the far right, hovering over the “Scaling Failed…” message shows this error

Scaling Failed. com.dremio.daas.ems.common.CloudNonRetriableException: software.amazon.awssdk.services.ec2.model.Ec2Exception: The subnet ID ‘subnet-…’ does not exist (Service: Ec2, Status Code: 400, Request ID: …, Extended Request ID: null)

Can you double check your VPC environment in AWS and then try again? We have guidelines for subnets here: Dremio

Cindy

8

Thank you very much. I was suspecting something wrong causing my engine not to start

9

Hi Dan,

No problem!

Once your engines are up and running, you can set your engine min replicas to “0” and disable your engines when not in use to save costs on the AWS side. They will incur EC2 costs even if there are no queries running so be sure to disable them when not in use!

Feel free to reach out if you need any additional help.

Cindy