Trying to add dynamodb as data source in dremio

Dremio University
1

Hi,
We are from Baker Hughes evaluating the option to use dynamodb with dremio.
we are referring to this link:

we downloaded the dremio code from repo:

and then floowed the steps did a mvn clean install and then compile dremio running fine.
then we put dremio-dynamodb-plugin-4.5.0-202006180205030236-1a5d28da.jar in jars folder of distrbution/server/jars
and Simba_DynamoDB_JDBC folder jars into jars/3rdparty along with Simba lic file.

restart the dremio app and login

go to external data sources add…no dynamodb option there.

2

Hi Rukmani!

I’ve updated the releases on my github. Try the latest release v12. Releases · dremio-brock/dremio-dynamodb-connector · GitHub. Let me know if this works.

1 Like
3

Hi b-rock,
I tried with v12 of dremio-dynamodb-connector.
still dynamo db not showing up in external datasources options.
I’m attaching the error screenshot which 'm encountring.dremio-withdynamodb-erroe

4

Can you post the error message. I can’t read the message from the picture.

5

java.lang.Exception: Unavailable: File /Users/omi/Projects/rukku/dremio-oss/dac/daemon/target/sample-data-8010747564968651473 does not exist
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:529)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
23:48:46.918 [start-LocalFS1] WARN c.d.e.catalog.ManagedStoragePlugin - Error starting new source: LocalFS1
java.lang.Exception: Unavailable: File /Users/omi/Projects/rukku/dremio-oss/dac/daemon/target/sample-data-8010747564968651473 does not exist
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:529)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
23:48:46.960 [main] ERROR c.dremio.exec.catalog.PluginsManager - Failure while starting plugin LocalFS2 after 70ms.
java.util.concurrent.ExecutionException: java.lang.Exception: Unavailable: File /Users/omi/Projects/rukku/dremio-oss/dac/daemon/target/sample-data-8010747564968651473 does not exist
at java.util.concurrent.CompletableFuture.reportGet(CompletableFuture.java:357)
at java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1908)
at com.dremio.exec.catalog.PluginsManager.start(PluginsManager.java:228)
at com.dremio.exec.catalog.CatalogServiceImpl.start(CatalogServiceImpl.java:198)
at com.dremio.service.SingletonRegistry$AbstractServiceReference.start(SingletonRegistry.java:137)
at com.dremio.service.ServiceRegistry.start(ServiceRegistry.java:88)
at com.dremio.service.SingletonRegistry.start(SingletonRegistry.java:33)
at com.dremio.dac.daemon.DACDaemon.startServices(DACDaemon.java:196)
at com.dremio.dac.daemon.DACDaemon.init(DACDaemon.java:202)
at com.dremio.dac.daemon.DremioDaemon.main(DremioDaemon.java:146)
Caused by: java.lang.Exception: Unavailable: File /Users/omi/Projects/rukku/dremio-oss/dac/daemon/target/sample-data-8010747564968651473 does not exist
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$newStartSupplier$1(ManagedStoragePlugin.java:529)
at com.dremio.exec.catalog.ManagedStoragePlugin.lambda$nameSupplier$3(ManagedStoragePlugin.java:591)
at java.util.concurrent.CompletableFuture$AsyncSupply.run(CompletableFuture.java:1604)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

6

Hi b-rock,
I hv added the error message as comment.