Datasources showing in Red

1

How to check why a DataSource is shown in Red, when I’m able to explore and query the tables?

2

Hi @HLNA,

You could take a look at the <$DREMIO_HOME>/log/server.log for issues with datasources, and you can use the REST API also to get information on your current datasources:

  1. curl -XPOST http://<Dremio_Hostname>:9047/apiv2/login -H ‘Content-Type: application/json’ -d’{“userName”:“<Dremio_username>”,“password”:“<Dremio_password>”}’
  2. curl -XGET http://<Dremio_Hostname>:9047/api/v3/source -H ‘Authorization: _dremio<token_string>’

Steps:

  1. Insert Dremio Admin username/password respectively and you need to take the token string from the result JSON
  2. Run the second command once inserted the token string after “_dremio”

Let me know if you needed further information on this.

Thanks,
Danny

3

@dbrody
On applying the token to 2nd REST API call .I get the followign error… where can I get the API reference? I think I’m trying to get the list of sources right?

restapicall_errorbash_%E2%80%94_272%C3%9767

4

You error looks like the hostname/ip address, was not replaced correctly (could not resolve host).

You can find more information on REST API calls for Dremio on this link:
http://docs.dremio.com/rest-api/overview.html

You get the Dremio token from the 1st request and you need to paste it in the 2nd command I’ve sent you, but first the “could not resolve host” error needs to be addressed.

5

@dbrody
I am using localhost and the bold portion is what I’ve replaced. The previous REST API works fine… I am checking on the API page… but, not sure, why this is looking the as host?

curl -XGET http://localhost:9047/api/v3/source -H ‘Authorization: _dremio****’

6

Replace both the localhost and add the token after dremio using the result of the 1st step:

curl -XGET http://localhost:9047/api/v3/source?pretty -H ‘Authorization: _dremio<token>’

If the first request worked this should return with results as well (obviously you need to use the correct port as well, the default is 9047).

Let me know if this worked for you.

7

Thanks @dbrody … with “pretty” qualifier , it worked. One of the connection (in RED) shows as “bad”… but, turned green on preview… whereas some other connections (with 0 objects) remains red…

43%20PM

8

Perfect @HLNA,

Would you like to go ahead and drop this DS, or alternatively you could try to “force” refresh it’s metadata to see if it becomes available again?

Let me know what would you like to do at this point.

Thanks,
Danny

9

hi @dbrody

Is it possible to force refresh the metadata using rest Api?

10

Not currently. You update the metadata via SQL: https://docs.dremio.com/sql-reference/sql-commands/datasets.html

11

@kelly Thanks! It worked. I believe I can not delte PDS? or tables under physical data store right?
I see an option to disable metadata refresh (which worked fine), but, what’s the usecase for it? will the data be not updated? or the underlying structure will not be updated?

12

Dremio cannot update underlying data sources.

If you disable metadata refresh this will not affect your ability to query the data. Honestly, I’m not sure what the use case is, but maybe one of my colleagues does. :slight_smile:

13

@kelly haha… sure :smiley:

14

Hi @HLNA

Wondering if you had any other questions on this thread?

Thanks,
@balaji.ramaswamy

15

@balaji.ramaswamy yes. what’s the use case for deactivating metadata refresh? as still I’m able to query the table. As the physical table is never extracted and if I am still able to query, what’s the impact/effect of deactivating metadata?

16

Hi @HLNA

It is a trade off on your query performance, see different metadata refresh options below in our docs

[https://docs.dremio.com/advanced-administration/metadata-caching.html?h=queried%20data](http://Metadata Refresh Documentation)

Kindly let me know if you have any further questions

Thanks,
@balaji.ramaswamy