Reload datasource without restarting Dremio

1

Hi,

i have a MySQL Datasource that I found every morning in red in Dremio.
I suppose this is due to a backup done during the night.
It seems Dremio is not able to recover that status by itself.
Is there anyway to force reload of the source ?
I also think about restarting dremio during the night, what would be the best way to restart Dremio in an Amazon Linux crontab ?

Thanks

2

Hello,
Did you try to use Refresh Every __ hours settings while configuring MySQL Source?

57%20AM
Screen Shot 2018-03-13 at 9.26.57 AM.png984×539 42 KB

3

The refresh option is to define how often reflections should be refreshed. It does not fix the lost connection to the data source.

4

Hi @dfleckinger,

@ssjsk has pointed you in the right direction - under your MySQL’s source settings screen you can set refresh frequency for both (metadata and accelerations).
The metadata refresh settings are collapsed by default - just click “show advanced options” in order to set them according to your needs.

5

Hi Sharon,

these are my settings :

image
image.png828×978 51.5 KB

I’m unsure what i should do to avoid the datasource to be in red on the morning ?
Thanks

6

Hi @dfleckinger,

This seems to be a known issue - we’ll be checking with the upcoming release where source management has been revamped so will keep you posted.

Just to clarify - Basically you shouldn’t restart Dremio. When you refresh your browser in the morning the source should turn back green. Is that not the case?

7

Red icon is disguise I’d say, you can see datasets/tables intact, I didn’t get any issue accessing data from my datasource.

8

I agree i shouldn’t restart Dremio.
I am not able to access the Mysql tables of this datasource as long as it is red.
The only way to get it back to greeen is to restart dremio.
Any idea when the upcoming release will happen ?
Thanks

9

I am not able to access the Mysql tables of this datasource as long as it is red.

I tested it on my end and I can’t reproduce that behavior.

Which version are you using?
Can you please upload your server.log file (under /log directory)?

10

Hi @Sharon,
i’m using Dremio Community Edition with version 1.4.4-201801230630490666-6d69d32
and Mysql Server 5.7.17.

So this morning the source is displayed in red, and when I try to access a table, i get a Table not found error
,with this in the log file

ERROR o.a.calcite.runtime.CalciteException - org.apache.calcite.sql.validate.SqlValidatorException: Table ‘CMS.prd_cms.affiliate_accounts’ not found

image
image1842×968 61.3 KB

Please find in the attachement the part of the server.log file corresponding to this error :
dremio_server.log_extract.log.zip (1.8 KB)

I can provide the full log by email if needed.