Hi,
We have more than 100 reflections and Jobs running throughout the day.
What is the best way to monitor the Reflections / Jobs so that we can be notified as and when error happens?
Approach 1:-
Send Dremio logs to Splunk and setup alerting within Splunk. What are the text to be used for alerting when there is a failure?
Approach 2:-
Write a query to read from the system table or so for the status. Is there any query for this?
You should be able to query sys.reflections, sys.materializations and sys.refreshes
For approach 1, you can use the API
@udayabaski There are also four reflection related JMX metric you can monitor. This is described here: Dremio
Sounds like you would be interested in reflections.failed This counter shows the reflections with refresh policies that are automatic but have permanently failed/given up after exceeding the number of re-tries.