Dremio - Reflection / Job Monitoring

Hi,

We have more than 100 reflections and Jobs running throughout the day.
What is the best way to monitor the Reflections / Jobs so that we can be notified as and when error happens?

Approach 1:-

Send Dremio logs to Splunk and setup alerting within Splunk. What are the text to be used for alerting when there is a failure?

Approach 2:-

Write a query to read from the system table or so for the status. Is there any query for this?

@udayabaski

You should be able to query sys.reflections, sys.materializations and sys.refreshes

For approach 1, you can use the API

https://docs.dremio.com/software/rest-api/jobs/get-job/

@udayabaski There are also four reflection related JMX metric you can monitor. This is described here: Dremio

Sounds like you would be interested in reflections.failed This counter shows the reflections with refresh policies that are automatic but have permanently failed/given up after exceeding the number of re-tries.