# Federated Monitoring Capabilities?

**URL:** <https://community.dremio.com/t/federated-monitoring-capabilities/1958>\
**Category:** Uncategorized\
**Created:** [October 2, 2018, 9:53pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958 "2018-10-02T21:53:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![LewG](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lewg/32/608_2.png) [@LewG](https://community.dremio.com/u/LewG)\
**Post date:** [October 2, 2018, 9:53pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/1 "2018-10-02T21:53:20Z")

</div>

As one of the core capabilities of Dremio is to provide end users a self-service capability by way of creating Virtual Data Sets, a concern arises that these users who may not be SQL savvy may inadvertently create VDSs that are built on high cost SQL queries - even unintentional Cartesian products could result. What is Dremio’s solution, if any, to this possibility in the realm of monitoring and preempting such occurrences? Is Dremio dependent on the various underlying data sources timeout mechanisms or external monitoring tools to preempt such occurances?

---

<div class="post-metadata">

**Author:** ![can](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/can/32/40_2.png) [@can](https://community.dremio.com/u/can)\
**Post date:** [October 2, 2018, 10:10pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/2 "2018-10-02T22:10:36Z")

</div>

Hey Lew, thanks for the question! We’re planning to address such scenarios as a part of our workload management work in our Enterprise Edition (more updates coming soon). As we release more capabilities here, admins will be able to manage this within Dremio. Over time, the idea is to provide a rule-based system where users can target queries based on operations in the query (full scan, cartesian join, no predicates, etc.) as well as run-time metrics (run-time, rows scanned, rows sorted, memory spilled/user, etc.). They can use this information to automatically reject or place queries into queues, as well as cancel active queries.

---

<div class="post-metadata">

**Author:** ![hieuiph](https://avatars.discourse-cdn.com/v4/letter/h/50afbb/32.png) [@hieuiph](https://community.dremio.com/u/hieuiph)\
**Post date:** [July 13, 2023, 2:11pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/3 "2023-07-13T14:11:43Z")

</div>

Hello,

I have the same question, now we are on version 20 Enterprise Edidition. How can we reject the “bad query” based on operations in the query or run-time metrics as you said @can ?

There are not documentation about this.

thanks

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [July 15, 2023, 8:07pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/4 "2023-07-15T20:07:28Z")

</div>

@hieuiph Via WLM you can set max job and queue memory limits, You can also set Max runtime limit. This way you can avoid log running full table scan queries that may use up all the memory available

---

<div class="post-metadata">

**Author:** ![hieuiph](https://avatars.discourse-cdn.com/v4/letter/h/50afbb/32.png) [@hieuiph](https://community.dremio.com/u/hieuiph)\
**Post date:** [July 17, 2023, 3:49pm UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/5 "2023-07-17T15:49:57Z")

</div>

Hello @balaji.ramaswamy , thanks for your answer. I tried all you suggested but it doesn’t work if a user run the “bad query” like “Select distinct \* …”. I set up limit query max =1, queue memory limits = 23 (max memory on every executor = 37GB, we have 4 executor).

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [July 22, 2023, 7:01am UTC](https://community.dremio.com/t/federated-monitoring-capabilities/1958/6 "2023-07-22T07:01:03Z")

</div>

@hieuiph Can you please send me the job profile
