# Job cancellation requested failure

**URL:** <https://community.dremio.com/t/job-cancellation-requested-failure/8077>\
**Category:** Uncategorized\
**Created:** [October 1, 2021, 10:57am UTC](https://community.dremio.com/t/job-cancellation-requested-failure/8077 "2021-10-01T10:57:43Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![lucbaro](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lucbaro/32/3107_2.png) [@lucbaro](https://community.dremio.com/u/lucbaro)\
**Post date:** [October 1, 2021, 10:57am UTC](https://community.dremio.com/t/job-cancellation-requested-failure/8077/1 "2021-10-01T10:57:43Z")

</div>

Hi there,

I’ve some troubles cancelling some long running jobs: when I try to cancel some of them from the Jobs UI, I get the **Job cancellation requested** notification, but after nothing happens, the job is still running.  
I do have to restart the master pod in order to get rid of them.

Our Dremio is running on a K8s cluster, the release info are:  
Build 17.0.0-202107060524010627-31b5222b  
Edition Community Edition  
Build Time 06/07/2021 07:41:44  
Change Hash 31b5222bbf667ebdfb277c2c1b727eb9943083d3  
Change Time 05/07/2021 22:51:06

Is there another way to cancel them ?

Also I can see in the documentation that there is a [query runtime limit](https://docs.dremio.com/advanced-administration/workload-management/#rules) that could be set but I do not know how to set it. Is it possible to do that ?

Thanks,

Luc

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [October 5, 2021, 6:29am UTC](https://community.dremio.com/t/job-cancellation-requested-failure/8077/2 "2021-10-05T06:29:29Z")

</div>

@lucbaro Query run time limit can be set under WLM queue settings

If the query is not cancelling , setting runtime may not help. We need to find out out which fragment is still running

Expand each phase and look at the status of each thread, if you find any RUNNING, log on to the executor that it was runing on and see i that executor logs and GC logs have anything to tell us, like Full GC events, ZK was suspended etc

---

<div class="post-metadata">

**Author:** ![lucbaro](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/lucbaro/32/3107_2.png) [@lucbaro](https://community.dremio.com/u/lucbaro)\
**Post date:** [October 8, 2021, 9:28am UTC](https://community.dremio.com/t/job-cancellation-requested-failure/8077/3 "2021-10-08T09:28:45Z")

</div>

Thanks @balaji.ramaswamy,

Are WLM settings available for Enterprise edition only ?  
I can only see `Queue timeout` setting in Dremio Admin \> Settings \> Engines \> Queue Control  
But no query runtime limit…

---

<div class="post-metadata">

**Author:** ![balaji.ramaswamy](https://sea2.discourse-cdn.com/flex020/user_avatar/community.dremio.com/balaji.ramaswamy/32/843_2.png) [@balaji.ramaswamy](https://community.dremio.com/u/balaji.ramaswamy)\
**Post date:** [October 13, 2021, 4:56am UTC](https://community.dremio.com/t/job-cancellation-requested-failure/8077/4 "2021-10-13T04:56:36Z")

</div>

@lucbaro My bad, yes it is