"Stuck" jobs in jobs list

1

Some time ago we noticed that there are a few “stuck” jobs in our jobs list. (see image below)
At the time of noticing this issue we were running version 20.0.0-202201050826310141-8cc7162b now we are on version 20.1.0-202202061055110045-36733c65 and we still see this issue.

The failed jobs open up just as expected, after these “stuck” jobs have appeared we have tried running: dremio-admin clean --delete-orphans --max-job-days 7 --reindex-data --delete-orphan-profiles --compact without success.

Does anyone know how we can get rid of these jobs from our job list?

image
image378×699 49.9 KB

2

We are back a few versions from you, but when we found stuck jobs in our logs like the ones you describe, we had to stop and restart the Dremio service.

3

Unfortunately that doesn’t help :frowning:

4

@GoldenGoose When you say stuck jobs, are you referring to job in enqueued state?

5

Look at image above, I have excluded “Failed” from the dropdown, but they still show in the list.

6

@GoldenGoose

Try this

  • Backup Dremio
  • Shutdown Dremio
  • On the coordinator cd bin
  • ./dremio-admin clean -i
  • Restart Dremio

Do you still see the issue?

7

@balaji.ramaswamy I’ve already tried dremio-admin clean --delete-orphans --max-job-days 7 --reindex-data --delete-orphan-profiles --compact (on master) without success :frowning_face:

8

Was there a conclusion to this as to how to cleanup old jobs?

9

@Muneer There have been a few bug fixes on this in the last 3 years. What is your exact issue? Job status filtering showing incorrect job types?

10

I have set the Jobs purge window to be 30 days. But it still has jobs from 7 months back. Is it the dremio-admin clean that actually clears the jobs down and therefore the size of the backup file?

I am on v25 dremio community edition.