Error on vacuum FileNotFoundException

1

Hello friends, we have a scheduled job that merge some temporal icebrg tables to main iceber table each hour, the storage icrease too much because by each merge icberg creates a snapshoot
so we run a vacuum and optimce each 4h or 1 by day, but this strange error

FileNotFoundException: File /data/lake/posts2/metadata/8247453a-9288-4fdb-acd3-a467442e8104-m6.avro does not exist

Is not a concurrence problem because we run merges and vacuum in secuential order.

also we have too space used even after run optimice and vacuum (related to this another post Iceberg file size on dremio - #19 by dacopan)

Job failed on vacuum
139033d3-06e7-4809-a90c-69a8d95347e2.zip (39,0 KB)

2

@dacopan Thanks for uploading the profile, I have asked internally if this is something we have seen

1 Like
3

Hi @balaji.ramaswamy maybe some news?

4

It could happen, if both Vacuum and Optimize queries are run simultaneously. Optimize delete the manifest files that Vacuum refer to. Is that something that might be happening?

5

hello @balaji.ramaswamy we run this commands secuential no simultaneously

6

@dacopan Any chance you can send me the profiles of optimize that ran before the vacuum that failed and the profile of the failed vacuum

7

sure, please give me some hours

8

Sorry for delay, @balaji.ramaswamy this are the 2 jobs,
one is OK then second has error, in this case OPTIMIZE is not ran yet

b40ff31a-a192-4594-8aca-ca10952c4537_ok.zip (29,8 KB)
87d80468-7bf2-4197-af04-6fa8b1d74e58_fail.zip (29,6 KB)

I validate and not other ran concurrently

this is the unique failed job between the time where vacuum run ok and then failed

9eeea6bd-9e8d-41c1-a30b-0439c4a3163d.zip (7,9 KB)

9

Hi @balaji.ramaswamy maybe some news?

10

Hi @balaji.ramaswamy or @Benny_Chow I make a new test, I’ve atached the first failed vacuum job also the server.log in debug/trace mode

0a9a87c8-c851-4759-80b5-4341d5abeb02.zip (29,5 KB)
server.2024-08-27.103.zip (127,1 KB)

11

Sorry about the delay @dacopan Will provide an update shortly

1 Like
12

@dacopan Yes valid issue, fix is in 25.1, thanks for reaching out

1 Like
13

thank you, approximate date to this new version?

14

Hello @balaji.ramaswamy I readed 25.x Release Notes | Dremio Documentation changelog but not mention about this error

15

@dacopan Let me see if it made it to release notes

1 Like