Dremio execution plan shows SENDING

Hi Dremio Team,

my queries are taking longer to run, looking at profile while the query is still running shows lot of stages showing “SENDING” for all the values, Can you suggest what’s going on here?

Attached screenshot while query is still running, (after 10 mins still running)

Thanks

@smora

Can you please send us the query profile

@balaji.ramaswamy

attached profile.
Please note it shows SENDING for a quite a sometime during the execution

f9b0f591-9e80-48f1-a8a5-74bc1f7b64fc.zip (147.5 KB)

@smora

The entire time is spent on FILTER (02-xx12) and PROJECT (02-xx-10). The time is all spent in the SETUP phase. This is because we need to build the Gandiva cache, if you run the query again, it should run fast. We are planning to have a configurable Gandiva cache that can help here

Thanks
Bali

Thanks @balaji.ramaswamy
yes, immediate rerun executes fast. but if i change dates in where clause it again takes longer on first run, Is gandiva cache based on exact select&where clause match?

does switching to java_only instead of gandiva recommended?

@smora

Do not switch to Gandiva, it could be Gandiva cache . Please send us profile of successful run

@balaji.ramaswamy

query profile is attached in prvs post, it does complete successfully, as you pointed it spent lot of time on FILTER (02-xx12) and PROJECT (02-xx-10) on first run, during execution lot of threads shows SENDING as per my prvs screenshot, but eventually query completed. since you mentioned its gandiva cache taking time, though of switching to java_only instead of gandiva.

@smora

We should stay back on Gandiva as overall execution time will be faster. We are coming up with a configurable cache for Gandiva and will keep you posted