Hi Dremio Team,
my queries are taking longer to run, looking at profile while the query is still running shows lot of stages showing “SENDING” for all the values, Can you suggest what’s going on here?
Attached screenshot while query is still running, (after 10 mins still running)
Thanks
Can you please send us the query profile
attached profile.
Please note it shows SENDING for a quite a sometime during the execution
f9b0f591-9e80-48f1-a8a5-74bc1f7b64fc.zip (147.5 KB)
The entire time is spent on FILTER (02-xx12) and PROJECT (02-xx-10). The time is all spent in the SETUP phase. This is because we need to build the Gandiva cache, if you run the query again, it should run fast. We are planning to have a configurable Gandiva cache that can help here
Thanks
Bali
Thanks @balaji.ramaswamy
yes, immediate rerun executes fast. but if i change dates in where clause it again takes longer on first run, Is gandiva cache based on exact select&where clause match?
does switching to java_only instead of gandiva recommended?
Do not switch to Gandiva, it could be Gandiva cache . Please send us profile of successful run
query profile is attached in prvs post, it does complete successfully, as you pointed it spent lot of time on FILTER (02-xx12) and PROJECT (02-xx-10) on first run, during execution lot of threads shows SENDING as per my prvs screenshot, but eventually query completed. since you mentioned its gandiva cache taking time, though of switching to java_only instead of gandiva.
We should stay back on Gandiva as overall execution time will be faster. We are coming up with a configurable cache for Gandiva and will keep you posted
