Hello guys, can you give me some documentation or explanation of how to interpret the profile in an advanced way?
similar for example in PostgreSQL we can detect that when buffers reads from disk can slow query but when was cached in OS was speed,
have Dremio some similar documentation to understand in detail each part of raw profile?
8523abb1-db11-4f1b-9f6d-c4fd1b8c8c50.zip (40,9 KB)
I’ve attached a example query profile that I want interprete why is “slow”
@dacopan Not currently, this is something that will be done soon.
In your case TABLE_FUNCTION 02-xx-21 is what is taking up almost 11s of the query run time. Since you have a single executor and it is only a 4 core machine there is delay, reading 24 Million records across 450 files
thank you @balaji.ramaswamy I’ll wait for this guide, it’ll be a powerful tool to optimize queries.
In some cases as me, this query was generate by PowerBi we only select a specific date “2023-11-30” and table are partitioned by date, so the records mut be apprx 700k is ot correct that Dremio first scan 24M records before apply filter by date.
You can learn a lot by looking at the apache drill query profile documentation. It is not a 1:1 mapping, but just to get an understanding of the overall concepts it is helpful
There is a whitepaper available, explaining how profiles can be interpreted: