Hi,
I want to analyze the last few days the jobs that Dremio in my company has been running to check what Views are generating the most “Row Scanned” and the frequency with which users execute them.
I tried to use sys.projetct.history.jobs / sys.history.jobs, but here we are using Dremio Software.
So I downloaded the “queries.2023-07-11.0.json” inside log/coordinator/archive, but this “queries.json” has only two columns, that is, Query ID and Outcome.
I thought that with the Query ID (from this “queries.json”) I could find for each one of them the “Row Scanned” information, but the API v3 return does not have this “Row Scanned”.
Can someone help me?
I just want something very similar to this, but on Excel or a CSV to analyze the statistics.
@Lincoln_Borges Are you saying your queries.json has only 2 columns in the entire file?
Are you running into this issue?
https://docs.dremio.com/software/release-notes/200-release/#breaking-changes
Can you please attach the queries.json if data in it is not sensitive?
@balaji.ramaswamy Im attaching the “queries.json” file that I have in the “/var/dremio_efs/log/coordinator” folder.
queries.zip (338.8 KB)
As you can see, just have these two pieces of information:
Is it possible to have more information so I can do a better analysis of the user and what they are consuming on my Dremio Software.
@Lincoln_Borges queries.json should have way more number of columns,
- What version of Dremio is this?
- What kind of deployment is this? Like K8’s/Yarn etc
- Send us your logback.xml from the
conf folder
@balaji.ramaswamy,
Send us your logback.xml from the conf folder
Here:
logback.zip (1.4 KB)
What kind of deployment is this? Like K8’s/Yarn etc
I will check with the tech department. The only thing I know is that Dremio is running on an AWS EC2 machine.
What version of Dremio is this?
This one:
@Lincoln_Borges It seems like you have a very old logback.xml, during upgrade, do you copy over conf folder from the previous version? Here is a logback.xml from 23.x which should be same as 24.x
Can try one of the 2 things below
- Copy over the logback.xml from v24 to your current logback.xml folder under the
<DREMIO-INSTALL-DIR>/conf
- Copy the one attached and overwrite the one under
<DREMIO-INSTALL-DIR>/conf
Restart coordinator, run some queries, check if queries.json has got back all columns
logback.xml.zip (1.8 KB)
@balaji.ramaswamy
It worked!
With this logback.xml you provided, my “queries.json” has lots of information now.
Thank you 
@Lincoln_Borges Glad it worked, thanks for the update
