Cannot Use all Cores When Running Jobs

Hi, new to Dremio. I have set up a standalone cluster on EC2 on AWS. Even for heavy usage, it seems that the CPU on the box will go to 12%, and there are 8 cores. I am assuming that Dremio is only making use of one core, though I’m not sure. When I scale the instance from 8 cores to 24, I see a very similar pattern using a single core.

I see this usage in the Web UI.

Any tips or tricks to get Dremio to use more of the CPU?

Thanks!

What is Dremio doing when monitoring CPU usage? Can you share a query profile of a “heavy usage” job?

When I try to export, I receive this error: Something went wrong /tmp/dremio-support/5be6e12d-836f-4b62-b7a6-a2cab1587121.zip (Permission denied)

It looks like the permissions on the exported file belong to ec2-user instead of dremio.

Also, that file is 300MB. What is in it and is it safe to post? Thanks!

Hey Scott, you should be able to either add write access to /tmp/dremio-support for the dremio user or change the support path by changing this system property: -Dsupport.temp=/a_path_that_dremio_can_write_to/

You can also inspect the zip file so you understand what it includes to make sure you comfortable sharing. Generally it includes the query, telemetry about the query, as well as potentially logging that happened during the query.