Hello
I wonder if anybody could tune workload feature to set queue depending on sourcename ? I mean when dremio engine calculates costs it knows which sourcename is involved so why would not it use the information to set job in proper queue ?
Also is there any tweak to route ‘stupid jobs’ is dedicated queue or just reject it. Logically Dremio Admin would be able to define what are stupid jobs.
Current rules available are Workload Management | Dremio Documentation
Are you not able to route using any of them like those long running jobs coming from a certain user or cost etc?
Hello
To route based on user name or group membership is done but it’s not enough. Cost is not predictible from an admin point of view and can only be seen via le logs.
My will and idea is to route based on source database name.
Why ?
because some sources can be queried with no restrictions and limits, while some other sources should be queried carefully.
@Arnie I will check if that feature request exists,
When queries come from a certain source via JDBC (if not machine generated), a ROUTING_TAG or a ROUTING_QUEUE can be added and that sent to a separate engine
According to me, Routing_tag or Routing_queue are parameters on the dremio client side (analytics tools) …
Also consider those clients use Arrow_Flight protocol.
My concern it to use a setting based on ‘SOURCE_NAME’ … is the query uses one of the source that, the query should be routed in a queue
(
USER IN (‘JRyan’, ‘PDirk’, ‘CPhillips’)
OR is_member(‘superadmins’)
)
AND query_type IN (‘ODBC’)
AND EXTRACT(HOUR FROM CURRENT_TIME)
BETWEEN 9 AND 18
AND SOURCE_NAME IN (‘hr_db’,‘account_db’)