Hello community,
I have this huge table in SQL server that has over 7 billion records, I am trying to retrieve a subset of the records through dremio UI and the index column of this table is part of my filter predicates.
But I am running into time out error in dremio as shown below, I tried this query directly on SQL Server and it output results after along run.
`Query cancelled by Workload Manager. Query runtime limit of 1800.00 seconds exceeded.`
An alternative approach would I have considered is a combination of a WHILE loop and OFFSET and FETCH NEXT but dremio does not have a WHILE functionality.
I am reaching out to know if anyone has encountered similar scenarios of big data query. And if there are any other way to make dremio UI execute this query without hitting the time out error.
Thank you,
Samson