Hi, I’m trying to figure out how to return a large number of lines when I go to view a dataset.
For now I can only get 3968 records (the calculation comes from 4096 - 4096/32 from how I understand it and 4096 is the Dremio Narrow Batch (?)).
I am using a JDBC plugin and from the dataset preview I get the following:
Results may be truncated for performance, execute over ODBC/JDBC to save complete results if more than a few thousand rows.
How can i act? Thank you.
Hello @andreat,
Where specifically are you seeing this error? Dremio has a UI preview (not JDBC/ODBC) and this type of query does a sampling of the result set.
Do you have query profile for this job?
The error was in both the preview and the run (UI). The error was mine (on the jdbc driver), i fixed it and now it can easily overcome the batch size. Thank you!
@andreat , My team is facing the same problem. We have been living with it for a year now. What was the error? How do we fix it?
Sorry for the delay, in any case the JDBC driver has a method which is setFetchSize (rows), this method is called by Dremio with the rows = 3968 parameter when you specify, through the JDBC plugin UI, a Fetch size greater than 3968, otherwise the rows parameter is the parameter inserted in the UI. I had modified the JDBC Driver so that the call to setFetchSize (rows) set the maximum number of rows to be fetched at the moment in which you go to make the SQL query. To solve this, I made sure that the setFetchSize method did nothing. Let me know if it’s unclear and I’ll try to help you.