Hi,
Source table count for a particular table is 140 million . While trying to query target file from Dremio ,its throwing error as per below snapshot.
When I am adding a filter clause to restrict data then count is giving valid result.Just wondering if there is any limitation on how many record count can be shown as output in dremio.
Please suggest on the fix.
Thanks in advance !
There isn’t a limit to the number of records you can count. It looks like you have an issue in your CSV data. What is the source you are querying?
If it is a file system and you defined the formatting through Dremio then you can experiment with some of the formatting settings to handle the unescaped quote in one of your records.
Source for Dremio is a csv file placed in Azure data lake which is sourced from Oracle database via Talend.
Above issue isn’t resolved even if I use different settings in Dremio.
What escape character are you configuring for the CSV file?
Double quote or Backslash.Both aren’t working.
@Santosh let us try to reproduce this internally. Could you provide a few line sample, and a screenshot of your formatting settings (delimiter, etc.)?
@can ,
Hi can i we are querying the count of subset of data we are able to get count , but in huge dataset 140 million of data we are unable to do , but if we applying format settings(quotes as single quote and escape character( back quotes ) then we can get the data from the whole data set , but its not working for other subset data count it returns Zero if we applying that confition, if we reformat to quotes and Escape character as " (double quote ) then we are able to get the data.
settings for whole dataset.
|690x208
