There is a problem when loading a CSV with Windows end of line (CrLf).
Generate a file with only 1. Here is the code to do so:
one = b’\x31\x3b’
with open(‘buggy.txt’, ‘wb’) as fw:
for _ range(100000):
–for _ range(50):
Now if you create a data source on this file. You import it with Windows end of line ‘\r\n’.
And you do:
select * from ‘buggy.txt’ where A = ‘’;
You will find some lines … You should not given that all fields are filled with 1.
Apparently, it is linked to the size of the file and for the lines that begin on an address + 1 divided by 4ko.
Our Dremio version is