New field crlf must be included in the query?

1

I have a CSV file, which contains 80 columns
When I tried to load it into dataset, there would be 81 columns (including new field: crlf)
If I tried to query this dataset like:
select col1, col2 from AAA
It returned only 1 record
(Sometimes it returned multiple records, but only the first record has data, the remaining contains null value)
But if I query like:
select col1, col2, crlf from AAA
The data returned as I expected
Could you please help to explain why
Thank you so much

2

Dremio typically doesn’t add more columns. CRLF = Carriage Return Line Feed. I assume there might of been an issue formatting the CSV during import. Did you change any of the default dataset settings like escape char, line delimiter, field delimiter, etc? It may be worth playing with those settings.

Also, is this a CSV you can share?

3

Hi Anthony,
I didn’t change anything, just use default settings
After that, I already tried to change settings, but crlf column is still added automatically
This CSV come from our customer, so I cannot share it unfortunately
If only crlf column is added, it would be no problem. But if we don’t use crlf in the query, the result would become incorrectly. This is a big problem because I have to use trick to overcome
Is there any clue to explain why crlf must be included in the query
Thank you so much

4

It’s important to mention this is not normal behavior. There is likely some formatting or encoding issue with your raw CSV that I want to investigate (hence why I asked for a sample). We have many users that work with CSV files that haven’t experienced this. The reason I asked to change some of the default dataset settings like escape char, line delimiter, field delimiter, etc is to not add the extra column during import. Is there anyway to mask or hide some of the data to share CSV?