How to handle CSV files that contain "\r" (Carriage return) character?

1

Hey Guys,

I am trying to create a physical dataset by dropping a CSV file in S3 folder. The file contains “\r”
(Carriage return) character. While changing the format of physical dataset, if i use “LF-UNIX/Linux” as the Line delimiter then “\r” character is not removed from the data and if i use “CRLF - Windows” as Line delimiter then the “\r” character is removed but data in the columns are shifted for few rows.

Can someone suggest a Line delimiter so that “\r” can be removed without shifting data in the columns?

I am using MAC OS and whenever i try to edit a file or create a new file “\r” gets added by default?

Can someone suggest a way around so that “\r” does not get added and i can use the default Line delimiter present in dremio?

2

@Varun, do you have a sample file with the problem you can share here?

3

Just use dos2unix command on all files giving you such problems.