Error when querying Redshift with IN clause

1

We connected Dremio to Redshift. We have some queries that are written in this format.

SELECT col_1, col_2, col_3
FROM redshift.our_db.our_schema.our_table
               WHERE col_2 IN (value_1, value_2....)

When the amount of records in the IN clause are bigger than 1.5k we started to receive weird errors like

Source ‘redshift’ returned error 'ERROR: syntax error at or near “,”

Also note that the error don’t appear, as far as we enable a reflection on a vds that use the phisical table. We cannot use reflections as data is refreshed more often than 1 hour, and reflections are currently stucked time to time.

Any idea about what is going on?

2

@nicor88 let uus try to solve both your issues,

#1 For the error returned by redshift, are you able to send us the profile of the failed job
#2 When you say reflections are stuck, do they take more than an hour to refresh or they never complete? Do you have a profile of such a job that is stuck?

3

#1 I cannot send the full profile as contains sensitive data. In attachment the log attempt.
#2 For the 2nd I have a another tread with details. See this.

log_attempt_0.json.zip (1.9 KB)