We connected Dremio to Redshift. We have some queries that are written in this format.
SELECT col_1, col_2, col_3 FROM redshift.our_db.our_schema.our_table WHERE col_2 IN (value_1, value_2....)
When the amount of records in the IN clause are bigger than 1.5k we started to receive weird errors like
Source ‘redshift’ returned error 'ERROR: syntax error at or near “,”
Also note that the error don’t appear, as far as we enable a reflection on a vds that use the phisical table. We cannot use reflections as data is refreshed more often than 1 hour, and reflections are currently stucked time to time.
Any idea about what is going on?