Hello, currently we have a ELT process that using CTA “copy” data from datasource (db2) and save as parquet file in local NAS, so the question is we need raw reflection on this PDS of parquet file? because already are in parquet, so raw reflection can accelerate and have better performance?
@dacopan Of course reflections can still improve your query performance. I think you should review these best practices on when to use reflections here: Dremio This was written for Dremio Cloud but it applies equally well to Dremio Software.
@bennychow If the reflection is a select * and writing back to NAS, not much, unless the original PARQUET dataset has too many small files. I would also recommend not creating it on all rows and all columns (if possible). Check the dashboard reporting requirements and see if you can create it on select columns. Selected rows might be difficult but if there is historical data also on the same PDS then that is a possibility, by doing this the foot print (size) of the reflections will be small so the time to refresh will be fast, the query using this reflection will run faster and it will take up less space to store
2 Likes