Hi Team,
We have deployed dremio 4.5 on the GKE cluster. After upgrading it to dremio 4.7, we found that new pvc(volumes) got created for the executors. Because of that physical reflection files are not available.
Is it expected? or Is there any way to handle this?
Thanks & Regards,
Ankita
Does that mean you are storing reflections in local disk on the executor? Have you configured to create new volumes in your helm charts. Have you used helm charts provided by Dremio? For now, best way forward is to refresh the reflections again
@balaji.ramaswamy Yes, we are storing reflections in the local disk on the executor. As we are using default configurations. We have used helm charts provided by Dremio, without making any changes we just did helm upgrade.
Earlier we were using “dremio-cloud-tools/charts/dremio at master · dremio/dremio-cloud-tools · GitHub” helm chart and now we are using “dremio-cloud-tools/charts/dremio_v2 at master · dremio/dremio-cloud-tools · GitHub” with dremio 4.7.
What is the recommended way to store reflections?
PFA screenshot of PVC
It would be best to store reflections on a distributed storage like S3/Azure Storage/HDFS and then create C3 for the reflection files. Were the reflection on “/opt/dremio” or “/opt/dremio/data”?
Reflections are stored in “/opt/dremio/data”. Does it have any performance impact if we store reflections on a distributed storage?
Slight performance impact, that’s why we have C3
