Hi, team Dremio.
Based on dremio-coordinator.yaml helm chart, I am seeing that web port 9047 isn’t in the container port. This implies that the load balancer does not balance REST API requests across master and coordinator nodes, contradicting the deployment architectural diagram.
My use case requires load balancing a heavy load of REST API requests, and thus I did manually update the helm chart to load balance REST API requests across master and coordinator nodes. Is there a reason why this wasn’t the default behaviour?
Thanks!
@kyleahn Scale out coordinator is designed only for JDBC/ODBC requests and not REST/UI queries
In that case, the architectural diagram in the official Dremio website should be updated!
I am also curious how one would be able to force manual refresh of external reflection’s metadata at scale. We have a use case where there are thousands of iceberg tables being updated every minute. 1 pod won’t be sufficient.
@kyleahn
We are coming with concept of seamless metadata refresh for Iceberg Tables. We should soon have something. This should eliminate the need for metadata refresh for Native Iceberg tables
Thanks
Bali
1 Like
