we have multiple dashboard on qlik they are scheduled at 1 hour gap, So dremio is running the query every hour but sometimes it gives this error after 4 or 5 mins: RpcException: Timeout while waiting for client to be in ready state.
And qlik goes for 3 hours because of this error.
I have also tried solution suggested by @balaji.ramaswamy where I have configured
this property from 50,000 to 120000.
and it is still not working
So @balaji.ramaswamy let me know what should I do to solve this, we are using dremio comunity and facing this error in production.
You should investigate what is happening on the client side. The timeout is 50 seconds to process each batch returned by the server which is quite long and you have increased to 120s. It looks like you are sending 4,916,352 records and close to 1 GB of data. Can you try and see if sending a subset of this dataset also times out?
@balaji.ramaswamy You mean BI tool? Currently we are using qliksense, and just for your information, it was working fine before, we are getting this error since last 3 weeks.
And I have also provided this solution in qlik, with each query we are connecting and disconnecting.
@fenilj That is a great data point that this issue has been happening fot the last 3 weeks. Did you change anything on the Dremio side (server) like version upgrade? driver change? Was flight introduced 3 weeks back?
Similarly, were there any changes on the client (Qlik) side?
@balaji.ramaswamy No there is no change in dremio side, But we upgraded the data gateway in qlik and also 3 weeks ago qlik was having some issue but it was resolved in one day.
Sorry for late reply, But i don’t know what exactly was the issue, But it was from qlik side for particular region and for data gateway, So we simply updated the gateway to latest gateway and we haven’t faced that problem again.