Basically planning is reading the db metadata. Look in your dremio conf on master node where it is storing the db folder - ensure that access to it is fast from the cluster which is running it slow.
If you have two clusters each with master node, then check they both have their own unique db folder end-point, not shared. If it is shared, one of the coordinators will automatically become a stand-by as the master will lock the db folder database.
Either way, the problem is in the environment. As a test you can do this test, stop the cluster that is running planning fast. Take the 2 config files dremio conf and env and copy them to slow planning running master node, start it and test it. Ensure you only do select type operations, no not create anything. Ensure no one is connecting to your cluster also.