I have two datasets that have about a billion records that have succesfully built raw reflections on them which I can query the count of in less than 1s using the reflections. Then I have a dataset that unions those two datasets (they have the same schema). Doing a count on them produces a memory error in dremio (after a long while). I can produce the same results by simply adding the two counts. Why is union performance in this case so bad? It appears that the union is using both the raw reflections of the corresponding datasets succesfully but the query is not accellerated by them.
I also am trying to build a raw reflection on the union itself, and its nearly pegging out a 15 node cluster which seems unexpected if it has the two reflections of the datasets that its unioning together.