Query becomes slow with "Limit" operator

1

Hi,
i’m using a dremio cluster with 4 nodes, version 4.1.3.
In my case i have these vds:

  • vds_a join
  • vds_b -----------> vds_d
  • vds_c

if i query vds_d (with a reflection on it) with a limit operator the query doesn’t uses the reflection on the vds_d but uses the reflection on the parent of the vds (a,b,c).
If the value of the limit is very high (near the value of input records in the “with-limit” profile), the query on vds_d use the reflection on it (profile truncated).

I attach the query profiles
no-limit.zip (48,6 KB)
with-limit.zip (47,9 KB)limit-truncated.zip (32,7 KB)

2

@LucGth

In case of a full run Dremio was able to match more reflections. See visualized plans between the limit and no limit queries. In the no limit query there were 2 extra reflections that were chosen

Previews ma sometimes not substitute the best reflections. When a query is run via JDBC/ODBC/REST we always default to “run”

3

thank you for the answer!
I see for all the profile “queryType”:“UI Run”. I didn’t use preview mode.

bye