Configuring the number of columns a query can scan in 23.0

1

The documentation for Dremio 23.0 seems to suggest that queries can no longer scan more than 800 columns. Is there a support key available for configuring this limit, for users who have to come rely on the ability to scan more than 800 columns?

From here:

The maximum number of leaf columns that a query is allowed to scan per table or per reflection is 800. (In earlier releases, this limit did not exist.) If a query attempts to scan more than this limit, then you receive this error message:
2

@pvallone-ni this should be it → store.plugin.max_leaf_columns_scanned (v23 & above only)

1 Like
3

Could we set this in Helm chart?

4

You can do this via SQL query:

 ALTER SYSTEM SET store.plugin.max_leaf_columns_scanned = 800;

If you want it via Helm, then this post can help you. Basically append the support key to -Ddremio.debug.sysopt.

1 Like
5

Spectacular! Thanks! Worked for me