Woke up today morning to multiple scheduled queries returning NullPointerException.
I found that queries with where expression of X not in (a,b,c) run into this problem. It works if I remove that condition.
Here’s a job profile with the bug.
137f82a8-da14-43b8-86d9-7dbc7333e5cf.zip (68.4 KB)
Here’s a job profile where I changed the query to work around the bug. But this is far slower for me.
97e0e259-fb27-416d-9f93-506515ba395e.zip (95.2 KB)
This bug is when I run the query on a cluster with 1 coordinator and 3 executors (12GB each).
I have another cluster with 1 coordinator and 1 executor (100GB). The query runs without any problem on the single executor cluster.