Code too large error in Dremio 3.0.0

1

I got such error for a 1000-element list in where clause.
E.g. where id in (1,2,3,…1000)

CompileException: File ‘com.dremio.exec.compile.DremioJavaFileObject[FiltererGen16448.java]’, Line -21997, Column 20: FiltererGen16448.java:43539: error: code too large

I searched community topics and saw one similar issue for project gen in 2.0 version.

Is there any limitation for SQL size?

2

Hello @pisces312,

Can you provide a profile for the query where you saw this error?

3

This occurred when I commit a Dremio SQL query via REST API. Sorry, I cannot provide the job profile.

The length of query string is 14000+ and the where clause has a 1000 id list.

Error
SYSTEM ERROR: CompileException: File ‘com.dremio.exec.compile.DremioJavaFileObject[FiltererGen16448.java]’, Line -21997, Column 20: FiltererGen16448.java:43539: error: code too large
public boolean doEval(int inIndex, int outIndex)
^ (compiler.err.limit.code)

SqlOperatorImpl FILTER
Location 1:26:3
SqlOperatorImpl FILTER
Location 1:26:3

4

It seems about Dremio generated Java method exceeds 64kb. The doEval method of FiltererGen16448 class is 385KB long.

5

@pisces312, can you share the exact size of the IN clause set? What’s the data type for this column?

6

@ben, yes, the total length of the int64 list string with brackets is 12001 long. Each int64 element has 11 digits.
The size of list is exactly 1000.

E.g. (12345678901,12345678901,…)