The specific usage of the "to_number" function causes "OutOfMemoryError"

1

Please test the following SQL:

SELECT TO_NUMBER('A','')

This SQL will execute for a very long time and get an error: unexpected error occurred.

This is not the intended use of “to_number”, but it is allowed.

If the following SQL statements are used, the memory will be consumed quickly and the dremio process will exit:

SELECT TO_NUMBER("城市",'') FROM orders

log:

Dumping heap to /home/dremio-community-21.1.1-202204292111390812-57b1832f/log/java_pid29128.hprof ...
Heap dump file created [2530035144 bytes in 21.807 secs]
Dremio is exiting. There was insufficient heap memory to continue operating.
java.lang.OutOfMemoryError: Java heap space

This is the data file I use:

orders.zip (403.7 KB)

I don’t think it has anything to do with memory setting, because I expect the operation to be very simple, and it should not occupy too much memory.

2

@balaji.ramaswamy Please take a look at this question.

3

@bigfacewo

Thanks a lot for pointing out this issue, we have an internal ticket to see what is going on

Thanks