[BUG] Limit with get_json_object UDF

1

If I use get_json_object with an object too big, I have this error:

SYSTEM ERROR: IndexOutOfBoundsException: index: 0, length: 2199 (expected: range(0, 1024))

SqlOperatorImpl PROJECT
Location 1:8:2
Fragment 1:8

[Error Id: 0538733f-2d71-4e3a-a5eb-0f7cc06acb84 on ip-10-102-22-244.eu-west-1.compute.internal:-1]

  (java.lang.IndexOutOfBoundsException) index: 0, length: 2199 (expected: range(0, 1024))
    io.netty.buffer.AbstractByteBuf.checkIndex0():1125
    io.netty.buffer.AbstractByteBuf.checkIndex():1120
    io.netty.buffer.UnsafeByteBufUtil.setBytes():349
    io.netty.buffer.PooledUnsafeDirectByteBuf.setBytes():199
    io.netty.buffer.WrappedByteBuf.setBytes():397
    io.netty.buffer.UnsafeDirectLittleEndian.setBytes():34
    io.netty.buffer.ArrowBuf.setBytes():759
    io.netty.buffer.ArrowBuf.setBytes():45
    io.netty.buffer.AbstractByteBuf.setBytes():528
    com.dremio.s.ProjectorGen722.doEval():127
    com.dremio.sabot.op.project.ProjectorTemplate.projectRecords():53
    com.dremio.sabot.op.project.ProjectOperator.outputData():192
    com.dremio.sabot.driver.SmartOp$SmartSingleInput.outputData():177
    com.dremio.sabot.driver.StraightPipe.pump():56
    com.dremio.sabot.driver.Pipeline.doPump():82
    com.dremio.sabot.driver.Pipeline.pumpOnce():72
    com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():288
    com.dremio.sabot.exec.fragment.FragmentExecutor$DoAsPumper.run():284
    java.security.AccessController.doPrivileged():-2
    javax.security.auth.Subject.doAs():422
    org.apache.hadoop.security.UserGroupInformation.doAs():1807
    com.dremio.sabot.exec.fragment.FragmentExecutor.run():243
    com.dremio.sabot.exec.fragment.FragmentExecutor.access$800():83
    com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():577
    com.dremio.sabot.task.AsyncTaskWrapper.run():92
    com.dremio.sabot.task.slicing.SlicingThread.run():71

Anyone have a workaround?

2

It is a bug where we are not expanding the buffer. Unfortunately there are no workarounds. We will have it fixed in 1.4.2 which we are expecting to release in a week. Sorry for the trouble.

1 Like
3

Good to know. Thanks for the reply.

4

When is it expected to release version 1.4.2?
thx

5

@JoyJava it should be released within the week. We’ll announce it on community and on our website.

1 Like
6

Very much looking forward to this version update

thx

7

@JoyJava

You should be able to download the latest version and this issue has been fixed

Thanks,
@balaji.ramaswamy