Re-Opening: Dremio 19.0.0 Regression - "java.lang.UnsupportedOperationException) Unknown type to copy."

1

Starting with Dremio 19.0.0, When I’m trying to run a simple query over a (big) Json file read from S3, I’m getting the following error if I add a WHERE filter over a BIGINT field:

  (java.lang.UnsupportedOperationException) Unknown type to copy.
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():999
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.FieldBufferCopier$ListCopier.<init>():722
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():987
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.FieldBufferCopier$StructCopier.<init>():632
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():978
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.FieldBufferCopier$StructCopier.<init>():632
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():978
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.FieldBufferCopier$StructCopier.<init>():632
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():978
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.FieldBufferCopier$ListCopier.<init>():722
    com.dremio.sabot.op.copier.FieldBufferCopier.addValueCopier():987
    com.dremio.sabot.op.copier.FieldBufferCopier.getCopiers():1014
    com.dremio.sabot.op.copier.VectorizedCopyOperator.setup():101
    com.dremio.sabot.driver.SmartOp$SmartSingleInput.setup():255
    com.dremio.sabot.driver.Pipe$SetupVisitor.visitSingleInput():73
    com.dremio.sabot.driver.Pipe$SetupVisitor.visitSingleInput():63
    com.dremio.sabot.driver.SmartOp$SmartSingleInput.accept():200
    com.dremio.sabot.driver.StraightPipe.setup():103
    com.dremio.sabot.driver.StraightPipe.setup():102
    com.dremio.sabot.driver.StraightPipe.setup():102
    com.dremio.sabot.driver.Pipeline.setup():68
    com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution():405
    com.dremio.sabot.exec.fragment.FragmentExecutor.run():269
    com.dremio.sabot.exec.fragment.FragmentExecutor.access$1600():94
    com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():747
    com.dremio.sabot.task.AsyncTaskWrapper.run():112
    com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop():243
    com.dremio.sabot.task.slicing.SlicingThread.run():171

Following the stacktrace to com.dremio.sabot.op.copier.FieldBufferCopier and git blaminng, it seems this class changed the way it calls addValueCopier since 19.0.0. This behavior is not spotted earlierFollowing the stacktrace to com.dremio.sabot.op.copier.FieldBufferCopier and git blaminng, it seems this class changed the way it calls addValueCopier since 19.0.0. This behavior was not spotted in earlier versions

The failed query profile is also attached

9999a11a-a69f-4951-b5f5-cec162be4a15.zip (29.3 KB)

2

@sheinbergon It looks like a known issue that will be fixed in the upcoming version, please watch out for the release notes