Hi all,
I’m using the Dremio 4.0.0 community, and encounter datatype transform error (number to decimal) from Snowflake. I also have the same datatype in Oracle, but Oracle does not have this problem…
Datatype:
- Snowflake: NUMBER(38,0)
- Dremio: Decimal
Here is part of my snowflake-arp.yaml, I follow the config that Dremio provide: snowflake-arp
- source:
name: “NUMBER”
max_precision: 38
max_scale: 37
dremio:
name: “DECIMAL”
Error log:
(java.lang.IllegalArgumentException) Gandiva only supports decimals of upto 38 precision. Input precision : 0
org.apache.arrow.util.Preconditions.checkArgument():136
org.apache.arrow.gandiva.expression.ArrowTypeHelper.initArrowTypeDecimal():118
org.apache.arrow.gandiva.expression.ArrowTypeHelper.arrowTypeToProtobuf():246
org.apache.arrow.gandiva.expression.FunctionNode.toProtobuf():44
org.apache.arrow.gandiva.expression.ExpressionTree.toProtobuf():42
org.apache.arrow.gandiva.evaluator.Projector.make():119
org.apache.arrow.gandiva.evaluator.Projector.make():79
com.dremio.sabot.op.llvm.NativeProjector.build():64
com.dremio.sabot.op.llvm.NativeProjectorBuilder.build():69
com.dremio.exec.expr.SplitStageExecutor.setupFinish():178
com.dremio.exec.expr.SplitStageExecutor.setupProjector():211
com.dremio.exec.expr.ExpressionSplitter.projectorSetup():295
com.dremio.exec.expr.ExpressionSplitter.setupProjector():396
com.dremio.exec.store.CoercionReader.newSchema():155
com.dremio.exec.store.CoercionReader.setup():119
com.dremio.sabot.op.scan.ScanOperator.setupReaderAsCorrectUser():214
com.dremio.sabot.op.scan.ScanOperator.setupReader():187
com.dremio.sabot.op.scan.ScanOperator.setup():173
com.dremio.sabot.driver.SmartOp$SmartProducer.setup():563
com.dremio.sabot.driver.Pipe$SetupVisitor.visitProducer():79
com.dremio.sabot.driver.Pipe$SetupVisitor.visitProducer():63
com.dremio.sabot.driver.SmartOp$SmartProducer.accept():533
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.StraightPipe.setup():102
com.dremio.sabot.driver.Pipeline.setup():68
com.dremio.sabot.exec.fragment.FragmentExecutor.setupExecution():372
com.dremio.sabot.exec.fragment.FragmentExecutor.run():258
com.dremio.sabot.exec.fragment.FragmentExecutor.access$1200():87
com.dremio.sabot.exec.fragment.FragmentExecutor$AsyncTaskImpl.run():658
com.dremio.sabot.task.AsyncTaskWrapper.run():104
com.dremio.sabot.task.slicing.SlicingThread.mainExecutionLoop():226
com.dremio.sabot.task.slicing.SlicingThread.run():156
Where should I go to make changes so that Dremio can catch the right precision?
Please give me some suggestion, any inputs will be appreciated!