BUG with Postgres source: UnsupportedOperationException: BigDecimal scale must equal that in the Arrow vector: 13 != 10

1

Connecting to Postgres using Dremio Community 4.01, I get this error: UnsupportedOperationException: BigDecimal scale must equal that in the Arrow vector: 13 != 10

This is with a query: select * from users

Work-around: select *, 1 from users

This suggests it’s not an issue interpreting a numeric (i.e. decimal) field.

PostgreSQL 9.6.15 on x86_64-pc-linux-gnu, compiled by gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-36), 64-bit

2

@dah33

Can we please have the below

  1. Profile from the failed job
  2. Profile from the successful job
  3. \d users – from postegres
3

1 and 2 attached, with a CREATE TABLE for 3. It’s a client project, so I don’t have access to psql, so hopefully this is fine. Downloads.zip (51.7 KB)

4

@dah33,

This error is caused by having numeric type columns in your Posgres table which did not have the precision or scale specified. Dremio 4.1.x will now simply truncate the values in this column to the smallest scale it encounters instead of throwing the error.

You can also fix this by altering the Postgres table to specify the precision and scale for the numeric columns where they are unspecified (e.g. column_a decimal --> column_a decimal(38,16))