How to handle field mismatch

Dremio University
1

I’m working with JSON fields and it seem that one of the fields has a mapping conflict in some of the fields.

Unable to coerce from the file’s data type “struct<options::list<struct<name::varchar, value::varchar>>, handle::varchar, title::varchar, id::varchar, sku::varchar, image::varchar>” to the column’s data type “varchar” in table “<table>”, column “line_items.properties.value” and file “<file>”

The line_items is an array of objects. I want to flatten the array and extract specific fields (sku, price, etc).

How do I work around this?

2

bump - any ideas/advice?

3

@yehosef-tw Sorry for the delay in response,

Is your file format PARQUET? it looks like you have heterogenous schema across files. Is that true?

4

Sorry for the delay - it’s raw json.

5

I got same issue with JSON File - how to deal with it ?

error begins with : UNSUPPORTED_OPERATION ERROR: Unable to coerce from the file’s data type "struct<@…
we are running 22.2.3 release

6

@Arnie Dremio stopped supporting mixed types since Dremio version 17. It looks like the JSON data has heterogenous data type, any chance you can confirm that is true?

7

Correct
Would not it be possible to choose types or nodes to coerce ratier than rejecting the whole file ?
Regards

8

@Arnie, Dremio does coerce certain combinations but not all permutations/combinations.

9

Thanks @balaji.ramaswamy. As I understand Dremio dors not plan to improve this feature or fix errors.