ac64b3f0-2f0e-4d8d-9eb9-9c7ce85f4103.zip (9.1 KB)
If I try and return a simple query with FLATTEN and the array is empty, I get a query error
Unable to find the referenced field: [expr000].
@jhaynie, what type of data source is devdata? What are the formats of the source datasetsdevdata.jira.sprint and devdata.pinpoint.sprint_detail" (Parquet, JSON, Mongo docs, etc)
it’s a parquet file. the contents of the array are string (defined as a LIST in parquet with string value type). it appears that if there’s at least one row in the data set that has a value it works fine. when none of the rows have a value (such as []) it fails. to work around, we had to create a dummy row in the DB (which is horrible).
@jhaynie, any chance you can share a sample file that produces the flatten error?
yes, i’ll upload here shortly.
Hello @jhaynie,
Any chance you can supply the file here with the empty list?
Is this the logical parquet type to which you refer: https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#lists?