FLATTEN with empty rows fails


#1

ac64b3f0-2f0e-4d8d-9eb9-9c7ce85f4103.zip (9.1 KB)

If I try and return a simple query with FLATTEN and the array is empty, I get a query error

Unable to find the referenced field: [expr000].


#2

@jhaynie, what type of data source is devdata? What are the formats of the source datasetsdevdata.jira.sprint and devdata.pinpoint.sprint_detail" (Parquet, JSON, Mongo docs, etc)


#3

it’s a parquet file. the contents of the array are string (defined as a LIST in parquet with string value type). it appears that if there’s at least one row in the data set that has a value it works fine. when none of the rows have a value (such as []) it fails. to work around, we had to create a dummy row in the DB (which is horrible).


#4

@jhaynie, any chance you can share a sample file that produces the flatten error?


#5

yes, i’ll upload here shortly.