No columns for data from AVRO files

1

I’m trying to get some data in Dremio from AVRO files

  • I’ve tried both via S3 and via NAS.
  • I can select the file and it opens nicely in the preview, I can see the different columns in the table (the only strange thing is that the format drop down is empty and that there is no AVRO to be found)

but after I save the table, there doesn’t seem to be any column data. (although I don’t get any error messages)
When I open the SQL Editor view, I get “Selected table has no columns”

The avro file has a simpel model, 3 string fields, 1 array of strings field.

Could anybody point me in the right direction?
Thanks
Quentin

2

Hi @quentin

Are you trying to upload an AVRO file though the file upload via UI and getting the error in the attached screenshot?

avro-upload
avro-upload.png2458×1486 128 KB

3

When I try to do it via file upload, I get the same error indeed.

I tried it via adding them from S3 and from a NAS, and then at least the preview worked…

4

Hi @quentin

Would it be possible for you to map this AVRO file as a Hive external table and then query the AVRO file via Dremio-Hive? Currently we do not support direct reading of AVRO files

Thanks
@balaji.ramaswamy

5

Hi @balaji.ramaswamy

We tried creating hive external table which is in avro format. And its working fine from hue and hive cli.
However in dremio its failing with error "Failure while initializing Hive Reader "

message: "cannot find field transaction_id from [0:error_error_error_error_error_error_error, 1:cannot_determine_schema, 2:check, 3:schema, 4:url, 5:and, 6:literal]"
6

@mmohan

Can you please send me the “describe formatted cards_transactions” and “show create table cards_transactions” from Hive cli. Als ocan you also get us the avro tool schema for the avro file?

7

By any chance it is sensitive on case of column names?
and hive partition tables in avro format is an issue?
queries.zip (1.0 KB)