Support for sequence file from S3

Does DremIo support reading sequence files from S3 storage ?

yes, Dremio can read sequence files stored on S3 by creating a Hive connection in Dremio and accessing tables that are backed by S3.

Thanks Jason. Few questions

  1. If i write data in S3 directly in parquet format. Then try to query through

a) Directly from DremIO - S3 connector
b) Through Hive as external table to S3 bucket and then from DremIO-Hive connector.

Which one is recommended as efficient to use in DremIO from performance and scale prospective ?

If you connect to Hive Dremio will pick up the metadata from Hive’s metastore but will read the data using Dremio’s vectorized Parquet reader. So, the performance should be the same.

For some formats like Avro Dremio will use Hive’s readers instead of its native readers. But for Parquet, JSON, CSV, and other file formats Dremio will use native readers.