Hi Team ,
I am trying to write data into tabular format : CSV file after processing the data retrieved from dremio using apache arrow flight connector using JAVA .
How can I write(batch) the processed data using apache arrow in tabular format - CSV (or parquet file) using JAVA. Plan is to store these data to azure data storage further post processing .
Could you please help.
For CSV, VectorSchemaRoot has a method to generate a TSV string, convertToTSVString(). This doesn’t seem very efficient to store back though.
For parquet, there are examples in C++ and Python in the Arrow documentation:
Dremio itself lets you download results as Parquet or CSV files in its web UI.
Thanks for response. Is there any java package to write dremio data into parquet file using flight connector client.