How do you store data into HDFS?

1

I want to save data from my vertica database into HDFS. How do I do that?

I have enabled CTAS for HDFS

2

@puspitab

This is how you do it.

You have already done step 1 by enabling CTAS on the HDFS source

Now run a query on any source say Hive and make sure it works. Then add a “create table hdfs-source-name/hdfs-folder1/some-table-name” as select statement and it should write the output of the query as Parquet files

Create a test folder on HDFS

$ ./hadoop fs -mkdir /ctasexample

Run CTAS query on a different source via Dremio

create table localhdfs.ctasexample.atp_hdfs as SELECT * FROM localhive.“default”.atp

See below PARQUET files created

$ ./hadoop fs -ls /ctasexample/atp_hdfs
Found 1 items
-rw-r–r-- 3 balajiramaswamy supergroup 1351 2019-11-22 09:04 /ctasexample/atp_hdfs/0_0_0.parquet

3

I’ve already tried it but it says ‘unexpected error occurred’

image