Hi,
Is it possible in dremio that we take different data sets from different source and combine them and convert into single file format like JSON and store into a single data source like s3?
@Kom_al You can union the difference queries (# and type of columns should be same) and then do a CTAS as PARQUET or ICEBERG
Thanks @balaji.ramaswamy but my question is I want to try Dremio as a tool through which we can connect multiple data sources and convert them into a single data source and that single data source we can utilize in our integration. By that our integration model will be fairly easy. So is it possible through dremio that after querying the data , it stored into S3, so that we can use S3 in our model as a single source of data.
@Kom_al You can write whatever query you want and CTAS into S3. Then query S3 as a single source
With V24 you can also use COPY INTO