I know Dremio is not ETL tool but claim to act as DWH without need of ETL.
could someone quickly advice what sql or dremio functions in dremio could replace following typical transformation functions:
- Source Qualifier Transformation
- Router Transformation
- Rank Transformation
- Sequence Generator Transformation
- Lookup and Re-usable transformation
- Expression Transformation
- Mapplets
these I know can be done:
- Aggregator Transformation
- Joiner transformation
Thanks in Advance for your help
@Palkers76 Dremio is a SQL engine for data lakes so wondering if each if these have a SQL function mapped and we can see if we can map a Dremio SQL function to it
Not everything i guess or even if its possible performance maybe an issue.
Or in some cases like lookup to mdm for surrogate Key which is essential for dwh.
So for each dimention record we need to get a Key or other tyle of record enrichment .
@Palkers76 Dremio would be suited for last mile ETL using CTAS jobs, so whatever is available via the SQL library is what is possible now
http://docs.dremio.com/sql-reference/