Hi Dremio,
sorry, it‘s again me. Data lineage in Dremio looks awesome. Thumps Up!
But looking at the (left) source side of DL in Dremio, it ends, before most of the fun even starts. Very often the source that Dremio will access is not the ultimate source of the data but again an interim step. E.g. a data warehouse runnning on top on an ERP system. I know that Dremio can not (yet) look into such sources and resolve views and stored procedures.
If we would be able to add a custom lineage graph (via API) for a given dataset somehow attached to the source then this would be very cool. Full data lineage!!!
As this dream is far, far away (at least I assume), I would like to propose a simple short cut for this:
Please provide the optional ability to add a custom link address to each data source that may occur as a small link icon below or above the source icon. When users click on this a new browser tap opens the link.
If you would then also allow to put some placeholders into such URLs like this…
https://someurl?svr={dremio-cluster-id}&id={dremio-dataset-id}&usr={dremio-user}
…then Dremio would be able to seemlessly interact with predecessor system and meta data tools on data lineage.
Just an idea…
Thomas