Long ago, I used JUnit, Maven and other things for check DB model with some SQL queries.
Nowadays, we need check the DB model again.
There is a easy way to do this with my model in dremio?
I mean, something that run a script (with a sql query) with some frecuency and tell me “pass” or not.
Thanks in advance!
and have a great 2019!
When you . say DB model, are you talking about the different relationships between the tables? In case of Dremio the VDS to PDS relationship?
DBUnit and the like are usually for mutable data testing.
As Dremio is readonly, what do you want to achieve ?
If your data is in a data warehouse, you can try dbt for testing.
Hi Balaji & Fetanchaud
Thanks to both for your replies. I taked a look to dbt (Data Build Tool) and I maybe this the correct road.
As I mention before we receive some files (.csv) from different sources like SAP and JDE that have data of tables (i.e. CDPOS, BKPF and others from SAP and the same way from JDE system we recieve F42199, F4211 and others )
Our main objetic is check if the files haven the right structures ( I mean the fields)
Suppose that I receive a new file CDPOS_20190109141558.csv this file have the new data ( or delta data) we want to load in a temp folder in dremio, run test (with dbt) and check if we have all fields we need in this file.
If we have all fields in the new files, then we move the new file to final folder.
Would be great that dbt_ write some king of log file or log table for monitoring.
When have news about that, I’ll mention here