Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is it possible to write a single dbt test which applies to all tables in the whole project?

To ensure that new data is constantly arriving in Snowflake without interruption, I want to add some tests to my dbt project. I want to use freshness_anomalies test in Elementary, which is a package supporting dbt.

I can't figure out a way to only set up this test once so that the test is done on all tables. What I can do so far is something like this: excerpt from sources.yml

But this way, I have to add these lines for the test for each and every table I am creating in dbt, and I have many...

Is it possible to write the test only once? Btw, every single table has the column "META_LOAD_DATETIME", so that shouldn't be an issue.

like image 969
Bish Avatar asked Oct 20 '25 04:10

Bish


1 Answers

No, unfortunately for you, tests are Properties, not Configs, so they can only be defined in "Property" files (i.e., schema.yml files). Source. This means you need to write out the test property on every model/seed/source/etc. that you want it to run on.

.yml files are easy to generate programmatically, so if you wanted to do this hundreds or thousands of times, I would write a script for that. You could also use something like dbt_pre_commit to enforce the presence of this test for new models.

like image 184
tconbeer Avatar answered Oct 25 '25 15:10

tconbeer



Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!