I'm using DBT generic tests to monitor my BQ DB.
While using the "not_null" test, it's compiled as :
select * from 'server.schema.table' where tested_column is null
In my case, selecting the column itself vs selecting all is ~70 Gb difference.
Is there any way to edit generic tests in the code?
Tried to read documentation.
I would expect the compiler to change the query to select the tested column and not ALL columns.
While there is some customisation you can do the generic tests are a one-size-fits-all way of doing things.
It's possible to define a 'custom generic test' to do what you want.
However, an alternative might be to look at various DBT packages that define a variety of generic tests. For example, DBT Expectations defines an expect_column_values_to_not_be_null test which extends the functionality of the basic not_null test. From the docs it sounds like it would do what you want.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With