Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Dealing with missing metrices in calculations

I use a formula to calculate approx. maximum memory consumption of MySQL instances that (simplified and written in PromQL) reads:

(
    mysql_global_variables_key_buffer_size +
    mysql_global_variables_query_cache_size +
    mysql_global_variables_tmp_table_size +
    mysql_global_variables_innodb_buffer_pool_size +
    mysql_global_variables_innodb_additional_mem_pool_size +
    mysql_global_variables_innodb_log_buffer_size +
    (
        mysql_global_variables_max_connections *
        (
            mysql_global_variables_sort_buffer_size +
            mysql_global_variables_read_buffer_size +
            mysql_global_variables_read_rnd_buffer_size +
            mysql_global_variables_join_buffer_size +
            mysql_global_variables_thread_stack +
            mysql_global_variables_binlog_cache_size
        )
    )
)

Unfortunately the mysql_global_variables_innodb_additional_mem_pool_size metric is not always present for every instance resulting in "no data" if this it is included in the calculation.

There's the absent(v instant-vector) function that may be used to solve this but I am not sure how.

I would like nonexistent metrices be replaced with a constant (0 in this case). Is it possible?

Could you please provide me with some hints on how to deal with missing metrices in calculations in PromQL?

like image 233
mjf Avatar asked Jan 03 '23 15:01

mjf


2 Answers

mysql_global_variables_innodb_additional_mem_pool_size or up * 0

https://www.robustperception.io/existential-issues-with-metrics/ looks at this problem in more detail.

like image 110
brian-brazil Avatar answered Feb 20 '23 22:02

brian-brazil


Accepted solution won't work well in case we would try to sum two metrics both of them can be missing. In my specific case it is mysql_info_schema_innodb_metrics_transaction_trx_rseg_history_len (from mariadb) and mysql_global_status_innodb_history_list_length (from mysql). Provided solution gave me 3 graphs for a single host.

I used the following workaround:

(metric1{hostname="h"} or on() vector(0))+(metric2{hostname="h"} or on() vector(0))

taken from here: https://github.com/grafana/grafana/issues/2393#issuecomment-192522042

like image 26
ravnur Avatar answered Feb 20 '23 23:02

ravnur