How to compute & plot Equal Error Rate (EER) from FAR/FRR values using matlab

Question

I have the following values against FAR/FRR. i want to compute EER rates and then plot in matlab.

FAR              FRR
19.64            20
21.29            18.61
24.92            17.08
19.14            20.28
17.99            21.39
16.83            23.47
15.35            26.39
13.20            29.17
7.92             42.92
3.96             60.56
1.82             84.31
1.65             98.33
26.07            16.39
29.04            13.13
34.49            9.31
40.76            6.81
50.33            5.42
66.83            1.67
82.51            0.28

Is there any matlab function available to do this. can somebody explain this to me. Thanks.

Answer 1

Let me try to answer your question

1) For your data EER can be the mean/max/min of [19.64,20]

1.1) The idea of EER is try to measure the system performance against another system (the lower the better) by finding the equal(if not equal then at least nearly equal or have the min distance) between False Alarm Rate (FAR) and False Reject Rate (FRR, or missing rate) .

Refer to your data, [19.64,20] gives min distance, thus it could used as EER, you can take mean/max/min value of these two value, however since it means to compare between systems, thus make sure other system use the same method(mean/max/min) to pick EER value.

The difference among mean/max/min can be ignored if the there are large amount of data. In some speaker verification task, there will be 100k data sample.

2) To understand EER ,better compute it by yourself, here is how:

two things you need to know:

A) The system score for each test case (trial)

B) The true/false for each trial

After you have A and B, then you can create [trial, score,true/false] pairs then sort it by the score value, after that loop through the score, eg from min-> max. At each loop assume threshold is that score and compute the FAR,FRR. After loop through the score find the FAR,FRR with "equal" value.

For the code you can refer to my pyeer.py , in function processDataTable2

https://github.com/StevenLOL/Research_speech_speaker_verification_nist_sre2010/blob/master/SRE2010/sid/pyeer.py

This function is written for the NIST SRE 2010 evaluation.

4) There are other measures similar to EER, such as minDCF which only play with the weights of FAR and FRR. You can refer to "Performance Measure" of http://www.nist.gov/itl/iad/mig/sre10results.cfm

5) You can also refer to this package https://sites.google.com/site/bosaristoolkit/ and DETware_v2.1.tar.gz at http://www.itl.nist.gov/iad/mig/tools/ for computing and plotting EER in Matlab

Plotting in DETWare_v2.1

Pmiss=1:50;Pfa=50:-1:1;
Plot_DET(Pmiss/100.0,Pfa/100.0,'r')