Need help understanding function passing in Python

Question

I am trying to teach myself Python by working through some problems I came up with, and I need some help understanding how to pass functions.

Let's say I am trying to predict tomorrow's temperature based on today's and yesterday's temperature, and I have written the following function:

def predict_temp(temp_today, temp_yest, k1, k2):
    return k1*temp_today + k2*temp_yest

And I have also written an error function to compare a list of predicted temperatures with actual temperatures and return the mean absolute error:

def mean_abs_error(predictions, expected):
    return sum([abs(x - y) for (x,y) in zip(predictions,expected)]) / float(len(predictions))

Now if I have a list of daily temperatures for some interval in the past, I can see how my prediction function would have done with specific k1 and k2 parameters like this:

>>> past_temps = [41, 35, 37, 42, 48, 30, 39, 42, 33]
>>> pred_temps = [predict_temp(past_temps[i-1],past_temps[i-2],0.5,0.5) for i in xrange(2,len(past_temps))]
>>> print pred_temps
[38.0, 36.0, 39.5, 45.0, 39.0, 34.5, 40.5]
>>> print mean_abs_error(pred_temps, past_temps[2:])
6.5

But how do I design a function to minimize my parameters k1 and k2 of my predict_temp function given an error function and my past_temps data?

Specifically I would like to write a function minimize(args*) that takes a prediction function, an error function, some training data, and that uses some search/optimization method (gradient descent for example) to estimate and return the values of k1 and k2 that minimize my error given the data?

I am not asking how to implement the optimization method. Assume I can do that. Rather, I would just like to know how to pass my predict and error functions (and my data) to my minimize function, and how to tell my minimize function that it should optimize the parameters k1 and k2, so that my minimize function can automatically search a bunch of different settings of k1 and k2, applying my prediction function with those parameters each time to the data and computing error (like I did manually for k1=0.5 and k2=0.5 above) and then return the best results.

I would like to be able to pass these functions so I can easily swap in different prediction and error functions (differing by more than just parameter settings that is). Each prediction function might have a different number of free parameters.

My minimize function should look something like this, but I don't know how to proceed:

def minimize(prediction_function, which_args_to_optimize, error_function, data):
    # 1: guess initial parameters
    # 2: apply prediction function with current parameters to data to compute predictions
    # 3: use error function to compute error between predictions and data
    # 4: if stopping criterion is met, return parameters
    # 5: update parameters
    # 6: GOTO 2

Edit: It's that easy?? This is no fun. I am going back to Java.

On a more serious note, I think I was also getting hung up on how to use different prediction functions with different numbers of parameters to tune. If I just take all the free parameters in as one tuple I can keep the form of the function the same so it easy to pass and use.

Answer 1

Here is an example of how to pass a function into another function. apply_func_to will take a function f and a number num as parameters and return f(num).

def my_func(x):
    return x*x

def apply_func_to(f, num):
    return f(num)

>>>apply_func_to(my_func, 2)
4

If you wanna be clever you can use lambda (anonymous functions too). These allow you to pass functions "on the fly" without having to define them separately

>>>apply_func_to(lambda x:x*x, 3)
9

Hope this helps.

Answer 2

Function passing in Python is easy, you just use the name of the function as a variable which contains the function itself.

def predict(...):
    ...

minimize(predict, ..., mean_abs_error, ...)

As for the rest of the question: I'd suggest looking at the way SciPy implements this as a model. Basically, they have a function leastsq which minimizes the sum of the squares of the residuals (I presume you know what least-squares minimization is ;-). What you pass to leastsq is a function to compute the residuals, initial guesses for the parameters, and an arbitrary parameter which gets passed on to your residual-computing function (the closure), which includes the data:

# params will be an array of your k's, i.e. [k1, k2]
def residuals(params, measurements, times):
    return predict(params, times) - measurements

leastsq(residuals, initial_parameters, args = (measurements, times))

Note that SciPy doesn't actually concern itself with how you come up with the residuals. The measurements array is just passed unaltered to your residuals function.

I can look up an example I did recently if you want more information - or you can find examples online, of course, but in my experience they're not quite as clear. The particular bit of code I wrote would relate well to your scenario.