Can a neural network be used to find a functions minimum(a)?

Question

I had been interested in neural networks for a bit and thought about using one in python for a light project that compares various minimization techniques in a time domain (which is fastest).

Then I realized I didn't even know if a NN is good for minimization. What do you think?

Answer 1

Although this comes a bit too late for the the author of this question. Maybe somebody wants to test some optimization algorithms, when he reads this...

If you are working with regressions in machine learning (NN, SVM, Multiple Linear Regression, K Nearest Neighbor) and you want to minimize (maximize) your regression-function, actually this is possible but the efficiency of such algorithms depends on smootheness, (step-size... etc.) of the region you are searching in.

In order to construct such "Machine Learning Regressions" you could use scikit- learn. You have to train and validate your MLR Support Vector Regression. ("fit" method)

SVR.fit(Sm_Data_X,Sm_Data_y)

Then you have to define a function which returns a prediction of your regression for an array "x".

def fun(x):
    return SVR.predict(x)

You can use scipiy.optimize.minimize for optimization. See the examples following the doc-links.