Why is neural network not predicting?

Question

I am trying to train a neural network using backpropagation algo. in OpenCV 2.3. However it is not predicting correctly....not even on training dataset. Could anybody please help me find whats wrong here?

training_feature_matrix - Nx69 matrix of float values

training_age_matrix - Nx4 matrix of float values

test_feature_matrix - Mx69 matrix of float values

test_age_matrix - Mx4 matrix of float values

the feature matrices (mentioned above) are like: [0.123435, 0.4542665, 0.587545, ...68-such values + last value '1.0 or 2.0' depending upon its male/female)

the age-matrices (mentioned above) are like: [1, 0, 0 ,0; 1, 0, 0, 0; 0, 1, 0, 0; ...] here 1s show the class of age (baby, child, adult, old) the corresponding row of feature matrix belongs to.

here is the code: I call 'mlp' function using above matrices as parameters)

cv::Mat mlp(cv::Mat& training_feature_matrix, cv::Mat& training_age_matrix, cv::Mat& test_feature_matrix, cv::Mat& test_age_matrix)
{
cv::Mat layers = cv::Mat(3, 1, CV_32SC1);
layers.row(0)  = cv::Scalar(69);
layers.row(1)  = cv::Scalar(36);
layers.row(2)  = cv::Scalar(4);    //   cout<<layers<<"\n";

CvANN_MLP ann;
CvANN_MLP_TrainParams params;
CvTermCriteria criteria;
criteria.max_iter = 10000;
criteria.epsilon  = 0.001;
criteria.type     = CV_TERMCRIT_ITER + CV_TERMCRIT_EPS;
params.train_method = CvANN_MLP_TrainParams::BACKPROP;
params.bp_dw_scale  = 0.1;
params.bp_moment_scale = 0.1;
params.term_crit  = criteria;

ann.create(layers, CvANN_MLP::SIGMOID_SYM);
ann.train(training_feature_matrix, training_age_matrix, cv::Mat(), cv::Mat(), params);

cv::Mat predicted(test_age_matrix.rows, 4, CV_32SC1);
for(int i = 0; i < test_feature_matrix.rows; i++)
{
  cv::Mat response(1, 4, CV_32F);
  cv::Mat sample = test_feature_matrix.row(i);
  ann.predict(sample, response);
  for (int g = 0; g < 4; g++)
  {
    predicted.at<int>(i,g) = response.at<float>(0,g);
  } 
}
   cout << "\n";
   cout << ann.get_weights(0) << "\n";
   cout << ann.get_layer_sizes() << "\n";
   cout << ann.get_layer_count() << "\n\n";

return predicted;
}

EDIT Also, the ann.get_weights(0) & ann.get_layer_sizes() are returning garbage values but ann.get_layer_count() is returning correct value 3.

Thanks :)

Answer 1

It has been long since that question asked but I will share the answer. I had a similar problem with sigmoid's output values. It is resolved now. You can check my issue here :

OpenCV Neural Network Sigmoid Output

To summarize the error, it is occuring because of the default parameters of mlp's create function. Use like this : ann.create(layers, CvANN_MLP::SIGMOID_SYM, 1, 1).

Answer 2

Back propagation does not always converge. It is quite likely to blow up and produce nonsense. This is likely if the epsilon or momentum_scale values are too large. Your momentum looks to be at the top end of what might work and I would try reducing it.