neural networks regression using pybrain

Question

I need to solve a regression problem with a feed forward network and I've been trying to use PyBrain to do it. Since there are no examples of regression on pybrain's reference, I tried to adapt it's classification example for regression instead, but with no success (The classification example can be found here: http://pybrain.org/docs/tutorial/fnn.html). Following is my code:

This first function converts my data in numpy array form to a pybrain SupervisedDataset. I use the SupervisedDataset because according to pybrain's reference it is the dataset to use when the problem is regression. The parameters are an array with the feature vectors (data) and their expected output (values):

def convertDataNeuralNetwork(data, values):

fulldata = SupervisedDataSet(data.shape[1], 1)

for d, v in zip(data, values):

    fulldata.addSample(d, v)    

return fulldata

Next, is the function to run the regression. train_data and train_values are the train feature vectors and their expected output, test_data and test_values are the test feature vectors and their expected output:

regressionTrain = convertDataNeuralNetwork(train_data, train_values)

regressionTest = convertDataNeuralNetwork(test_data, test_values)

fnn = FeedForwardNetwork()

inLayer = LinearLayer(regressionTrain.indim)
hiddenLayer = LinearLayer(5)
outLayer = GaussianLayer(regressionTrain.outdim)

fnn.addInputModule(inLayer)
fnn.addModule(hiddenLayer)
fnn.addOutputModule(outLayer)

in_to_hidden = FullConnection(inLayer, hiddenLayer)
hidden_to_out = FullConnection(hiddenLayer, outLayer)

fnn.addConnection(in_to_hidden)
fnn.addConnection(hidden_to_out)

fnn.sortModules()

trainer = BackpropTrainer(fnn, dataset=regressionTrain, momentum=0.1, verbose=True, weightdecay=0.01)

for i in range(10):

    trainer.trainEpochs(5)

    res = trainer.testOnClassData(dataset=regressionTest )

    print res

when I print res, all it's values are 0. I've tried to use the buildNetwork function as a shortcut to build the network, but it didn't work as well. I've also tried different kinds of layers and different number of nodes in the hidden layer, with no luck.

Does somebody have any idea of what I am doing wrong? Also, some pybrain regression examples would really help! I couldn't find any when I looked.

Thanks in advance

Answer 1

pybrain.tools.neuralnets.NNregression is a tool which

Learns to numerically predict the targets of a set of data, with optional online progress plots.

so it seems like something well suited for constructing a neural network for your regression task.