Regression with lasagne: error

Question

I am trying to run a regression with lasagne/nolearn. I am having trouble finding documentation how to do that (new to deep learning in general).

Starting off with a simple network (one hidden layer)

from lasagne import layers

from lasagne.nonlinearities import softmax
from lasagne.updates import nesterov_momentum
from nolearn.lasagne import NeuralNet    

print(np.shape(X)) # (137, 43)
print(np.shape(y)) # (137,)

layers_s = [('input', layers.InputLayer),
           ('dense0', layers.DenseLayer),
           ('output', layers.DenseLayer)]

net_s = NeuralNet(layers=layers_s,

                 input_shape=(None, num_features),
                 dense0_num_units=43,
                 output_num_units=1,
                 output_nonlinearity=None,

                 regression=True,

                 update=nesterov_momentum,
                 update_learning_rate=0.001,
                 update_momentum=0.9,

                 eval_size=0.2,
                 verbose=1,
                 max_epochs=100)

net_s.fit(X, y)

I get the following error:

TypeError                                 Traceback (most recent call last)
<ipython-input-23-23c15ceec104> in <module>()
----> 1 net_s.fit(X, y)

/home/alex/anaconda3/lib/python3.4/site-packages/nolearn/lasagne.py in fit(self, X, y)
    148             out, self.loss, self.update,
    149             self.X_tensor_type,
--> 150             self.y_tensor_type,
    151             )
    152         self.train_iter_, self.eval_iter_, self.predict_iter_ = iter_funcs

/home/alex/anaconda3/lib/python3.4/site-packages/nolearn/lasagne.py in _create_iter_funcs(self, output_layer, loss_func, update, input_type, output_type)
    298         all_params = get_all_params(output_layer)
    299         update_params = self._get_params_for('update')
--> 300         updates = update(loss_train, all_params, **update_params)
    301 
    302         train_iter = theano.function(

/home/alex/src/lasagne/lasagne/updates.py in nesterov_momentum(loss, all_params, learning_rate, momentum)
     38 # such that the gradient can be evaluated at the current parameters.
     39 def nesterov_momentum(loss, all_params, learning_rate, momentum=0.9):
---> 40     all_grads = theano.grad(loss, all_params)
     41     updates = []
     42 

/home/alex/anaconda3/lib/python3.4/site-packages/theano/gradient.py in grad(cost, wrt, consider_constant, disconnected_inputs, add_names, known_grads, return_disconnected)
    431 
    432     if cost is not None and cost.ndim != 0:
--> 433         raise TypeError("cost must be a scalar.")
    434 
    435     if isinstance(wrt, set):

TypeError: cost must be a scalar.

Thanks!..

Answer 1

Make sure that you're using versions of nolearn and Lasagne that are known to work together.

Say you've been following the Using convolutional neural nets to detect facial keypoints tutorial. Then the right thing to do is to install the dependencies from this requirements.txt file, like so:

pip uninstall Lasagne
pip uninstall nolearn
pip install -r https://raw.githubusercontent.com/dnouri/kfkd-tutorial/master/requirements.txt

If, however, you're using nolearn from Git master, then make sure you install the Lasagne version that's in the requirements.txt file found there:

pip uninstall Lasagne
pip install -r https://raw.githubusercontent.com/dnouri/nolearn/master/requirements.txt

Answer 2

Not sure what version of nolearn and lasagne you are using. I did notice that you have y as being of shape (137,). From my usage this needs to be (137, 1) to work for your case, and, in general, dim 2 needs to match the output_num_units.

Try y.reshape((-1, 1)).

If this doesn't work it may be a Python 3 compatibility issue.