Why does my MATLAB Neural Network train slower when using parallel processing?

Question

If I use the function:

net=feedfowardnet([60 60])

net2=train(net,x,t)

It takes about 20 minuets to train. (I have done this on multiple computers {with the same specs}, and the average time is always around 20 mins)

If I use the function:

parpool  %//starts a local parallel pool connected to 2 workers
net2=train(net,x,t,'useParallel','yes')

It takes around 40 minuets to complete training. I have two cores, so this is counter intuitive, it should be twice as fast, not twice as slow. I am using the same starting network, and the same training inputs and targets.

Also, when I open the task manager during NN training, it shows that both CPUs are working at 100%, even when parpool and useParallel are turned off.

This page of the Mathworks website says that "Parallel Computing Toolbox™ allows Neural Network Toolbox™ to simulate and train networks faster and on larger datasets than can fit on one PC. Parallel training is currently supported for backpropagation training only, not for self-organizing maps."

I am using 2000 training examples in the data set. There are 32 inputs and 3 outputs so this is definitely a large data set. The parallel pool is also definitely turned off when I just use the net2=train(net,x,t) function.

I have tested the use of parpool with other functions, (ones containing parfor loops), and the calculation is usually twice as fast. It is just seems to be the neural network training that goes slower.

Is there any reason for this?

I am using an Intel Core 2 Duo E8400 Cpu @3GHz, and I am using MATLAB version R2013 b. I am also using a computer on a network (inside a university). I'm not sure if this makes a difference.

---

More info on the university computer network. I am using multiple computers on the network at the same time. I have not connected them together in a way to do distributed computing, each one is just doing its own thing using parallel computing on its own 2 processors. However I am not sure if the computers are still interfering with each other in some way because they are logged on with the same user. I load the training input and target data into the matlab workspace on each computer using:

load('H:\18-03-14\x.mat')
load('H:\18-03-14\net.mat')
load('H:\18-03-14\t.mat')

Where H: is the network drive. I am not sure if once these are in the matlab workspace, they are still somehow connected, and interfere with each other across different computers. Do they?

Answer 1

There are two types of parallelism going on: multicore and multithread. MATLAB implements basic matrix operations with multicore support, so even without the parallel computing toolbox, all your cores are being utilized.

The Parallel Computing Toolbox additionally allows for multiple threads. Multiple threads have two advantages. The first is that they allow calculations to be threaded across multiple PCs using MATLAB Distributed Computing Server for a reliable linear speed up. The second is less obvious, which is that multiple threads on a single PC can improve on already single-threaded multicore calculations, even though that might seem counter intuitive. However, there is extra overhead for using parallel threads so this is not always the case. More cores and large problems are more likely to see a speed up.

Your problem is not a large one. A large problem would be 10's of thousands of examples or more, whereas your problem only has 2000.

Also, two hidden layers with 60 neurons each is almost certainly a far larger network than you need. Your problem has 2000 samples * 3 outputs = 6000 constraints. Your network has 32*60+60*60+60*3 weights and 32+60+3 biases for a total of 5795 adjustable variables which is almost as many as the number of constraints. I would suggest far fewer weights and probably only a single hidden layer. This will train much faster, and is also likely to generalize better.

So maybe start with feedforwardnet(100) and then increase that if the desired accuracy is not found.

You can see the benefits of multiple threads on a larger problem using this Neural Network Toolbox example dataset with 68,308 examples:

[x,t] = vinyl_dataset;
net = feedforwardnet(140,'trainscg');
rng(0), tic, net2 = train(net,x,t); toc
parpool
rng(0), tic, net2 = train(net,x,t,'useParallel','yes'); toc