Menu
I looked around online but couldn't find anything, but I may well have missed a piece of literature on this. I am running a basic neural net on a 289 component vector to produce a 285 component vector. In my input, the last 4 pieces of data are critical to change the rest of the input into the resultant 285 for the output. That is to say, the input is 285 + 4, such that the 4 morph the rest of the input into the output.
But when running a neural network on this, I am not sure how to reflect this. Would I need to use convolution on the rest of the input? I want my system to emphasize the 4 data points that critically affect the other 285. I am still new to all of this, so a few pointers would be great!
Again, if there is something already written on this, then that would be awesome too.
549
The very most disadvantage of a neural network is its black box nature. Because it has the ability to approximate any function, study its structure but don't give any insights on the structure of the function being approximated.
There are some techniques to avoid overfitting: Regularisation of data (L1 or L2). Dropouts — Randomly dropping connections between neurons, forcing the network to find new paths and generalise. Early Stopping — Precipitates the training of the neural network, leading to reduction in error in the test set.
Due to its mathematical complexity, the theoretical foundations of neural networks are not covered. However, the universal approximation theorem (and the tools used in its proof) gives a profound insight into why neural networks are so powerful. It even lays the groundwork for engineering novel architectures.
I don't think you have any reason doing this since the network will infer that on its own. The weights will be reduced or enhanced for each input according to their importance considering the output.
What you could do though, is to have a preliminary network that is going to have the 285 component as an input, and then a new network that is going to have the 4 critical components and the output of the preliminary network as an input.
[285 compo.]---[neural network]---+---[neural network]---[output 285 compo.]
|
[4 compo.]-+
For instance, you could treat a picture with convolution networks and then add some meta information later in a fully connected network to process everything.
197
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
