Neural network for letter recognition

Question

I'm trying to add to the code for a single layer neural network which takes a bitmap as input and has 26 outputs for the likelihood of each letter in the alphabet.

The first question I have is regarding the single hidden layer that is being added. Am I correct in thinking that the hidden layer will have it's own set of output values and weights only? It doesn't need to have it's own bias'?

Can I also confirm that I'm thinking about the feedforward aspect correctly? Here's some pseudocode:

// input => hidden
for j in hiddenOutput.length:
    sum=inputs*hiddenWeights
    hiddenOutput[j] = activationFunction(sum)
// hidden => output
for j in output.length:
    sum=hiddenOutputs*weights
    output[j] = activationFunction(sum)

Assuming that is correct, would the training be something like this?

def train(input[], desired[]):
    iterate through output and determine errors[]
    update weights & bias accordingly
    iterate through hiddenOutput and determine hiddenErrors[]
    update hiddenWeights & (same bias?) accordingly

Thanks in advance for any help, I've read so many examples and tutorials and I'm still having trouble determining how to do everything correctly.

Answer 1

Dylan, this is probably long after your homework assignment was due, but I do have a few thoughts about what you've posted.

• Make the hidden layer much bigger than the size of the input bitmaps.
• You should have different weights and biases from input -> hidden than from hidden -> output.
• Spend a lot of time on your error function (discriminator).
• Understand that neural nets have a tendency to get quickly locked in to a set of weights (usually incorrect). You'll need to start over and train in a different order.

The thing I learned about neural nets is that you never know why they're working (or not working). That alone is reason to keep it out of the realms of medicine and finance.