How can I deal with a randomization issue in Echo State Networks?

Question

I am using Echo State Networks(ESN) as a Q-function in a Reinforcement Learning task. I have managed to achieve high accuracy, 90% in average, on the test phase with particular reservoir topology (spectral radius =0.9, regularization coefficient = 10, #input unit = 2, #output units = 1, #reservoir units = 8, and no leaking rate).

The system achieved high accuracy in test phase after training for 100 episodes. But when I initialized the networks weights with different random seeds, it's behavior became very unstable and failed to achieve high performance as before. I want to know how can I overcome this randomness issue and have ESN that is robust to the different random initialization of its input and reservoir weight and can generalized well ?

here how I initialized my network.  Input weights and reservoir weights sampled from Normal Distribution (mean = 0 and std = 1). Input weight matrix is normalized with unified variance and the reservoir weights normalized by division with maximum of absolute Eigen values and multiplied with spectral radius.  

Thanks In Advance

Ramin

Answer 1

I tend to agree with your comment: your reservoir needs more neurons in order to increase the probability of capturing the right dynamics. However, regarding your second question, I'd say the principle is not very much different from conventional feedforward NNs, you will need empirical parameter searching. More specifically, for ESNs, I do following:

• If your ESN works for some random seeds but not for others, increase the size of the reservoir
• If your ESN is overfitting (high training performance and low test performance), increase the regularization parameter (which, BTW, I think is already pretty high for your NN, but it depends on your dataset)
• If your ESN is "forgetting" too fast, increase the spectral radius. Note that even a value greater than 1 is acceptable in some cases. If the spectral radius is already too high and you still get a very short-term memory, include a leaking rate
• Note that you can also tweak the sparseness of your reservoir weights (more sparseness = less chaotic behavior)