Dead simple example of synaptic js lstm rnn algorithm

Question

It's pretty crazy that there isn't a dead simple example of the LSTM RNN predicting time series data.

https://github.com/cazala/synaptic

https://github.com/cazala/synaptic/wiki/Architect#lstm

I'd like to use the historical data in the following array:

const array = [
    0,
    0,
    0,
    1,
    0,
    0,
    0,
    1
];

Some pretty mind blowing data right there right?

I'd like to A) train the algorithm with the array then B) test the algorithm with the following array:

const array = [
    0,
    0,
    0,
    1,
    0,
    0,
    0,
    1,
    0
];

Should result in it predicting a 0.

Unfortunately the documentation is pretty bad, no clear code examples exist. Anyone have any examples?

Answer 1

This answer is not written with Synaptic, but with Neataptic. I decided to make a quick answer that I will include in the documentation soon. This is the code, it works 9/10 times:

var network = new neataptic.architect.LSTM(1,6,1);

// when the timeseries is [0,0,0,1,0,0,0,1...]
var trainingData = [
  { input: [0], output: [0] },
  { input: [0], output: [0] },
  { input: [0], output: [1] },
  { input: [1], output: [0] },
  { input: [0], output: [0] },
  { input: [0], output: [0] },
  { input: [0], output: [1] },
];

network.train(trainingData, {
  log: 500,
  iterations: 6000,
  error: 0.03,
  clear: true,
  rate: 0.05,
});

Run it on JSFIDDLE to see the prediction! For more predictions, open this one.

Explanation to some choices I made:

• I set option clear to true, as you want do a chronological timeseries prediction. This makes sure that the network starts from the 'beginning' every training iteration, instead of continuing on from the 'end' of the last iteration.
• Rate is fairly low, higher rates will get stuck at an MSE error of ~0.2
• The LSTM has 1 block of 6 memory nodes, lower amounts don't seem to work as well.