Tensorflow: is it possible to create 2D LSTM?

Question

I have data that consists of 4 different time series, e.g.:

    [35, 45, 47, 39...]
    [47, 60, 57, 55...]
    [42, 42, 61, 69...]
    [62, 70, 62, 65...]

Thing is, besides temporal dependency (horizontal one), there also exists vertical dependency (in columns, if we look at this example 'matrix').

Output vectors would be these same time series, only shifted for one step.

Is it possible to create LSTM network for each of time series (so, 4 networks in my case, and also 4 outputs) but also connect them vertically, i.e. create 2D LSTM?

If so, how would one achieve that in Tensorflow?

Is it also possible to make this kind of network deeper (have additional LSTM layers appended to each of these 4 networks)?

I hope I was clear enough with explanation.

Answer 1

One solution is to use Multi-dimensional RNN or LSTM as described in https://arxiv.org/pdf/0705.2011.pdf. In this case, your data will be treated as a sequence with 4 dimensions. This github repo provides an implementation of 2D LSTM https://github.com/philipperemy/tensorflow-multi-dimensional-lstm. Hope this helps