How to verify structure a neural network in keras model?

Question

I'm new in Keras and Neural Networks. I'm writing a thesis and trying to create a SimpleRNN in Keras as it is illustrated below:

As it is shown in the picture, I need to create a model with 4 inputs + 2 outputs and with any number of neurons in the hidden layer.

This is my code:

model = Sequential()

model.add(SimpleRNN(4, input_shape=(1, 4), activation='sigmoid', return_sequences=True))
model.add(Dense(2))
model.compile(loss='mean_absolute_error', optimizer='adam')
model.fit(data, target, epochs=5000, batch_size=1, verbose=2)

predict = model.predict(data)

1) Does my model implement the graph?
2) Is it possible to specify connections between neurons Input and Hidden layers or Output and Input layers?

Explanation:

I am going to use backpropagation to train my network. I have input and target values

Input is a 10*4 array and target is a 10*2 array which I then reshape:

input = input.reshape((10, 1, 4))
target = target.reshape((10, 1, 2))

It is crucial for to able to specify connections between neurons as they can be different. For instance, here you can have an example:

Answer 1

1) Not really. But I'm not sure about what exactly you want in that graph. (Let's see how Keras recurrent layers work below)

2) Yes, it's possible to connect every layer to every layer, but you can't use Sequential for that, you must use Model.

This answer may not be what you're looking for. What exactly do you want to achieve? What kind of data you have, what output you expect, what is the model supposed to do? etc...

---

1 - How does a recurrent layer work?

Documentation

Recurrent layers in keras work with an "input sequence" and may output a single result or a sequence result. It's recurrency is totally contained in it and doesn't interact with other layers.

You should have inputs with shape (NumberOrExamples, TimeStepsInSequence, DimensionOfEachStep). This means input_shape=(TimeSteps,Dimension).

The recurrent layer will work internally with each time step. The cycles happen from step to step and this behavior is totally invisible. The layer seems to work just like any other layer.

This doesn't seem to be what you want. Unless you have a "sequence" to input. The only way I know if using recurrent layers in Keras that is similar to you graph is when you have a segment of a sequence and want to predict the next step. If that's the case, see some examples by searching for "predicting the next element" in Google.

2 - How to connect layers using Model:

Instead of adding layers to a sequential model (which will always follow a straight line), start using the layers independently, starting from an input tensor:

from keras.layers import *
from keras.models import Model

inputTensor = Input(shapeOfYourInput) #it seems the shape is "(2,)", but we must see your data.    

#A dense layer with 2 outputs:
myDense = Dense(2, activation=ItsAGoodIdeaToUseAnActivation)


#The output tensor of that layer when you give it the input:
denseOut1 = myDense(inputTensor)

#You can do as many cycles as you want here:
denseOut2 = myDense(denseOut1)

#you can even make a loop:
denseOut = Activation(ItsAGoodIdeaToUseAnActivation)(inputTensor) #you may create a layer and call it with the input tensor in just one line if you're not going to reuse the layer
    #I'm applying this activation layer here because since we defined an activation for the dense layer and we're going to cycle it, it's not going to behave very well receiving huge values in the first pass and small values the next passes....
for i in range(n):
    denseOut = myDense(denseOut)

This kind of usage allows you to create any kind of model, with branches, alternative ways, connections from anywhere to anywhere, provided you respect the shape rules. For a cycle like that, inputs and outputs must have the same shape.

At the end, you must define a model from one or many inputs to one or many outputs (you must have training data to match all inputs and outputs you choose):

model = Model(inputTensor,denseOut)

But notice that this model is static. If you want to change the number of cycles, you will have to create a new model.

In this case, it would be as simple as repeating the loop step denseOut = myDense(denseOut) and creating another model2=Model(inputTensor,denseOut).

---

3 - Trying to create something like the image below:

I am supposing C and F will participate in all iterations. If not,

Since there are four actual inputs, and we are going to treat them all separately, let's create 4 inputs instead, all like (1,). Your input array should be divided in 4 arrays, all being (10,1).

from keras.models import Model
from keras.layers import *

inputA = Input((1,))
inputB = Input((1,))
inputC = Input((1,))
inputF = Input((1,))

Now the layers N2 and N3, that will be used only once, since C and F are constant:

outN2 = Dense(1)(inputC)
outN3 = Dense(1)(inputF)

Now the recurrent layer N1, without giving it the tensors yet:

layN1 = Dense(1)

For the loop, let's create outA and outB. They start as actual inputs and will be given to the layer N1, but in the loop they will be replaced

outA = inputA   
outB = inputB

Now in the loop, let's do the "passes":

for i in range(n):
    #unite A and B in one 
    inputAB = Concatenate()([outA,outB])

    #pass through N1
    outN1 = layN1(inputAB)

    #sum results of N1 and N2 into A
    outA = Add()([outN1,outN2])

    #this is constant for all the passes except the first
    outB = outN3 #looks like B is never changing in your image.... 

Now the model:

finalOut = Concatenate()([outA,outB])
model = Model([inputA,inputB,inputC,inputF], finalOut)