Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Is tf.layers.dense a single layer?

If I just use a single layer like this:

layer = tf.layers.dense(tf_x, 1, tf.nn.relu)

Is this just a single layer with a single node?

Or is it actually a set of layers (input, hidden, output) with 1 node? My network seemed to work properly with just 1 layer, so I was curious about the setup.

Consequently, does this setup below have 2 hidden layers (are layer1 and layer2 here both hidden layers)? Or actually just 1 (just layer 1)?

layer1 = tf.layers.dense(tf_x, 10, tf.nn.relu)
layer2 = tf.layers.dense(layer1, 1, tf.nn.relu)

tf_x is my input features tensor.

like image 977
sandboxj Avatar asked Aug 15 '17 12:08

sandboxj


People also ask

What is the layers dense in TensorFlow?

This function is used to create fully connected layers, in which every output depends on every input. Syntax: tf.layers.dense(args) Parameters: This function takes the args object as a parameter which can have the following properties: units: It is a positive number that defines the dimensionality of the output space.

What is dense layer in deep learning?

Dense layer, also called fully-connected layer, refers to the layer whose inside neurons connect to every neuron in the preceding layer.

Is dense layer same as linear layer?

Yes, it is the same. model. add (Dense(10, activation = None)) or nn. linear(128, 10) is the same, because it is not activated in both, therefore if you don't specify anything, no activation is applied.

How many dense layers do I need?

It's depend more on number of classes. For 20 classes 2 layers 512 should be more then enough. If you want to experiment you can try also 2 x 256 and 2 x 1024. Less then 256 may work too, but you may underutilize power of previous conv layers.


2 Answers

tf.layers.dense adds a single layer to your network. The second argument is the number of neurons/nodes of the layer. For example:

# no hidden layers, dimension output layer = 1
output = tf.layers.dense(tf_x, 1, tf.nn.relu)

# one hidden layer, dimension hidden layer = 10,  dimension output layer = 1
hidden = tf.layers.dense(tf_x, 10, tf.nn.relu)
output = tf.layers.dense(hidden, 1, tf.nn.relu)

My network seemed to work properly with just 1 layer, so I was curious about the setup.

That is possible, for some tasks you will get decent results without hidden layers.

like image 129
GeertH Avatar answered Oct 13 '22 02:10

GeertH


tf.layers.dense (tf.compat.v1.layers.dense) is only one layer with a amount of nodes. You can check on TensorFlow web site about tf.layers.dense (tf.compat.v1.layers.dense)

layer1 = tf.layers.dense(inputs=pool2_flat, units=1024, activation=tf.nn.relu)
layer2 = tf.layers.dense(inputs=layer1, units=1024, activation=tf.nn.relu)
like image 33
Christian Frei Avatar answered Oct 13 '22 00:10

Christian Frei