How to use "top" and "bottom" parameters to build network architecture

Question

In Caffe prototxt, every layer includes either "top" or "bottom" parameter to specify connections between layers. There are sometimes, however, cases when, for example, the "top" is the layer itself (why do we have to link it to itself?) or there are several "top" layers. What is the exact meaning of "top" and "bottom" parameters and the rules behind using them?

Answer 1

In Caffe:

1. The names of the links between layers (top/bottom parameters) are very important.
2. Outward links from a layer are "top" and Incoming links into a layer are "bottom". So, a top from one layer connects as bottom into another layer. Sort of like a highway (top/bottom) between two towns (layers). Caffe determines the structure of your network from the names of your top/bottom links.
3. The names of the layers themselves are far less important, and do not carry structural information. You only need these to be sensible and unique. The same highway carries traffic between two towns regardless of how they are named.
4. The namespace of Layers and top/bottoms are separate. So, you can name a layer the same as a top or bottom. This doesn't mean anything. But it is confusing and should be avoided.

Answer 2

There is a confusion here between layers and blobs.

In Caffe, all data is represented in the form of blobs. Each layer takes in zero or more blobs, transforms them, and sends out zero or more blobs. For example, a ReLU layer accepts a single blob with the data, applies the function f(x) = x if x>0, 0 otherwise, and outputs the result as a single blob. A data layer for classification problems usually has two output blobs, one for the data and the other for the labels, and no input blob.

The blobs are visualized as if they move through the network from the bottom to the top. So, the input blob is called the bottom blob and the output blob is called the top blob.

Now, in the prototxt definition, the name attribute stores the name of the layer. The bottom attribute stores the name of the input blob. The top attribute stores the name of the output blob, which for convenience, is generally taken to be the same as the name of the layer. If there are multiple input blobs for that layer, there are multiple bottom attributes, and if there are multiple output blobs, there are multiple top attributes.