Theano Shared Variables on Python

Question

I am now learning the Theano library, and I am just feeling confused about Theano shared variables. By reading the tutorial, I think I didn't understand its detailed meaning. The following is the definition of the Theano shared variables from the tutorial:

"Variable with Storage that is shared between functions that it appears in. These variables are meant to be created by registered shared constructors."

Also, I am wondering if the Theano shared variables can be a python class data member. For example:

class A(object):   
    data = None
    ...

Can "data" be or initialized as a Theano Shared variable? I really appreciate if anyone could help me.

Answer 1

Theano shared variables behave more like ordinary Python variables. They have an explicit value that is persistent. In contrast, symbolic variables are not given an explicit value until one is assigned on the execution of a compiled Theano function.

Symbolic variables can be thought of as representing state for the duration of a single execution. Shared variables on the other hand represent state that remains in memory for the lifetime of the Python reference (often similar to the lifetime of the program).

Shared variables are usually used to store/represent neural network weights because we want these values to remain around across many executions of a Theano training or testing function. Often, the purpose of a Theano training function is to update the weights stored in a shared variable. And a testing function needs the current weights to perform the network's forward pass.

As far as Python is concerned Theano variables (shared or symbolic) are just objects -- instances of classes defined within the Theano library. So, yes, references to shared variables can be stored in your own classes, just like any other Python object.