Advanced Custom activation function in keras + tensorflow

Question

def newactivation(x):
    if x>0:
        return K.relu(x, alpha=0, max_value=None)
    else :
        return x * K.sigmoid(0.7* x)

get_custom_objects().update({'newactivation': Activation(newactivation)})

I am trying to use this activation function for my model in keras, but I am having hard time by finding what to replace

if x>0:

ERROR i got:

File "/usr/local/lib/python3.4/dist-packages/tensorflow/python/framework/ops.py", line 614, in bool raise TypeError("Using a tf.Tensor as a Python bool is not allowed. "

TypeError: Using a tf.Tensor as a Python bool is not allowed. Use if >t is not None: instead of if t: to test if a tensor is defined, and >use TensorFlow ops such as tf.cond to execute subgraphs conditioned on >the value of a tensor.

Can someone make it clear for me?

Answer 1

if x > 0 doesn't make sense because x > 0 is a tensor, and not a boolean value.

To do a conditional statement in Keras use keras.backend.switch.

For example your

if x > 0:
   return t1
else:
   return t2

Would become

keras.backend.switch(x > 0, t1, t2)

Answer 2

Try something like:

def newactivation(x):
    return tf.cond(x>0, x, x * tf.sigmoid(0.7* x))

x isn't a python variable, it's a Tensor that will hold a value when the model is run. The value of x is only known when that op is evaluated, so the condition needs to be evaluated by TensorFlow (or Keras).