TensorFlow: numpy.repeat() alternative

Question

I want to compare the predicted values yp from my neural network in a pairwise fashion, and so I was using (back in my old numpy implementation):

idx = np.repeat(np.arange(len(yp)), len(yp))
jdx = np.tile(np.arange(len(yp)), len(yp))
s = yp[[idx]] - yp[[jdx]]

This basically create a indexing mesh which I then use. idx=[0,0,0,1,1,1,...] while jdx=[0,1,2,0,1,2...]. I do not know if there is a simpler manner of doing it...

Anyhow, TensorFlow has a tf.tile(), but it seems to be lacking a tf.repeat().

idx = np.repeat(np.arange(n), n)
v2 = v[idx]

And I get the error:

TypeError: Bad slice index [  0   0   0 ..., 215 215 215] of type <type 'numpy.ndarray'>

It also does not work to use a TensorFlow constant for the indexing:

idx = tf.constant(np.repeat(np.arange(n), n))
v2 = v[idx]

-

TypeError: Bad slice index Tensor("Const:0", shape=TensorShape([Dimension(46656)]), dtype=int64) of type <class 'tensorflow.python.framework.ops.Tensor'>

The idea is to convert my RankNet implementation to TensorFlow.

Answer 1

You can achieve the effect of np.repeat() using a combination of tf.tile() and tf.reshape():

idx = tf.range(len(yp))
idx = tf.reshape(idx, [-1, 1])    # Convert to a len(yp) x 1 matrix.
idx = tf.tile(idx, [1, len(yp)])  # Create multiple columns.
idx = tf.reshape(idx, [-1])       # Convert back to a vector.

You can simply compute jdx using tf.tile():

jdx = tf.range(len(yp))
jdx = tf.tile(jdx, [len(yp)])

For the indexing, you could try using tf.gather() to extract non-contiguous slices from the yp tensor:

s = tf.gather(yp, idx) - tf.gather(yp, jdx)

Answer 2

According to tf api document, tf.keras.backend.repeat_elements() does the same work with np.repeat() . For example,

x = tf.constant([1, 3, 3, 1], dtype=tf.float32)
rep_x = tf.keras.backend.repeat_elements(x, 5, axis=0)
# result: [1. 1. 1. 1. 1. 3. 3. 3. 3. 3. 3. 3. 3. 3. 3. 1. 1. 1. 1. 1.]

Answer 3

Just for 1-d tensors, I've made this function

def tf_repeat(y,repeat_num):   
        return tf.reshape(tf.tile(tf.expand_dims(y,axis=-1),[1,repeat_num]),[-1]) 

Answer 4

It looks like your question is so popular that people refer it on TF tracker. Sadly the same function is not still implemented in TF.

You can implement it by combining tf.tile, tf.reshape, tf.squeeze. Here is a way to convert examples from np.repeat:

import numpy as np
import tensorflow as tf

x = [[1,2],[3,4]]
print np.repeat(3, 4)
print np.repeat(x, 2)
print np.repeat(x, 3, axis=1)

x = tf.constant([[1,2],[3,4]])
with tf.Session() as sess:
    print sess.run(tf.tile([3], [4]))
    print sess.run(tf.squeeze(tf.reshape(tf.tile(tf.reshape(x, (-1, 1)), (1, 2)), (1, -1))))
    print sess.run(tf.reshape(tf.tile(tf.reshape(x, (-1, 1)), (1, 3)), (2, -1)))

In the last case where repeats are different for each element you most probably will need loops.