Tensorflow doesn't update weights

Question

I'm doing a tensoflow tutorial and I have some trouble on updating weights.

from sklearn.datasets import load_digits
mnist = load_digits(2)
X,y = mnist.data, mnist.target
print("y [shape - %s]:" % (str(y.shape)), y[:10])#y [shape - (360,)]: [0 1 0 1 0 1 0 0 1 1]
print("X [shape - %s]:" % (str(X.shape)))#X [shape - (360, 64)]:

# inputs and shareds
shared_weights = tf.Variable(initial_value=tf.random_uniform([64]))#<student.code_variable()>
input_X = tf.placeholder(shape=(None,64),dtype="float32",name="features")
input_y = tf.placeholder(shape=(None,),dtype="float32",name="label")

reduced_sum=tf.reduce_sum(input_X*shared_weights/256, axis=1)
predicted_y = tf.nn.sigmoid(reduced_sum)#<predicted probabilities for input_X>
loss =tf.losses.log_loss(labels=input_y,predictions=predicted_y)#<logistic loss (scalar, mean over sample)>
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(loss)

#I don't understand how i have to implement this (I don't use it)
train_function = lambda X,y: s.run(optimizer,feed_dict={input_X:X,input_y:y})
                              #<compile function that takes X and y, returns log loss and updates weights>
predict_function =lambda X: s.run(predicted_y,feed_dict={input_X:X})  
    #<compile function that takes X and computes probabilities of y>

from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y)

from sklearn.metrics import roc_auc_score
s.run(tf.global_variables_initializer())

for i in range(5):
    #print(reduced_sum.eval(feed_dict={input_X:X_train}))
    #print(shared_weights.eval())
    s.run(optimizer,feed_dict={input_X:X_train,input_y:y_train})#    <run optimizer operation>
    loss_i = loss.eval(feed_dict={input_X:X_train, input_y:y_train}) #<compute loss at iteration i>

    print("loss at iter %i:%.4f" % (i, loss_i))

    print("train auc:",roc_auc_score(y_train, predict_function(X_train)))
    print("test auc:",roc_auc_score(y_test, predict_function(X_test)))

print ("resulting weights:")
plt.imshow(shared_weights.get_value().reshape(8, -1))
plt.colorbar();

this part of the tutorial is guided with jolly to fill. The batch ( a single minibatch) optimization was forced. I printed loss and weights but they don't change, why?

Answer 1

Use a bigger step, for example 50.0, in your optimizer and more iterations, for example 50.

The resulting weights that points Sergey are correct.

And is not need to call s.run and loss.eval replace it by

r = train_function(X_train,y_train)
loss_i = r[0]
shared_weights = r[1]