How to plot gradient vector on contour plot in python

Question

I have a loss function of two variables W1, W2 and an output z = F(W1,W2).

Now I plot the contour map of this loss function. Now say, I have calculated gradient vector at two points, therefore I have two gradient vectors now. I want to plot these gradient vector on my contour plot but I have no idea how to procces. Any help is appreciated

enter code here
%matplotlib inline
import matplotlib.pyplot as plt 
import numpy as np 

feature_x = np.arange(-50, 50, 2) 
feature_y = np.arange(-50, 50, 3) 

# Creating 2-D grid of features 
[X, Y] = np.meshgrid(feature_x, feature_y) 

fig, ax = plt.subplots(1, 1) 

z = 0.5*np.array((Y-X)*(Y-X) + 0.5*(1-X)*(1-X))

# plots contour lines 
ax.contour(X, Y, z, 10, cmap = 'jet') 
ax.grid(True)
ax.axis('scaled')
#ax.clabel(cp, inline=1, fontsize=10)  
ax.set_title('Contour Plot') 
ax.set_xlabel('feature_x') 
ax.set_ylabel('feature_y') 

plt.show() 

Answer 1

You could use FancyArrowPatch to draw the gradients at a few selected positions.

from matplotlib.patches import FancyArrowPatch
x1 = -20     # position of the gradient
y1 = 10
dz1_dx = 10  # value of the gradient at that position
dz1_dy = -5
arrow = FancyArrowPatch((x1, y1), (x1+dz1_dx, y1+dz1_dy),    
                        arrowstyle='simple', color='k', mutation_scale=10)
ax.add_patch(arrow)

Otherwise if you want to plot the whole vector field quiver might be an option:

feature_x = np.arange(-50, 50, 2)
feature_y = np.arange(-50, 50, 2)

x, y = np.meshgrid(feature_x, feature_y)
z = 0.5*(y-x)**2 + 0.5*(1-x)**2
u = 2*x - y - 1
v = y - x

# Normalize all gradients to focus on the direction not the magnitude
norm = np.linalg.norm(np.array((u, v)), axis=0)
u = u / norm
v = v / norm

fig, ax = plt.subplots(1, 1)
ax.set_aspect(1)
ax.plot(feature_x, feature_y, c='k')
ax.quiver(x, y, u, v, units='xy', scale=0.5, color='gray')
ax.contour(x, y, z, 10, cmap='jet', lw=2)

arrow = FancyArrowPatch((35, 35), (35+34*0.2, 35+0), arrowstyle='simple',
                        color='r', mutation_scale=10)  
ax.add_patch(arrow)  # NOTE: this gradient is scaled to make it better visible

I added the line y = x in this plot and marked the point where this lines intersects with a contour line. Here you can see clearly that:

Gradients are orthogonal to level surfaces

So for your point (80, 80) the gradient (79, 0) is correct even so the general shape of isolines maybe suggest that there should be be a part in y-direction. But if you look along the line y=x you see that the gradients there are always only in x-direction.