Menu
I am following the PyTorch tutorial here. It says that
x = torch.randn(3, requires_grad=True)
y = x * 2
while y.data.norm() < 1000:
y = y * 2
print(y)
Out:
tensor([-590.4467, 97.6760, 921.0221])
Could someone explain what data.norm() does here?
When I change .randn to .ones its output is tensor([ 1024., 1024., 1024.]).
382
norm() method computes a vector or matrix norm. Norm is always a non-negative real number which is a measure of the magnitude of the matrix. It accepts a vector or matrix or batch of matrices as the input. It supports inputs of only float, double, cfloat, and cdouble dtypes.
For complex inputs, the norm is calculated using the absolute value of each element. If the input is complex and neither dtype nor out is specified, the result's data type will be the corresponding floating point type (e.g. float if input is complexfloat).
The L2 norm is calculated as the square root of the sum of the squared vector values."
Thus, the Euclidean norm of a vector which is a point on a line, surface, or hypersurface may be interpreted geometrically as the distance between this point and the origin.
PyTorch norm | How to use PyTorch norm? | What is PyTorch norm? PyTorch provides the different types of functionality to the user, in which that norm is one the functionality that is provided by the PyTorch. Basically in deep learning sometimes we need to fetch the matrix or vector from the input tensor.
Returns the matrix norm or vector norm of a given tensor. torch.norm is deprecated and may be removed in a future PyTorch release. Its documentation and behavior may be incorrect, and it is no longer actively maintained.
torch.norm is deprecated and may be removed in a future PyTorch release. Use torch.linalg.norm (), instead, or torch.linalg.vector_norm () when computing vector norms and torch.linalg.matrix_norm () when computing matrix norms. Note, however, the signature for these functions is slightly different than the signature for torch.norm.
Here, we can obtain the mean and standard deviation values by simply using the corresponding PyTorch tensor methods. The hard way is hard because we need to manually implement the formulas for the mean and standard deviation and iterate over smaller batches of the dataset.
It's simply the L2 norm (a.k.a Euclidean norm) of the tensor. Below is a reproducible illustration:
In [15]: x = torch.randn(3, requires_grad=True)
In [16]: y = x * 2
In [17]: y.data
Out[17]: tensor([-1.2510, -0.6302, 1.2898])
In [18]: y.data.norm()
Out[18]: tensor(1.9041)
# computing the norm using elementary operations
In [19]: torch.sqrt(torch.sum(torch.pow(y, 2)))
Out[19]: tensor(1.9041)
Explanation: First, it takes a square of every element in the input tensor x, then it sums them together, and finally it takes a square root of the resulting sum. All in all, these operations compute the so-called L2 or Euclidean norm.
133
Building on what @kmario23 says, the code multiplies the elements of a vector by 2 until the Euclidean magnitude (distance from origin) / L2 norm of the vector is at least 1000.
With the example of the vector with (1,1,1): it increases to (512, 512, 512), where the l2 norm is about 886. This is less than 1000, so it gets multiplied by 2 again and becomes (1024, 1024, 1024). This has a magnitude greater than 1000, so it stops.
38
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
