How to rescale a pytorch tensor to interval [0,1]?

Question

I would like to rescale a pytorch tensor named outmap to the [0, 1] interval. I tried this:

outmap_min = torch.min(outmap, dim=1, keepdim=True)
outmap_max = torch.max(outmap, dim=1, keepdim=True)
outmap = (outmap - outmap_min) / (outmap_max - outmap_min)

And I am getting this error: TypeError: unsupported operand type(s) for -: 'Tensor' and 'torch.return_types.min'

Answer 1

There are many ways to answer the question posed in your title (e.g., min-max normalization, or one of many non-linear functions mapping (-infinity, infinity) to [0, 1]). Based on the context of the post, I'm assuming you just want to implement min-max normalization.

torch.min() does not return a tensor; it returns a type akin to a tuple (and in fact the documentation says it's a namedtuple). It's a two-tuple; the first element is the tensor you're looking for (min), and the second element is an indexing tensor specifying the indices of the minimum values (argmin). The same goes for torch.max().

So, suppose your tensor is of size [N, M]. If you're trying to min-max normalize each "row" (dimension 0) based on the min and max of the M elements (columns) in that row, you'd compute the min and max along dimension 1. This would give you N mins and N maxes -- a min and max for each row. You'd then apply the normalization. This is what your code was intended to do, but you have to unpack the return values of torch.min() and torch.max(), discarding the argmin and argmax in this case:

outmap_min, _ = torch.min(outmap, dim=1, keepdim=True)
outmap_max, _ = torch.max(outmap, dim=1, keepdim=True)
outmap = (outmap - outmap_min) / (outmap_max - outmap_min) # Broadcasting rules apply

If your tensor is of size [N, ...], where ... indicates an arbitrary number of dimensions each of arbitrary size, and you want to min-max normalize each row as before, then the solution is a bit more tricky. Unfortunately, torch.min() and torch.max() each only accept a single dimension. If you want to compute extrema along multiple dimensions, you either have to call torch.min() / torch.max() once for each dimension, or you have to reshape / flatten your tensor so that all of the targeted dimensions are merged into one. So if your tensor is of size [N, ...], it just has to be reshaped to size [N, R, 1, 1, ...], where R is the product of the sizes of the remaining dimensions, and the 1's preserve the number of dimensions (this makes broadcasting easier later). For instance, if your tensor is of size [N, A, B, C], you should reshape it to size [N, A*B*C, 1, 1]. Following that, you can compute mins / maxes along the R-sized dimension 1, as before:

flattened_outmap = outmap.view(outmap.shape[0], -1, 1, 1) # Use 1's to preserve the number of dimensions for broadcasting later, as explained
outmap_min, _ = torch.min(flattened_outmap, dim=1, keepdim=True)
outmap_max, _ = torch.max(flattened_outmap, dim=1, keepdim=True)
outmap = (outmap - outmap_min) / (outmap_max - outmap_min) # Broadcasting rules apply

If you wanted to, you could programmatically construct the placeholder 1's as a list whose length is equal to the number of dimensions minus 2 (to account for the [N, R] dimensions)), and then expand the list. e.g.:

flattened_outmap = outmap.view(outmap.shape[0], -1, *([1] * (len(outmap.shape) - 2)))

In more complicated cases where you want the min / max normalization statistics to be computed across multiple non-contiguous dimensions, you'll probably have to permute the tensor prior to reshaping it. This can't be done with a view, so you'd have to create an extra copy of the tensor. This would surely require more memory than simply calling torch.min() / torch.max() once per dimension, but the tensorization of it might make it faster (it'd probably depend on the number of dimensions; you'd have to benchmark it).