Why cv2.rectangle sometimes return np.ndarray, while sometimes cv2.UMat

Question

I'm currently working on visualizing some images and found this weired behaviour of opencv's cv2.rectangle:

• when input image is an np.ndarray, say arr, cv2.rectangle() returns an np.ndarray, and arr is drawn with a rectangle.

• when input image is some variant of arr, like arr[:, :, [2, 0, 1]], cv2.rectangle() returns a cv2.UMat, and no rectangle is drawn.

My current environment is :

• Python 3.7
• Opencv 4.1

Here are the codes:

1. Firstly generates a random image.

import numpy as np
import cv2
import copy

img = np.random.randint(0, 255, (100, 120, 3)).astype("uint8")

2. Now add a rectangle

a = copy.deepcopy(img)
ret = cv2.rectangle(a, (0, 0), (10, 10), color=(255, 255, 255), thickness=2)

3. You'll find:

   • ret is an np.ndarray
   • visualization of ret and a show that one rectangle is drawn

4. Try another way:

b = copy.deepcopy(img)
c = b[:, :, [2, 1, 0]]
ret = cv2.rectangle(c, (0, 0), (10, 10), color=(255, 255, 255), thickness=2)

5. You'll find:

   • ret is a cv2.UMat
   • visualization of ret or c show that no rectangle is drawn

I'm really curious that is there anything wrong with my code? Or there is something hidden behind it?

Answer 1

I'll make the effor to answer this since I stumbled upon this problem a lot and in the comments I see a lot of correct stuff!

OpenCV can only deal with contiguous arrays, meaning they have to be laid out in a certain way in memory. When slicing an np.array, numpy just changes the read order to increase the speed (instead of time consuming copying) and makes it therefore non-contiguous (found here).

Both @Das Masek and @Eric are correct with their statements. Using index-array to slice an np.array creates always a copy as documented here. However, unfortunately numpy copies the array but won't change it back to a contiguous array (which to me seems like bad behavior).

The solution would be one of the following:

1. copy() the np.array; by explicitely copying, numpy changes the layout back to contiguous, as not done with index-array-slicing. You can check the flags of your array with a.flags and so forth. This is obiously the most expensive if you want to automate something because you are literally copying every single time.
2. the more elegant version for me would be to use np.ascontiguousarray(). This function changes the layout of the array only if it is already non-contiguous, and does not copy it.

---

On a different note: according the documentation, all OpenCV drawing function actually have a None return value, since they are inplace functions. Therefore I would recommend using them as such.