How do I efficiently determine if a polygon is convex, non-convex or complex?

Question

From the man page for XFillPolygon:

• If shape is Complex, the path may self-intersect. Note that contiguous coincident points in the path are not treated as self-intersection.

• If shape is Convex, for every pair of points inside the polygon, the line segment connecting them does not intersect the path. If known by the client, specifying Convex can improve performance. If you specify Convex for a path that is not convex, the graphics results are undefined.

• If shape is Nonconvex, the path does not self-intersect, but the shape is not wholly convex. If known by the client, specifying Nonconvex instead of Complex may improve performance. If you specify Nonconvex for a self-intersecting path, the graphics results are undefined.

I am having performance problems with fill XFillPolygon and, as the man page suggests, the first step I want to take is to specify the correct shape of the polygon. I am currently using Complex to be on the safe side.

Is there an efficient algorithm to determine if a polygon (defined by a series of coordinates) is convex, non-convex or complex?

Answer 1

You can make things a lot easier than the Gift-Wrapping Algorithm... that's a good answer when you have a set of points w/o any particular boundary and need to find the convex hull.

In contrast, consider the case where the polygon is not self-intersecting, and it consists of a set of points in a list where the consecutive points form the boundary. In this case it is much easier to figure out whether a polygon is convex or not (and you don't have to calculate any angles, either):

For each consecutive pair of edges of the polygon (each triplet of points), compute the z-component of the cross product of the vectors defined by the edges pointing towards the points in increasing order. Take the cross product of these vectors:

 given p[k], p[k+1], p[k+2] each with coordinates x, y:  dx1 = x[k+1]-x[k]  dy1 = y[k+1]-y[k]  dx2 = x[k+2]-x[k+1]  dy2 = y[k+2]-y[k+1]  zcrossproduct = dx1*dy2 - dy1*dx2 

The polygon is convex if the z-components of the cross products are either all positive or all negative. Otherwise the polygon is nonconvex.

If there are N points, make sure you calculate N cross products, e.g. be sure to use the triplets (p[N-2],p[N-1],p[0]) and (p[N-1],p[0],p[1]).

---

If the polygon is self-intersecting, then it fails the technical definition of convexity even if its directed angles are all in the same direction, in which case the above approach would not produce the correct result.

Answer 2

This question is now the first item in either Bing or Google when you search for "determine convex polygon." However, none of the answers are good enough.

The (now deleted) answer by @EugeneYokota works by checking whether an unordered set of points can be made into a convex polygon, but that's not what the OP asked for. He asked for a method to check whether a given polygon is convex or not. (A "polygon" in computer science is usually defined [as in the XFillPolygon documentation] as an ordered array of 2D points, with consecutive points joined with a side as well as the last point to the first.) Also, the gift wrapping algorithm in this case would have the time-complexity of O(n^2) for n points - which is much larger than actually needed to solve this problem, while the question asks for an efficient algorithm.

@JasonS's answer, along with the other answers that follow his idea, accepts star polygons such as a pentagram or the one in @zenna's comment, but star polygons are not considered to be convex. As @plasmacel notes in a comment, this is a good approach to use if you have prior knowledge that the polygon is not self-intersecting, but it can fail if you do not have that knowledge.

@Sekhat's answer is correct but it also has the time-complexity of O(n^2) and thus is inefficient.

@LorenPechtel's added answer after her edit is the best one here but it is vague.

A correct algorithm with optimal complexity

The algorithm I present here has the time-complexity of O(n), correctly tests whether a polygon is convex or not, and passes all the tests I have thrown at it. The idea is to traverse the sides of the polygon, noting the direction of each side and the signed change of direction between consecutive sides. "Signed" here means left-ward is positive and right-ward is negative (or the reverse) and straight-ahead is zero. Those angles are normalized to be between minus-pi (exclusive) and pi (inclusive). Summing all these direction-change angles (a.k.a the deflection angles) together will result in plus-or-minus one turn (i.e. 360 degrees) for a convex polygon, while a star-like polygon (or a self-intersecting loop) will have a different sum ( n * 360 degrees, for n turns overall, for polygons where all the deflection angles are of the same sign). So we must check that the sum of the direction-change angles is plus-or-minus one turn. We also check that the direction-change angles are all positive or all negative and not reverses (pi radians), all points are actual 2D points, and that no consecutive vertices are identical. (That last point is debatable--you may want to allow repeated vertices but I prefer to prohibit them.) The combination of those checks catches all convex and non-convex polygons.

Here is code for Python 3 that implements the algorithm and includes some minor efficiencies. The code looks longer than it really is due to the the comment lines and the bookkeeping involved in avoiding repeated point accesses.

TWO_PI = 2 * pi  def is_convex_polygon(polygon):     """Return True if the polynomial defined by the sequence of 2D     points is 'strictly convex': points are valid, side lengths non-     zero, interior angles are strictly between zero and a straight     angle, and the polygon does not intersect itself.      NOTES:  1.  Algorithm: the signed changes of the direction angles                 from one side to the next side must be all positive or                 all negative, and their sum must equal plus-or-minus                 one full turn (2 pi radians). Also check for too few,                 invalid, or repeated points.             2.  No check is explicitly done for zero internal angles                 (180 degree direction-change angle) as this is covered                 in other ways, including the `n < 3` check.     """     try:  # needed for any bad points or direction changes         # Check for too few points         if len(polygon) < 3:             return False         # Get starting information         old_x, old_y = polygon[-2]         new_x, new_y = polygon[-1]         new_direction = atan2(new_y - old_y, new_x - old_x)         angle_sum = 0.0         # Check each point (the side ending there, its angle) and accum. angles         for ndx, newpoint in enumerate(polygon):             # Update point coordinates and side directions, check side length             old_x, old_y, old_direction = new_x, new_y, new_direction             new_x, new_y = newpoint             new_direction = atan2(new_y - old_y, new_x - old_x)             if old_x == new_x and old_y == new_y:                 return False  # repeated consecutive points             # Calculate & check the normalized direction-change angle             angle = new_direction - old_direction             if angle <= -pi:                 angle += TWO_PI  # make it in half-open interval (-Pi, Pi]             elif angle > pi:                 angle -= TWO_PI             if ndx == 0:  # if first time through loop, initialize orientation                 if angle == 0.0:                     return False                 orientation = 1.0 if angle > 0.0 else -1.0             else:  # if other time through loop, check orientation is stable                 if orientation * angle <= 0.0:  # not both pos. or both neg.                     return False             # Accumulate the direction-change angle             angle_sum += angle         # Check that the total number of full turns is plus-or-minus 1         return abs(round(angle_sum / TWO_PI)) == 1     except (ArithmeticError, TypeError, ValueError):         return False  # any exception means not a proper convex polygon