Separating Axis Theorem: Finding which edge normals to use

Question

I'm using the implementation of the algorithm described here: http://www.codezealot.org/archives/55

Using this implementation, when I want to find the minimal displacement vector for two colliding rectangles, the algorithm will sometimes (in one of two cases to be exact) give me the wrong "direction" in which one rectangle needs to pushed in order to get away from the rectangle that it's colliding with. It will instead produce the exact opposite direction from the one which it is supposed to go in.

This is because (of course) the two opposing edges of a rectangle are of course mathematically similar and hence produce the same results.

This is the implementation of the algorithm in Java:

http://pastebin.com/fgAwBsGv

You can assume that Vector2f, Projection2D etc. function correctly and produce correct values. By the way,

I'm using left hand normals because the polygon is assembled counterclock-wise. Principally, this should be enough, since the left-hand normal will always point away from the polygon. The problem is that since the two overlaps of opposing edges of a rectangle will be exactly the same, the algorithm will simply use the first one and its corresponding axis.

thanks for your help!

Answer 1

Hey, I forgot that I created this question a while ago, so I'm going to answer it now in case somebody else has the same problem:

The solution is to simply turn around the displacement vector (multiply it by -1) if it's pointing towards the shape from which the object needs to be pushed away.

To find out if the displacement vector is pointing towards the shape, you first have to get the general direction from object a to object b by subtracting their centers from each other. After that, you check the dot product between the displacement vector and the direction(a,b) vector that you just created. If it's > 0, the displacement vector and direction(a,b) are pointing in the same direction, hence you need to flip the displacement vector.