Faster-RCNN, why don't we just use only RPN for detection?

Question

As we know, faster-RCNN has two main parts: one is region proposal network(RPN), and another one is fast-RCNN.

My question is, now that region proposal network(RPN) can output class scores and bounding boxes and is trainable, why do we need Fast-RCNN?

Am I thinking it right that the RPN is enough for detection (red circle), and Fast-RCNN is now becoming redundant (blue circle)?

Answer 1

Short answer: no they are not redundant. The R-CNN article and its variants popularized the use of what we used to call a cascade. Back then for detection it was fairly common to use different detectors often very similar in structures to do detection because of their complementary power.

If the detections are partly orthogonal it allows to remove false positive along the way.

Furthermore by definition both parts of R-CNN have different roles the first one is used to discriminate objects from background and the second one to discriminate fine grained categories of objects from themselves (and from the background also).

But you are right if there is only 1 class vs the background one could use only the RPN part to to detection but even in that case it would probably better the result to chain two different classifiers (or not see e.g. this article)

PS: I answered because I wanted to but this question is definitely unsuited for stackoverflow