Finding the Sky/Ground separation in OpenCV

Question

I am trying to detect the horizon in an image, and return a mask of the sky (or inverted as the ground). While there seems to be many uses for it, I am battling to find a good solution. What's worse is that it seems like such a simple problem, and most humans have NO issue in detecting the horizon.

The following makes it harder:

• The horizon is rarely a straight line in the images used (mountainous landscapes), therefore a edge detection and Hough line transform will not work.
• It needs to work in all light conditions. Thresholding (such as the Otsu thresholding) works but does not work well in low contrast conditions such as before sunrise. Fixed value thresholding does not work as the light changes too much throughout the day.

What I have tried for now is to use a colour filter limiting it to low saturations, then find contours and detect and fill the largest contour. After this, I flood fill the area above the contour. This does work, but I still can't imagine this problem to be so difficult.

I am writing the code in Delphi XE8, using a OpenCV wrapper, but answers or ideas in any other language are welcome!

Answer 1

In my understanding you are looking for a horizontal line - if exists at all - to separate the exclusively sky part from the rest.

I would compute image statistics row by row, so a horizontal histogram or similar to that.

It could be based even on a global threshold or a custom 'skyness' function. Decide somehow (intensity, hue) if pixels are sky or not and count them within scanlines.

Then half the image horizontally, sum row values for both parts and decide which direction your 'horizont line' should be moved. Half that part too and continue until you get to the proper row. With such a binary search you should be able to extract which line separates sky from foreground. If it is the first line: no sky, if last: all sky.

This problem has surely has other approaches so I am looking forward to seeing more suggestions.