Digital camera algorithms

Question

I'm working on a simple video device and I'd like to introduce some standard cool camera features. Amongst all I'd like to introduce

• Focus indicator
• Auto focus
• Auto exposure (ideal exposure time estimation)

Right now I'm looking for some examples, how these features can be implemented. Do you have any useful links?

EDIT : Ok, I will use standard CCD camera, which can provide me ~ 20fps in ~1MPix resolution. I'm planning to write it in C#, in case of performance issues, I'll use C++. I'll have lens + CCD camera + motor.

EDIT : I'd like to see some more detailed algorithm description. I'm sure some have to be taught in university courses, but I have troubles finding some. For focus indicator I've tried a primitive approach, but in some cases it failed.

 int verticalPoints = 0, horizontalPoints = 0;  ///Calculate the vertical differences  for (int x = 0; x < toAnalyze.Width; x++)  {      for (int y = 1; y < toAnalyze.Height; y++)      {         byte* pixel = (byte*)data.Scan0 + y * stride + x;         verticalDiff += Math.Abs(*pixel - *(pixel - stride));;      }            }   verticalDiff /= toAnalyze.Width * (toAnalyze.Height-1);   ///Calculate horizontal differences   for (int y = 0; y < toAnalyze.Height; y++)   {      for (int x = 1; x < toAnalyze.Width; x++)      {         byte* pixel = (byte*)data.Scan0 + y * stride + x;         horizontalDiff += Math.Abs(*pixel - *(pixel - 1));      }    }    horizontalDiff /= (toAnalyze.Width-1) * toAnalyze.Height;    ///And return the average value    return(verticalDiff + horizontalDiff) / 2; 

Thanks

Answer 1

Starting from the end, so to speak:

Auto-exposure is pretty simple: measure the light level and figure out how long of an exposure is needed for that average light to produce ~15-18% gray level. There are lots of attempts at improving that (usually by metering a number of sections of the picture separately, and processing those results), but that's the starting point.

There are two separate types of autofocus. Most video cameras use one based on detecting contrast -- look at the input from the sensor, and when the differences between adjacent pixels are maximized, you consider that "in focus."

Contrast detection autofocus does make it a bit difficult to do focus indication though -- in particular, you never really know when you've achieved maximum contrast until the contrast starts to fall again. When you're doing autofocus, you focus until you see a peak and then see it start to fall again, and then drive it back to where it was highest. For manual focus with an indicator, you can't recognize maximum contrast until it starts to fall again. The user would have to follow roughly the same pattern, moving past best focus, then back to optimum.

Alternatively, you could use phase detection. This uses the alignment of the "pictures" coming through two prisms, much like the split-image viewfinders that were used in many (most?) SLRs before autofocus came into use.

Answer 2

Just to inform you. I am working on a professional forensic 5 Megapixel digital camera software in WPF. In DotNet not C++. There are some threading issus to know but it works perfectly fast. More performant because GPU is used.

Jerry did a good work with his answer. Focus detection is "Contrast detection based on time / frames". Logic is simple, to keep it performant it is not easy. Auto Focus detection

To check the exposure time, it is easy if you have created the histogram of image. Image histogram In any case you need to do it for

• Red channel
• Green channel
• Blue channel
• Gain
• Exposure time

This mix makes it a bit more complicated because you also can use Color gain channels to increase brightness of image. RGB image digital. Luminance can have the same result like with "Gain" and "Exposure" time.

If you calculate the exposure time automatically, keep good in mind that you need a frame to calculate it and as smaller the exposure time as more frames you will get. That means, if you want to have a good algorithm, always try to have a very small exposure time and increase it slowly. Not use a linear algorithm where you decrease the value slowly.

There are also more methodes for digital cameras like Pixel Binning Pixel Binning to increase framerate to get quick focus results.

Here is a sample of how focus can work to generate a focus intensity image :

 Private Sub GetFocusValue(ByRef C1 As Color, ByVal LCol1 As List(Of Color), ByVal LCol2 As List(Of Color), ByVal AmplifierPercent As Single)         Dim MaxDiff1 As Integer = 0         Dim MaxDiff2 As Integer = 0         Dim Factor As Single = 0         Dim D As Integer          Dim LR1 As New List(Of Integer)         Dim LR2 As New List(Of Integer)         Dim LG1 As New List(Of Integer)         Dim LG2 As New List(Of Integer)         Dim LB1 As New List(Of Integer)         Dim LB2 As New List(Of Integer)          For Each C As Color In LCol1             LR1.Add(C.R)             LG1.Add(C.G)             LB1.Add(C.B)         Next           For Each C As Color In LCol2             LR2.Add(C.R)             LG2.Add(C.G)             LB2.Add(C.B)         Next            MaxDiff1 = Me.GetMaxDiff(LR1)         MaxDiff1 = Math.Max(MaxDiff1, Me.GetMaxDiff(LG1))         MaxDiff1 = Math.Max(MaxDiff1, Me.GetMaxDiff(LB1))           MaxDiff2 = Me.GetMaxDiff(LR2)         MaxDiff2 = Math.Max(MaxDiff2, Me.GetMaxDiff(LG2))         MaxDiff2 = Math.Max(MaxDiff2, Me.GetMaxDiff(LB2))            If MaxDiff1 > MaxDiff2 Then             D = MaxDiff1 - MaxDiff2             Factor = D / 255             Factor = Factor / (AmplifierPercent / 100)             Factor = Math.Min(Factor, 1)             Factor = 1 - Factor 'invert result             'TB.Math.Swap(MaxDiff1, MaxDiff2)             'Factor = 255 'the original BM1 is better         Else             D = MaxDiff2 - MaxDiff1             Factor = D / 255             Factor = Factor * (AmplifierPercent / 100)             Factor = Math.Min(Factor, 1)             'Factor = 0 'the BM2 is better         End If         Factor = Factor * 255            C1 = Color.FromArgb(Convert.ToByte(Factor), C1.R, C1.G, C1.B)       End Sub