O(nlogn) Algorithm - Find three evenly spaced ones within binary string

Question

Finally! Following up leads in sdcvvc's answer, we have it: the O(n log n) algorithm for the problem! It is simple too, after you understand it. Those who guessed FFT were right.

The problem: we are given a binary string S of length n, and we want to find three evenly spaced 1s in it. For example, S may be 110110010, where n=9. It has evenly spaced 1s at positions 2, 5, and 8.

Scan S left to right, and make a list L of positions of 1. For the S=110110010 above, we have the list L = [1, 2, 4, 5, 8]. This step is O(n). The problem is now to find an arithmetic progression of length 3 in L, i.e. to find distinct a, b, c in L such that b-a = c-b, or equivalently a+c=2b. For the example above, we want to find the progression (2, 5, 8).
Make a polynomial p with terms x^k for each k in L. For the example above, we make the polynomial p(x) = (x + x² + x⁴ + x⁵+x⁸). This step is O(n).
Find the polynomial q = p², using the Fast Fourier Transform. For the example above, we get the polynomial q(x) = x¹⁶ + 2x¹³ + 2x¹² + 3x¹⁰ + 4x⁹ + x⁸ + 2x⁷ + 4x⁶ + 2x⁵ + x⁴ + 2x³ + x². This step is O(n log n).
Ignore all terms except those corresponding to x^2k for some k in L. For the example above, we get the terms x¹⁶, 3x¹⁰, x⁸, x⁴, x². This step is O(n), if you choose to do it at all.

Here's the crucial point: the coefficient of any x^2b for b in L is precisely the number of pairs (a,c) in L such that a+c=2b. [CLRS, Ex. 30.1-7] One such pair is (b,b) always (so the coefficient is at least 1), but if there exists any other pair (a,c), then the coefficient is at least 3, from (a,c) and (c,a). For the example above, we have the coefficient of x¹⁰ to be 3 precisely because of the AP (2,5,8). (These coefficients x^2b will always be odd numbers, for the reasons above. And all other coefficients in q will always be even.)

So then, the algorithm is to look at the coefficients of these terms x^2b, and see if any of them is greater than 1. If there is none, then there are no evenly spaced 1s. If there is a b in L for which the coefficient of x^2b is greater than 1, then we know that there is some pair (a,c) — other than (b,b) — for which a+c=2b. To find the actual pair, we simply try each a in L (the corresponding c would be 2b-a) and see if there is a 1 at position 2b-a in S. This step is O(n).

That's all, folks.

One might ask: do we need to use FFT? Many answers, such as beta's, flybywire's, and rsp's, suggest that the approach that checks each pair of 1s and sees if there is a 1 at the "third" position, might work in O(n log n), based on the intuition that if there are too many 1s, we would find a triple easily, and if there are too few 1s, checking all pairs takes little time. Unfortunately, while this intuition is correct and the simple approach is better than O(n²), it is not significantly better. As in sdcvvc's answer, we can take the "Cantor-like set" of strings of length n=3^k, with 1s at the positions whose ternary representation has only 0s and 2s (no 1s) in it. Such a string has 2^k = n^{(log 2)/(log 3)} ≈ n^0.63 ones in it and no evenly spaced 1s, so checking all pairs would be of the order of the square of the number of 1s in it: that's 4^k ≈ n^1.26 which unfortunately is asymptotically much larger than (n log n). In fact, the worst case is even worse: Leo Moser in 1953 constructed (effectively) such strings which have n^{1-c/√(log n)} 1s in them but no evenly spaced 1s, which means that on such strings, the simple approach would take Θ(n^{2-2c/√(log n)}) — only a tiny bit better than Θ(n²), surprisingly!

About the maximum number of 1s in a string of length n with no 3 evenly spaced ones (which we saw above was at least n^0.63 from the easy Cantor-like construction, and at least n^{1-c/√(log n)} with Moser's construction) — this is OEIS A003002. It can also be calculated directly from OEIS A065825 as the k such that A065825(k) ≤ n < A065825(k+1). I wrote a program to find these, and it turns out that the greedy algorithm does not give the longest such string. For example, for n=9, we can get 5 1s (110100011) but the greedy gives only 4 (110110000), for n=26 we can get 11 1s (11001010001000010110001101) but the greedy gives only 8 (11011000011011000000000000), and for n=74 we can get 22 1s (11000010110001000001011010001000000000000000010001011010000010001101000011) but the greedy gives only 16 (11011000011011000000000000011011000011011000000000000000000000000000000000). They do agree at quite a few places until 50 (e.g. all of 38 to 50), though. As the OEIS references say, it seems that Jaroslaw Wroblewski is interested in this question, and he maintains a website on these non-averaging sets. The exact numbers are known only up to 194.

Your problem is called AVERAGE in this paper (1999):

A problem is 3SUM-hard if there is a sub-quadratic reduction from the problem 3SUM: Given a set A of n integers, are there elements a,b,c in A such that a+b+c = 0? It is not known whether AVERAGE is 3SUM-hard. However, there is a simple linear-time reduction from AVERAGE to 3SUM, whose description we omit.

Wikipedia:

When the integers are in the range [−u ... u], 3SUM can be solved in time O(n + u lg u) by representing S as a bit vector and performing a convolution using FFT.

This is enough to solve your problem :).

What is very important is that O(n log n) is complexity in terms of number of zeroes and ones, not the count of ones (which could be given as an array, like [1,5,9,15]). Checking if a set has an arithmetic progression, terms of number of 1's, is hard, and according to that paper as of 1999 no faster algorithm than O(n²) is known, and is conjectured that it doesn't exist. Everybody who doesn't take this into account is attempting to solve an open problem.

Other interesting info, mostly irrevelant:

Lower bound:

An easy lower bound is Cantor-like set (numbers 1..3^n-1 not containing 1 in their ternary expansion) - its density is n^(log_3 2) (circa 0.631). So any checking if the set isn't too large, and then checking all pairs is not enough to get O(n log n). You have to investigate the sequence smarter. A better lower bound is quoted here - it's n^{1-c/(log(n))^(1/2)}. This means Cantor set is not optimal.

Upper bound - my old algorithm:

It is known that for large n, a subset of {1,2,...,n} not containing arithmetic progression has at most n/(log n)^(1/20) elements. The paper On triples in arithmetic progression proves more: the set cannot contain more than n * 2²⁸ * (log log n / log n)^1/2 elements. So you could check if that bound is achieved and if not, naively check pairs. This is O(n² * log log n / log n) algorithm, faster than O(n²). Unfortunately "On triples..." is on Springer - but the first page is available, and Ben Green's exposition is available here, page 28, theorem 24.

By the way, the papers are from 1999 - the same year as the first one I mentioned, so that's probably why the first one doesn't mention that result.

This is not a solution, but a similar line of thought to what Olexiy was thinking

I was playing around with creating sequences with maximum number of ones, and they are all quite interesting, I got up to 125 digits and here are the first 3 numbers it found by attempting to insert as many '1' bits as possible:

11011000011011000000000000001101100001101100000000000000000000000000000000000000000110110000110110000000000000011011000011011
10110100010110100000000000010110100010110100000000000000000000000000000000000000000101101000101101000000000000101101000101101
10011001010011001000000000010011001010011001000000000000000000000000000000000000010011001010011001000000000010011001010011001

Notice they are all fractals (not too surprising given the constraints). There may be something in thinking backwards, perhaps if the string is not a fractal of with a characteristic, then it must have a repeating pattern?

Thanks to beta for the better term to describe these numbers.

Update: Alas it looks like the pattern breaks down when starting with a large enough initial string, such as: 10000000000001:

100000000000011
10000000000001101
100000000000011011
10000000000001101100001
100000000000011011000011
10000000000001101100001101
100000000000011011000011010000000001
100000000000011011000011010000000001001
1000000000000110110000110100000000010011
1000000000000110110000110100000000010011001
10000000000001101100001101000000000100110010000000001
10000000000001101100001101000000000100110010000000001000001
1000000000000110110000110100000000010011001000000000100000100000000000001
10000000000001101100001101000000000100110010000000001000001000000000000011
1000000000000110110000110100000000010011001000000000100000100000000000001101
100000000000011011000011010000000001001100100000000010000010000000000000110100001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001100010000000010000000000000000000000000000000000000000100000010000000000000011
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001100010000000010000000000000000000000000000000000000000100000010000000000000011000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000011
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000011000001
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001100010000000010000000000000000000000000000000000000000100000010000000000000011000000001100100000000100100000000000010000000010000100000100100010010000010000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000110000010000000000000000000001
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001100010000000010000000000000000000000000000000000000000100000010000000000000011000000001100100000000100100000000000010000000010000100000100100010010000010000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000110000010000000000000000000001001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001100000100000000000000000000010010000000000000000000000000000000000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000011000001000000000000000000000100100000000000000000000000000000000000011
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000011000001000000000000000000000100100000000000000000000000000000000000011001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001100000100000000000000000000010010000000000000000000000000000000000001100100000000000000000000001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001100000100000000000000000000010010000000000000000000000000000000000001100100000000000000000000001001
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001100000100000000000000000000010010000000000000000000000000000000000001100100000000000000000000001001000001
100000000000011011000011010000000001001100100000000010000010000000000000110100001001000001000000110001000000001000000000000000000000000000000000000000010000001000000000000001100000000110010000000010010000000000001000000001000010000010010001001000001000000000000010000000000000000000000000000000000000000000000000000000000000000000000000000000000001000000000000000011000001000000000000000000000100100000000000000000000000000000000000011001000000000000000000000010010000010000001
1000000000000110110000110100000000010011001000000000100000100000000000001101000010010000010000001100010000000010000000000000000000000000000000000000000100000010000000000000011000000001100100000000100100000000000010000000010000100000100100010010000010000000000000100000000000000000000000000000000000000000000000000000000000000000000000000000000000010000000000000000110000010000000000000000000001001000000000000000000000000000000000000110010000000000000000000000100100000100000011
10000000000001101100001101000000000100110010000000001000001000000000000011010000100100000100000011000100000000100000000000000000000000000000000000000001000000100000000000000110000000011001000000001001000000000000100000000100001000001001000100100000100000000000001000000000000000000000000000000000000000000000000000000000000000000000000000000000000100000000000000001100000100000000000000000000010010000000000000000000000000000000000001100100000000000000000000001001000001000000110000000000001

I suspect that a simple approach that looks like O(n^2) will actually yield something better, like O(n ln(n)). The sequences that take the longest to test (for any given n) are the ones that contain no trios, and that puts severe restrictions on the number of 1's that can be in the sequence.

I've come up with some hand-waving arguments, but I haven't been able to find a tidy proof. I'm going to take a stab in the dark: the answer is a very clever idea that the professor has known for so long that it's come to seem obvious, but it's much too hard for the students. (Either that or you slept through the lecture that covered it.)

Revision: 2009-10-17 23:00

I've run this on large numbers (like, strings of 20 million) and I now believe this algorithm is not O(n logn). Notwithstanding that, it's a cool enough implementation and contains a number of optimizations that makes it run really fast. It evaluates all the arrangements of binary strings 24 or fewer digits in under 25 seconds.

I've updated the code to include the 0 <= L < M < U <= X-1 observation from earlier today.

Original

This is, in concept, similar to another question I answered. That code also looked at three values in a series and determined if a triplet satisfied a condition. Here is C# code adapted from that:

using System;
using System.Collections.Generic;

namespace StackOverflow1560523
{
    class Program
    {
        public struct Pair<T>
        {
            public T Low, High;
        }
        static bool FindCandidate(int candidate, 
            List<int> arr, 
            List<int> pool, 
            Pair<int> pair, 
            ref int iterations)
        {
            int lower = pair.Low, upper = pair.High;
            while ((lower >= 0) && (upper < pool.Count))
            {
                int lowRange = candidate - arr[pool[lower]];
                int highRange = arr[pool[upper]] - candidate;
                iterations++;
                if (lowRange < highRange)
                    lower -= 1;
                else if (lowRange > highRange)
                    upper += 1;
                else
                    return true;
            }
            return false;
        }
        static List<int> BuildOnesArray(string s)
        {
            List<int> arr = new List<int>();
            for (int i = 0; i < s.Length; i++)
                if (s[i] == '1')
                    arr.Add(i);
            return arr;
        }
        static void BuildIndexes(List<int> arr, 
            ref List<int> even, ref List<int> odd, 
            ref List<Pair<int>> evenIndex, ref List<Pair<int>> oddIndex)
        {
            for (int i = 0; i < arr.Count; i++)
            {
                bool isEven = (arr[i] & 1) == 0;
                if (isEven)
                {
                    evenIndex.Add(new Pair<int> {Low=even.Count-1, High=even.Count+1});
                    oddIndex.Add(new Pair<int> {Low=odd.Count-1, High=odd.Count});
                    even.Add(i);
                }
                else
                {
                    oddIndex.Add(new Pair<int> {Low=odd.Count-1, High=odd.Count+1});
                    evenIndex.Add(new Pair<int> {Low=even.Count-1, High=even.Count});
                    odd.Add(i);
                }
            }
        }

        static int FindSpacedOnes(string s)
        {
            // List of indexes of 1s in the string
            List<int> arr = BuildOnesArray(s);
            //if (s.Length < 3)
            //    return 0;

            //  List of indexes to odd indexes in arr
            List<int> odd = new List<int>(), even = new List<int>();

            //  evenIndex has indexes into arr to bracket even numbers
            //  oddIndex has indexes into arr to bracket odd numbers
            List<Pair<int>> evenIndex = new List<Pair<int>>(), 
                oddIndex = new List<Pair<int>>(); 
            BuildIndexes(arr, 
                ref even, ref odd, 
                ref evenIndex, ref oddIndex);

            int iterations = 0;
            for (int i = 1; i < arr.Count-1; i++)
            {
                int target = arr[i];
                bool found = FindCandidate(target, arr, odd, oddIndex[i], ref iterations) || 
                    FindCandidate(target, arr, even, evenIndex[i], ref iterations);
                if (found)
                    return iterations;
            }
            return iterations;
        }
        static IEnumerable<string> PowerSet(int n)
        {
            for (long i = (1L << (n-1)); i < (1L << n); i++)
            {
                yield return Convert.ToString(i, 2).PadLeft(n, '0');
            }
        }
        static void Main(string[] args)
        {
            for (int i = 5; i < 64; i++)
            {
                int c = 0;
                string hardest_string = "";
                foreach (string s in PowerSet(i))
                {
                    int cost = find_spaced_ones(s);
                    if (cost > c)
                    {
                        hardest_string = s;
                        c = cost;
                        Console.Write("{0} {1} {2}\r", i, c, hardest_string);
                    }
                }
                Console.WriteLine("{0} {1} {2}", i, c, hardest_string);
            }
        }
    }
}

The principal differences are:

Exhaustive search of solutions
This code generates a power set of data to find the hardest input to solve for this algorithm.
All solutions versus hardest to solve
The code for the previous question generated all the solutions using a python generator. This code just displays the hardest for each pattern length.
Scoring algorithm
This code checks the distance from the middle element to its left- and right-hand edge. The python code tested whether a sum was above or below 0.
Convergence on a candidate
The current code works from the middle towards the edge to find a candidate. The code in the previous problem worked from the edges towards the middle. This last change gives a large performance improvement.
Use of even and odd pools
Based on the observations at the end of this write-up, the code searches pairs of even numbers of pairs of odd numbers to find L and U, keeping M fixed. This reduces the number of searches by pre-computing information. Accordingly, the code uses two levels of indirection in the main loop of FindCandidate and requires two calls to FindCandidate for each middle element: once for even numbers and once for odd ones.

The general idea is to work on indexes, not the raw representation of the data. Calculating an array where the 1's appear allows the algorithm to run in time proportional to the number of 1's in the data rather than in time proportional to the length of the data. This is a standard transformation: create a data structure that allows faster operation while keeping the problem equivalent.

The results are out of date: removed.

Edit: 2009-10-16 18:48

On yx's data, which is given some credence in the other responses as representative of hard data to calculate on, I get these results... I removed these. They are out of date.

I would point out that this data is not the hardest for my algorithm, so I think the assumption that yx's fractals are the hardest to solve is mistaken. The worst case for a particular algorithm, I expect, will depend upon the algorithm itself and will not likely be consistent across different algorithms.

Edit: 2009-10-17 13:30

Further observations on this.

First, convert the string of 0's and 1's into an array of indexes for each position of the 1's. Say the length of that array A is X. Then the goal is to find

0 <= L < M < U <= X-1

such that

A[M] - A[L] = A[U] - A[M]

or

2*A[M] = A[L] + A[U]

Since A[L] and A[U] sum to an even number, they can't be (even, odd) or (odd, even). The search for a match could be improved by splitting A[] into odd and even pools and searching for matches on A[M] in the pools of odd and even candidates in turn.

However, this is more of a performance optimization than an algorithmic improvement, I think. The number of comparisons should drop, but the order of the algorithm should be the same.

Edit 2009-10-18 00:45

Yet another optimization occurs to me, in the same vein as separating the candidates into even and odd. Since the three indexes have to add to a multiple of 3 (a, a+x, a+2x -- mod 3 is 0, regardless of a and x), you can separate L, M, and U into their mod 3 values:

In fact, you could combine this with the even/odd observation and separate them into their mod 6 values:

and so on. This would provide a further performance optimization but not an algorithmic speedup.

O(nlogn) Algorithm - Find three evenly spaced ones within binary string

Tags:

algorithm

big-o

Recent Activity

Donate For Us

O(nlogn) Algorithm - Find three evenly spaced ones within binary string

Tags:

algorithm

big-o

Related questions

Recent Activity

Donate For Us