Constants in the formal definition of Big O

Question

I'm revising the formal definitions of Big O and the other associated bounds and something is tripping me up. In the book I'm reading (Skiena) Big O is defined as:

f(n) = O(g(n)) when there exists a constant c such that f(n) is always <= c*g(n) for a some value of n > n0

This generally makes sense to me. We are only concerned with large enough values of n that the growth rates actually matter. But why multiply g(n) by c? It seem like I could choose a very large value for c, and make the whole thing arbitrary by blowing out the size of smaller g(n) values.

Secondary question: When choosing to classify an algorithm into a complexity class, is the general rule of thumb to just choose the lowest growth class that still holds according to the definition of Big O? According to the definition it seems valid to classify a constant time algorithm as O(n!), simply because f(n) will be <= c*g(n). Off course this offers no value.

Thanks!

Answer 1

You can multiply g(n) by an arbitrary constant c is because you want functions that are only a constant c factor away from f(n). In simple terms you perform your analysis based on n, not constants so what you are concerned with is how those functions change depending on the input size only. For instance when you have n^3 and n there's no way you can choose a c where c*n >= n^3 unless c >= n^2 which isn't constant anymore so g(n) will be running away from f(n) with n.

As Ed mentioned this analysis won't give you an exact run time but a growth rate depending on input n. If g(n) and f(n) are always only (at most) a constant factor away from each other than the growth rate will be the same for both.

In this kind of time complexity analysis we don't really care about constant which in most cases is ok but in some cases you actually should take it into account. For instance if you are working on small sets an O(n^2) algorithm might actually be faster than O(nlogn) because of constants.

Second question: yes this is a common problem with BigO, you can use an arbitrary function that's why we usually are trying to find the "tightest" g(n) we can, otherwise there's no much point in finding it. That's also why *BigTheta is much more useful than BigO as it tells you a tight bound, instead of an upper bound.

Answer 2

When choosing to classify an algorithm into a complexity class, is the general rule of thumb to just choose the lowest growth class that still holds according to the definition of Big O?

In terms of notations, just like we have big-O for upper bounds we have big-Omega for lower bounds and big-Theta for when you are able to show that the upper bound and the lower bounds match.

https://en.wikipedia.org/wiki/Big_O_notation#The_Knuth_definition

Assuming that Knuth quote is correct, then we can say that you are not alone in assuming that results involving tight asymptotic bounds are more useful :) Sometimes people say big-O when they actually meant to say big-Theta but some other times they just don't care or haven't managed to find the lower bound.

It seem like I could choose a very large value for c, and make the whole thing arbitrary by blowing out the size of smaller g(n) values.

For functions with different asymptotic growth rates, the c doesn't matter. No matter how big or how small you choose c to be, there will be an n when the faster growing function catches up. The constant factor is there to allow you to ignore constant multipliers when things have the same growth rate. For example, when it comes to big-O, f(x) = 2x and g(x) = 3x both have the same growth rate.