Understanding Big-O Notation: Growth Rates of Functions - Prof. Dali Wang, Study notes of Discrete Structures and Graph Theory

Download Understanding Big-O Notation: Growth Rates of Functions - Prof. Dali Wang and more Study notes Discrete Structures and Graph Theory in PDF only on Docsity!

Functions, Complexity ofAlgorithms

Section 2.2 The Growth of Functions

The Growth of Functions

Topics „

Big-O Definition „

Big-O by little-O „

Complexity Classes „

Properties and theorems of Big-O „

Big Omega and Big Theta

The Growth of Functions

Overview „

Example: algorithm Performance EstimatingRunning Time for Input of Size

n

1 hr.

→

60 hrs.

squares it

2

n

1 hr.

→

1 hr. 1

min.

increase by small constant

log

2

n

1 hr.

→

4 hrs.

multiplies by 4

n

2

1 hr.

→

2 hrs.

doubles

n

For example

then doubling input sizedoes following to run time

If runningtimegrows like:

The Growth of Functions

The Big-O Notation „

Purpose:

to describe the growth rate of a

function.e.g. does it grow like

log

n

? like

n

? like

n

2?

Examples: 2

n

2

+ 3

n +

^1

“grows like"

n

2.

0.5(

n

log

n

)^

-^

3

n

+ 7

“grows like"

n

log

n

.

Big-O is used to denote an upper bound on

growth rate:

f

( n

) grows no faster than

g

( n

)

The Growth of Functions

Big-O by Definition „

Example 1: f(n) = n + 3

0

≤

n

• 3

≤

n

n

= 2

n

for

n >

Therefore,

n

• 3 is

O

( n

).

„

Example 2

: f(n) = 4n

2

• n

0

≤

4

(^2) n

n

≤

4

(^2) n

n

2

= 5

(^2) n

for

n >

^1

.

Therefore

4

(^2) n

n

is

O

( n

2 )

.

The Growth of Functions

Big-O by Definition „

Example 3:

Show that

n

3

is not

O

n

If

n

3

is

O

(

n

2 ) then there are constants

C

and

k

such that

n

3

≤

100

Cn

2

for

n > k

. Then

n

≤

100

C

for all

n > k

. This is a contradiction since

n

grows

without bound.

The Growth of Functions

Examples „

Example 1:

3n + 5 is O(n

2 )

Proof: It's easy to show

Hence 3n + 5 is o(n

2 ) and so it is O(n

2 ).

Q. E. D. „

Example 2:

7n

2

is O(n

2 )

„

Example 3:

7n

2

is O(n

3 )

„

Example 4:

is O(n)

0 5

3 lim

2

=

∞ →^

n n

n

n

The Growth of Functions

Complexity Classes „

Note that O(g) is a set called a

complexity

class

.^

It contains all the functions which g

dominates. „

Important Complexity Classes O

O

(log

n

O

( n

)^

O

( n

log

n

)^

O

( n

2

O

( n

j^ )

O

( c

n^ )

O

( n

where j>2 and c>1.

The Growth of Functions

Properties of Big-O „

Part 2: the set O(g) is closed undermultiplication by a scalar

(real number):

If f is O(g) then

f is O(g)

„

Example: f^1

=2x

2

(O(x

2 )), f

=6x 2

2 : O(?)

g

=x (O(x)), g 1

=100x: O(?) 2

The Growth of Functions

Big-O Theorems „

Theorem: If f

1

is O(g

) and f 1

2

is O(g

) then 2

„

i) f

f 1

2

is O(g

g 1

)^2

„

ii) f

1

• f

2

is O(max{ g

, g 1

}) 2

„

Examples:

„

1. f

=2n 1

2 , f

=3n 2

f^1

f^2

: O(?) f^1

+f

: O(?) 2

„

2. f

=5log n, f 1

=5 2

f^1

f^2 f^1

+f

2

The Growth of Functions

Big-O Estimate Examples „

Example 1: Find Big-O (complexity class) ofthe function^ f(n)=

3n

2 +15n

„

Example 2:

Find Big-O (complexity class) of

the function f(n)=

(3n

2 +15n)(n+1)

3

„

Example 3:

Find the complexity class of the

function ( nn

n+

n

100

n

n^ +

n

n^ )

If a flop takes a nanosecond, how long it takes tosolve a problem for n=50.

The Growth of Functions

Big Omega and Big Theta „

Big O, Big Omega and Big Theta

„^

Upper bound: Big-O „^

Lower bound: Big-Omega „^

Both upper and lower bound: Big-Theta

„

The Big-Omega Definition:

Let f and g be functions

from N to R. Then f is Big-Omega of g

,^ denoted

Ω

(g), iff

∃ k

∃ C

∀

n [

n^

^

k^

→

|^ f

(

n ) |

≥^

C

|^

g (

n ) |]

„

The Big-Theta Definition:

Let f and g be functions from

N to R. Then f is Big-Theta of g

,^ denoted

Θ

(g), if f is

O(g) and f is

Ω

(g).