Time complexities and certain laws, Lecture notes of Discrete Mathematics

Download Time complexities and certain laws and more Lecture notes Discrete Mathematics in PDF only on Docsity!

CSCE 222 Discrete Structures for Computing

Analysis of Algorithms 1: Basic Principles

Dr. Hyunyoung Lee

Analysis of Algorithms

The purpose of the analysis is to get some insights into the running time or the space requirements of the computation.

Analysis of Algorithms

The purpose of the analysis is to get some insights into the running time or the space requirements of the computation.

Since too many details of the compiler and computer architecture affect the precise time or space requirements, one has to settle for the more modest goal of obtaining reasonable bounds on these quantities.

We will focus on the analysis of the running time of an algorithm.

Computational Model: Random Access Machine

Since we are settling for bounds on the running time rather than a precise estimate, we can use a simplified model of computation such as the random access machine.

Computational Model: Random Access Machine

Since we are settling for bounds on the running time rather than a precise estimate, we can use a simplified model of computation such as the random access machine.

A random access machine is a very simple single-processor machine that executes instructions one after another without pipelining, speculative execution, branch prediction, or parallelism. It even lacks a memory hierarchy. Yet, still good prediction of running time.

We follow the approach taken in Cormen, Leiserson, Rivest, and Stein and assume that elementary operations take a constant – yet unspecified – amount of time. Then there is no need to translate the pseudocode into random access machine instructions, which simplifies matters.

Input Size

The running time of an algorithm is measured as a function of the size of the input. What is the size of the input? Answer depends on the application.

Input Size

The running time of an algorithm is measured as a function of the size of the input. What is the size of the input? Answer depends on the application.

We assume that we are given a function s that associates to an input x its size spxq.

Example If the input to a sorting algorithm is an array x with n elements, then spxq “ n would be a natural measure for the size of the input.

Input Size

Example If the input x is an integer, then spxq “ tlog 10 xu ` 1 may be a good measure of input size for algorithms that deal with long integer arithmetic.

Example If the integers remain bounded, then spxq “ 1 would be a reasonable choice.

How to Determine Worst-Case Running Time?

So how do we determine the worst-case running time of an algorithm in practice? The answer is that we simply analyze the algorithm line-by-line and essentially count operations.

The main complications are conditional statements, loops, and function or procedure calls.

We now put forth the axioms that govern the run-time analysis. The list is incomplete, but you will see the pattern and it should be easy to extend this list by further language constructs.

Elementary Operations

Axiom A1. Elementary operations have constant running time.

Compound Statements

The running time of a compound statement is bounded by the sum of the running times of the individual component statements. Axiom A2. If the statements S 1 and S 2 respectively have worst case running time T pS 1 q and T pS 2 q on an input of size n, then the compound statement S 1 ; S 2 has worst-case running time

T pS 1 ; S 2 q ď T pS 1 q ` T pS 2 q

Conditional Statements

The next axiom gives a coarse-grained view of the running-time of a conditional statement. We should add that sometimes it makes sense to distinguish case depending on the value of the condition C. Axiom A3. The conditional statement

if C then S 1 else S 2

has worst-case running time T pC q ` maxtT pS 1 q, T pS 2 qu when the conditional statement C and the statements S 1 and S 2 have worst case running time T pC q, T pS 1 q and T pS 2 q on an input of size n, respectively.

Do-While Loops

Axiom A5. The running-time of a do-while loop statement S given by begin S 1 end while C

has worst-case running time T pSq ď řmpnq k“ 1 pT^ pC^ q `^ T^ pS^1 ,^ kqq, where mpnq is the maximal number of loop iterations on an input of size n and T pC q and T pS 1 q are the worst-case running times of the conditional statement C and the statement S 1 , respectively.