Order Statistics and Selection Algorithm: Finding the Kth Order Statistic, Thesis of Engineering

Download Order Statistics and Selection Algorithm: Finding the Kth Order Statistic and more Thesis Engineering in PDF only on Docsity!

Divide-and-Conquer Algorithms

Part Four

Announcements

●

Problem Set 2 due right now.

● Can submit by Monday at 2:15PM using one late period. ●

Problem Set 3 out, due July 22.

● (^) Play around with divide-and-conquer algorithms and recurrence relations! ● Covers material up through and including today's lecture.

Outline for Today

●

The Selection Problem

● A problem halfway between searching and sorting. ●

A Linear-Time Selection Algorithm

● (^) A nonobvious algorithm with a nontrivial runtime. ●

The Substitution Method

● (^) Solving recurrences the Master Theorem can't handle.

Order Statistics

● Given a collection of data, the k th order statistic is the k th smallest value in the data set. ● For the purposes of this course, we'll use zero-indexing, so the smallest element would be given by the 0 th order statistic. ● To give a robust definition: the k th order statistic is the element that would appear at position k if the data were sorted.

An Initial Solution

●

Any ideas how to solve this?

●

Here is one simple solution:

● Sort the array. ● Return the element at the k th position. ●

Unless we know something special about

the array, this will run in time O( n log n ).

●

Can we do better?

A Useful Subroutine: Partition

● Given an input array, a partition algorithm chooses some element p (called the pivot ), then rearranges the array so that ● All elements less than or equal to p are before p. ● (^) All elements greater p are after p. ● p is in the position it would occupy if the array were sorted. ● The algorithm then returns the index of p. ● We'll talk about how to choose which element should be the pivot later; right now, assume the algorithm chooses one arbitrarily.

Partitioning an Array

Partitioning and Selection

● There is a close connection between partitioning and the selection problem. ● Let k be the desired index and p be the pivot index after a partition step. Then: ● If p = k , return A[ k ]. ● If p > k , recursively select element k from the elements before the pivot. ● If p < k , recursively select element ( k – p – 1) from the elements after the pivot.

Some Facts

●

The partitioning algorithm on an array of

length n can be made to run in time Θ( n ).

● Check the Problem Set Advice handout for an outline of an algorithm to do this. ●

Partitioning algorithms give no

guarantee about which element is

selected as the pivot.

●

Each recursive call does Θ( n ) work, then

makes a recursive call on a smaller array.

Analyzing the Runtime

● The runtime of our algorithm depends on our choice of pivot. ● In the best-case, if we pick a pivot that ends up at position k , the runtime is Θ( n ). ● In the worst case, we pick always pick pivot that is the minimum or maximum value in the array. The runtime is given by this recurrence:

T(1) = Θ(1)

T( n ) = T( n – 1) + Θ( n )

T(1) = Θ(1)

T( n ) = T( n – 1) + Θ( n )

The Story So Far

●

If we have no control over the pivot in

the partition step, our algorithm has

runtime Ω( n ) and O( n

2

●

Using heapsort, we could guarantee

O( n log n ) behavior.

●

Can we improve our worst-case bounds?

Finding a Good Pivot

●

Recall: We recurse on one of the two

pieces of the array if we don't

immediately find the element we want.

●

A good pivot should split the array so

that each piece is some constant fraction

of the size of the array.

● (^) (Those sizes don't have to be the same, though.)

Analyzing the Runtime

●

Our algorithm

● Recursively calls itself on the first 2/3 of the array. ● Runs a partition step. ● Then, either immediately terminates, or recurses in a piece of size n / 3 or a piece of size 2 n / 3. ●

This gives the following recurrence:

T(1) = Θ(1)

T( n ) ≤ 2T(2 n / 3) + Θ( n )

T(1) = Θ(1)

T( n ) ≤ 2T(2 n / 3) + Θ( n )

Analyzing the Runtime

●

We have the following recurrence:

●

Can we apply the Master Theorem?

●

What are a , b , and d?

● a = 2 , b = 3 / 2 , and d = 1.

● Since log

3 / 2

2 > 1, the runtime is

T(1) = Θ(1)

T( n ) ≤ 2T(2 n / 3) + Θ( n )

T(1) = Θ(1)

T( n ) ≤ 2T(2 n / 3) + Θ( n )

O( n

log 3 / 2 2

) ≈ O( n