Download Algorithm Complexity: Matrix Inversion, Integer Arithmetic, and Convex Hull and more Slides Computer Science in PDF only on Docsity!
Reductions
Contents
Contents.^ n
"Linear-time reductions." n^
Undirected and directed shortest path. n^
Matrix inversion and multiplication. n^
Integer division and multiplication. n^
Sorting and convex hull.
3
Reduction
Intuitively, decision problem X reduces to problem Y if:^ n
Any instance of X can be "rephrased" as an instance of Y. n^
The solution to instance of Y provides solution to instance of X. Consequences:^ n
Used to establish relative difficulty between two problems. n^
Given algorithm for Y, we can also solve X. (design algorithms) n^
If X is hard, then so is Y. (prove intractability)
Reduction
Problem X linearly reduces to problem Y if, given a black box thatsolves Y in O(f(N)) time, we can devise an O(f(N)) algorithm for X.Ex 1. X = PRIME linearly reduces to Y = COMPOSITE.^ n
PRIME(x): Is x prime? n^
COMPOSITE(x): Is x composite? n^
To compute PRIME(x), call COMPOSITE(x) and return oppositeanswer.
Docsity.com
5
Reduction: Undirected to Directed Shortest PathEx 2. Undirected shortest path (with nonnegative weights) linearlyreduces to directed shortest path.^ n
Replace each directed arc by two undirected arcs. n^
Shortest directed path will use each arc at most once.
s
2 3
5 6
t
10 5
(^91512)
10 12
15
4
s
2 3
5 6
t
10 5
(^91512)
(^1012)
4
9
10
4
15 12
12
10
15 15
Reduction: Undirected to Directed Shortest PathEx 2. Undirected shortest path (with nonnegative weights) linearlyreduces to directed shortest path.^ n
Replace each directed arc by two undirected arcs. n^
Shortest directed path will use each arc at most once. n^
Note: reduction invalid in networks with negative cost arcs, even ifno negative cycles.
t
2
s^
7
t
2
s^
7
7
7
Network Flow Running Times and Linear Time Reductions
undirected shortest pathnonnegative weights
O(m)
shortest path nonnegative weights
O(m + n log n)
undirected shortest path
no negative cyclesO(mn + n
2 log n)
shortest path no negative cycles
O(mn)
assignment
(weighted bipartite matching)
O(mn + n
2 log n)
weighted non-bipartite matchingO(mn + n
2 log n)
directed MSTO(m + n log n)
MST undirected O(m
α(m,n) log
α(m,n))
non-bipartitematching O(mn
1/2 )
bipartite matching
O(mn
1/2 )
max flow bipartite DAGO(mn log(m/ n
2 ))
max flow O(mn log(m/ n
2 ))
min cut O(mn log(m/ n
2 ))
max flowundirected
min cutundirected
min cost flow O(m
2 log n + mn log
2 n)
transportation O(m
2 log n + mn log
2 n)
min vertex cover
bipartiteO(mn
1/2 )
Matrix Inversion
Fundamental problem in numerical analysis.^ n
Intimately tied to solving system of linear equations. n^
Note: avoid explicitly taking inverses in practice.
1 2 3
x x x
x
b
A
3
2
1
3
2
1
3
2
1
3
2
1
x
x
x
x
x
x
x
x
x
x
x
x
=^
−
−
1
1
b
A
x
A
Docsity.com
13
Integer Arithmetic
Fundamental questions.^ n
Is integer addition easier than integer multiplication? n^
Is integer multiplication easier than integer division? n^
Is integer division easier than integer multiplication?
OperationAddition
O(N)
Upper Bound
(N)
Lower Bound
Multiplication
O(N log N log log N)
(N)
Division
O(N log N log log N)
(N)
Warmup: Squaring vs. Multiplication
Integer multiplication: given two N-digit integer s and t, compute st.Integer squaring: given an N-digit integer s, compute s
Theorem. Integer squaring and integer multiplication have the sameasymptotic complexity.Proof.^ n
Squaring linearly reduces to multiplication.^ –
trivial: multiply s and s n^
Multiplication linearly reduces to squaring.^ –
regularity assumption: S(N+1) = O(S(N))
((^
2 2 2
12
t s t s
st
15
Integer Division (See Kozen, Chapter 30)
Integer division: given two integers s and t of at most N digits each,compute the quotient q and remainder r:^ n
q =
s / t
^ , r = s mod t.
n^
Alternatively, s = qt +r, 0
r < t.
Example.^ n
s = 1000, t = 110
q = 9, r = 10.
n^
s = 4905648605986590685, t = 100
r = 85.
We show integer division linearly reduces to integer multiplication.
Integer Division: "Grade-School"
Divide two integers, eachis N bits or less.^ n
q =
s / t
n^
r = s mod t. Running time. O(N
n^
O(N) per iteration + recursive calls. n^
Denominator increases by factor of 2 each iteration.^ –
s < 2
N^ and does not change
-^
≤^
t^ ≤
s throughout
⇒
O(N) recursive calls
IF (s < t)
RETURN (0, t) (q’, r’)
IntegerDivision(s, 2t)
IF (r’ < t)
RETURN(2q’, r’) ELSE
RETURN (2q’ + 1, r’ - t)
(q, r) = IntegerDivision (s, t)
Docsity.com
17
Integer Division: "Grade-School"
The algorithm correctly compute q =
s / t
^ , r = s mod t.
Proof by reverse induction.^ n
Base case: t > s. n^
Inductive step: algorithm computes q’, r’ such that^ –
q’ =
s / 2t
^ , r’ = s mod 2t.
-^ s = q’ (2t) + r’, 0
r’ < 2t.
n^
Goal: show
otherwise 1
2
if
2 q
t r
q
s t
^
^
^
r t
q
t
r t q
s t
Newton’s Method
Given a differentiable function f(x), find a value x* such that f(x*) = 0.Newton’s method.^ n
Start with initial guess x
n^
Compute a sequence of approximations: n^
Equivalent to finding line of tangent to curve y = f(x) at x
andi^
taking x
i+
to be point where line crosses x-axis.
1
i^ i
i
i^
x
f
x
f
x
x^
xi
xi+
19
Newton’s Method
Convergence of Newton’s method.^ n
Not guaranteed to converge to a root x*. n^
If function is well-behaved, and x
0 sufficiently close to x* then
Newton’s method converges quadratically.^ –
number of bits of accuracy doubles at each iteration
Applications.^ n
Computing square roots: n^
Finding min / max of function.
Extends to multivariate case.
n^
Cornerstone problem in continuous optimization. n^
Interior point methods for linear programming.
(^12)
1
2
txi
i
i
x
x
x t
x f
Integer Division: Newton’s Method
Our application of Newton’s method.^ n
We will use exact binary arithmetic and obtain exact solution. n^
Approximately compute x = 1 / t using Newton’s method. n^
We’ll show exact answer is either
s x
^
or
s x
Theorem: given a O(M(N)) algorithm for multiplying two N-digitintegers, there exists an O(M(N)) algorithm for dividing two integers,each of which is at most N-digits.
2
1
i
i
i^
tx
x
x
x
t
x
f
Docsity.com
25
Analysis
L3: Algorithm terminates after O(log N) steps.^ n
By L2, after k =
log
log 2
(s / t) 2
steps, we have:
Note: 2
k^ = O(N), k = O(log N).
L4: Algorithm returns correct answer.^ n
By L1, x
k^
≤^
1 / t.
n^
Combining with proof of L3: n^
This implies,
s / t
^
is either
s x
k or
s x
k
the remainder can be found by subtraction.
≤^
k
sx
s t
2
t s
tx
k
k^
Analysis
Theorem: Newton’s method does integer division in O(M(N)) time,where M(N) is the time to do multiply two N-digit integers.^ n
By L3, 2
k^ = O(N), and the number of iterations is O(log N).
n^
Each Newton iteration involves two multiplications, one addition,and one subtraction. n^
Technical fact (not proved here): algorithm still works if we onlykeep track of 2
i^ significant digits in iteration i.
Bottleneck operation = multiplications. 3
2M(1) + 2M(2) + 2M(4) +... + 2M(
k ) = O(M(N)).
2
1
i
i
i^
tx
x
x
x
t
x
f
27
Integer Arithmetic
Theorem: The following integer operations have the same asymptoticbit complexity.^ n
Multiplication. n^
Squaring. n^
Division. n^
Reciprocal: N-significant bit approximation of 1/s.
^
− N^ s
1 (^22)
Sorting and Convex Hull
Sorting.^ n
Given N distinct integers, rearrange in increasing order. Convex hull.^ n
Given N points in the plane, find their convex hull in counter-clockwise order.
Find shortest fence enclosing N points.
Docsity.com
29
Sorting and Convex Hull
Sorting.^ n
Given N distinct integers, rearrange in increasing order. Convex hull.^ n
Given N points in the plane, find their convex hull in counter-clockwise order. Lower bounds.^ n
Recall, under comparison-based model of computation, sorting Nitems requires
(N log N) comparisons.
n^
We show sorting linearly reduces to convex hull. n^
Hence, finding convex hull of N points requires
(N log N)
comparisons.
Sorting Reduces to Convex Hull
Sorting instance:Convex hull instance.Key observation.^ n
Region {x : x
2
≥^
x} is convex
all points are on hull. n^
Counter-clockwise order ofconvex hull (starting at pointwith most negative x) yieldsitems in sorted order.
(^
2
22 2
(^21) 1
N N^
x x
x x
x x^
u
x^ N
x x^
,^
2 1
u
f(x) = x
(^2) ) 2 , (
xi xi^2 ) , (
xj xj
Docsity.com
