Download Analysis of Boolean Functions: Hardness of Approximation and Quasirandomness and more Slides Computer Architecture and Organization in PDF only on Docsity!
Analysis of Boolean Functions (CMU 18-859S, Spring 2007)
Lecture 5: Introduction to Hardness of Approximation
Jan. 30, 2007 Lecturer: Ryan O’Donnell Scribe: Eric Blais
In this lecture, we introduce some of the tools that will enable us to prove strong statements about hardness of approximation in the next lecture.
1 PCPP builders and hardness of approximation
During the last lecture, we introduced the concept of probabilistically checkable proofs of proxim- ity (PCPP). To recall:
Definition 1.1 A property P of m -bit strings has PCPPs of length l(m) if there exists a non- adaptive O(1) -query tester T with access to a string w ∈ {− 1 , 1 }m^ and a proof π ∈ {− 1 , 1 }l(m) such that:
1. If w ∈ P , then ∃π such that Pr[T (w, π) accepts ] = 1 _, and
- If_ w is -far from P , then ∀π Pr[T (w, π) accepts ] ≤ 1 − Ω().
After introducing PCPPs, we saw that every property P of m-bit strings has PCPPs of length 22 m. While this may not look so bad given that there are 22 m possible properties, we will see in this section how we can generate shorter PCPPs when we restrict our attention to properties that are decided by polynomial-size circuits. Furthermore, unlike the existential proof that was presented in the last lecture, our proof of the strengthened result provides an explicit construction for the desired PCPPs. This proof will rely on PCPP builders.
Definition 1.2 A poly( m )-size circuit C over m bits has a length-l(m) PCPP builder if there is a poly( l(m) )-time algorithm that given C outputs a tester TC for a length- l(m) PCPP for the property C_. Here we are identifying a circuit_ C with the property of being a string satisfying C_._
Before examining some results on PCPP builders, we also introduce the notion of weighted constraint satisfaction problems.
Definition 1.3 A weighted constraint satisfaction problem (CSP) over variables v 1 ,... , vN is a ta- ble containing predicates φ 1 , φ 2 ,... that act on those variables and nonnegative weights p 1 , p 2 ,... associated with each predicate, such that
i pi^ = 1_._
The following table shows an example of a possible CSP:
weight constraints p 1 φ 1 (v 1 , v 6 , v 10 ) p 2 φ 2 (v 7 , v 10 , v 20 ) p 3 φ 3 (v 4 , v 19 ) .. .
An important algorithmic problem related to CSPs is the following: given an instance of a CSP, find the assignment of values to the variables v 1 ,... , vN that maximizes the sum of the weights whose corresponding predicates are satisfied. We can also construct a CSP to correspond to any nonadaptive tester TC. In this case, we let the variables represent the possible characters in the string(s) to be tested. The constraints in the CSP are the possible predicates that the tester may check on its input, with each weight representing the probability that the tester runs the associated predicate.
Theorem 1.4 There is a PCPP builder of length... (a) 22 m (see Theorem 2.3 in Lecture 4; the proof is constructive) (b) 2 poly(m)^ (based on BLR tests) (c) poly(m) (The PCP Theorem + ) (d) m · polylog(m) (Dinur, Ben-Sasson & Sudan; requires size( C ) = m · polylog(m) )
The major result (c) is obtained using the results from (a) (or (b)). The basic idea is that the construction for (c) is recursive, and the base case of the recursion involves building PCPPs for properties of constant-length strings. Thus we can use (a) or (b), since we don’t mind blowing up one constant to another. The proof of (a) constitutes perhaps a quarter to a third of Dinur’s proof of the PCP theorem.
Fact 1.5 The following is on the homework: If there exists a length- l(m) PCPP builder, then there exists a length- poly(l(m)) PCPP builder where all tests are OR 3 predicates (with negations); i.e., look like xi 1 ∨ xi 2 ∨ xi 3 or xi 1 ∨ xi 2 ∨ xi 3 or xi 1 ∨ xi 2 ∨ xi 3 etc.
The PCP Theorem has important implications for hardness of approximation results. To present these results precisely, we first define the notion of value of a CSP.
Definition 1.6 Given a CSP C , we define val(C) to be the maximum total weight that can be satisfied by any assignment.
The notion of value can be used to rephrase standard hardness results.
Observation 1.7 “3-SAT is NP-hard” is equivalent to “Given C where all constraints are OR 3 , it is NP-hard to distinguish val(C) = 1 and val(C) < 1 .”
Furthermore, the notion of value of CSPs also extends naturally to hardness of approximation results.
Example 2.4 Pr[ H˚astad δ(f ) accepts ] = 12 + (^12)
S (1^ −^2 δ)|S|^ fˆ^ (S)^3 ⇒ Pr[ H˚astad δ({f 1 ,... , fd}) accepts ] = 12 + (^12)
S
(1 − 2 δ)|S|ˆh(S)^3.
The proof of Proposition 2.3 uses a derivation very similar to the one we used to prove H˚astad’s test: Proof: (Sketch.)
E x,y,z,j,k,l
[^12 + 12 fj (x)fk(y)fl(z)] = (^) xE,y,z[ E j,k,l
[· · · ]]
= E
x,y,z
[^12 + 12 E
j,k,l
[fj (x)fk(y)fl(z)]]
= E x,y,z
[^12 + 12 E
j
[fj (x)] E k
[fk(y)] E l
[fl(z)]]
= (^) xE,y,z[^12 + 12 h(x)h(y)h(z)],
where we used linearity of expectation and the independence of j, k, l. The rest of the proof follows the proof of H˚astad’s test identically. 2
Proposition 2.3 actually holds for all tests because the acceptance predicate, being a function on {− 1 , 1 }q, can be expressed as a multilinear formula (the Fourier expansion!) and this multilinearity is all that we used in the proof sketch above.
Definition 2.5 Given f : {− 1 , 1 }n^ → {− 1 , 1 } , define f †^ by f †(x) = −f (−x) , and f odd^ : {− 1 , 1 }n^ → [− 1 , 1] by f odd^ = (f + f †)/ 2_._
Fact 2.6 As we saw in Question 5 of the first homework, the Fourier expansion of f odd^ is
f odd^ =
S:|S| odd
f^ ˆ (S)χS.
Allowing for a bad pun, we define the H˚ast-Oddδ test on boolean functions f :
Definition 2.7 The H˚ast-Odd δ test is defined to be to be
H˚ast-Odd δ(f ) := H˚astad δ({f, f †}).
Remark 2.8 Note that the H˚ast-Odd δ can be applied to f using only an oracle for f ; this oracle can be used to “simulate” queries to f †.
Applying Proposition 2.3, we get a formula for evaluating the probability that the H˚ast-Oddδ test accepts a given function.
Corollary 2.9 Pr[ H˚ast-Odd δ(f ) accepts ] = 12 + (^12)
|S| odd (1^ −^2 δ)
|S| (^) fˆ (S) (^3).
One interesting property of the H˚ast-Oddδ test is that it provides a third way to fix the dictator test introduced in Lecture 2, since this test accepts constant functions only with probability 1 / 2. As an aside: the H˚ast-Oddδ test is called “folding” the H˚astadδ test in the literature, but we will not use that terminology in these lectures.
3 Influence of variables
In the first lecture, we alluded to the influence of coordinates on functions. Let us now define the term formally.
Definition 3.1 For f : {− 1 , 1 }n^ → R , i ∈ [n] , the influence of i on f is
Infi(f ) =
S⊆[n] s.t. i∈S
f^ ˆ (S)^2.
Notation 3.2 From now on we will abbreviate that summation as “
S 3 i ”. In Lecture 1 we gave an alternative definition for influences in reference to boolean-valued functions f : {− 1 , 1 }n^ → {− 1 , 1 }, namely: Infi(f ) = Pr[f (x) 6 = f (x(i))]. As we show in the next proposition, the definition is equivalent in this special case:
Proposition 3.3 For any boolean-valued f : {− 1 , 1 }n^ → {− 1 , 1 } ,
Infi(f ) = Pr x [f (x) 6 = f (x(i))].
Proof: Briefly,
Pr[f (x) 6 = f (x(i))] = E[^12 − 12 f (x)f (x(i))] = 12 − 12 E[f (x)f (x(i))] = 12 − (^12)
S,T
f^ ˆ (S) fˆ (T ) E[xS (x(i))T ]
S,T
f^ ˆ (S) fˆ (T ) E[xS 4 T · (−1)^1 i∈T^ ]
Now (−1)^1 i∈T^ can be pulled out of the sum, and E[xS 4 T ] = 1 if S = T , 0 otherwise. Hence:
Pr[f (x) 6 = f (x(i))] = 12 − (^12)
S
f^ ˆ (S)^2 · (−1)^1 i∈S
By Parseval’s Theorem, we can replace the first 12 above with (^12)
S fˆ^ (S)
(^2) ; hence:
Pr[f (x) 6 = f (x(i))] = (^12)
S
f^ ˆ (S)^2 −
S
f^ ˆ (S)^2 · (−1)^1 i∈S
S 3 i
f^ ˆ (S)^2.
For some functions, such as the parity function, all variables have high influence. However, when this is the case we might prefer to say that none of the variables are influential. Toward this goal, we define the concept of attenuated influence :
To complete the proof, we now need to prove Claim 3.7. A nice proof of this claim was provided in class by Daniel Golovin: Since (1 − δ) < 1 , then (1 − δ)|S|−^1 ≤ (1 − δ)i−^1 for every i ≤ |S|. So
|S|(1 − δ)|S|−^1 ≤
| ∑S|− 1
i=
(1 − δ)i^ ≤
∑^ ∞
i=
(1 − δ)i^ =
δ
This completes the proof of the claim and of the proposition. 2
4 Quasirandomness
With the notion of influence of variables, we are now ready to formally define another term that was presented in the first lecture: quasirandomness.
Definition 4.1 Given f : {− 1 , 1 }n^ → [− 1 , 1] , we say that f is (, δ)-quasirandom if for all i ∈ [n] ,
Inf(1 i −δ)(f ) < .
In other words, f is (, δ) -quasirandom if |I,δ| = ∅.
One reason we call such functions “quasirandom” is that any such function is close to being uncorrelated with any function on a small number of bits; this is reminiscent of the related notion for graphs. See Homework #2 for details.
Example 4.2 Many functions we have seen so far satisfy the definition of quasirandomness.
Function Quasirandom? Dictator no Majority yes Parity yes Random yes Constant yes
Observation 4.3 The definition of quasirandomness becomes stricter as → 0 and δ → 0_._
The reason we introduce the definition of quasirandom functions is to make strong claims about the H˚ast-Oddδ test: specifically, that it rejects quasirandom functions with high probability.
Theorem 4.4 If h : {− 1 , 1 }n^ → [− 1 , 1] is (^2 , δ) -quasirandom,
Pr[ H˚ast-Odd δ(h) accepts ] ≤ 12 + 12 .
We will see the proof of this theorem in the next lecture^1. During that lecture, we will also show how the results we have derived above can be used to establish even stronger results about hardness of approximation. (^1) In fact, time constraints prevented us from covering the proof of this theorem in the lectures. See the “H˚ast-Odd as a Dictatorship vs. Quasirandom test” post of Jan. 31 on the course blog for a link to the proof.
