%Communication Complexity Scribe Notes template \ifx\CompleteCourse\relax \ClassScribeSetupA \else \documentclass[11pt]{article} \newtheorem{theorem}{Theorem} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{homework}{Homework} \newenvironment{definition}{\begin{trivlist}\item[]{\bf Definition}\ }% {\end{trivlist}} \newenvironment{fact}{\begin{trivlist}\item[]{\bf Fact}\ }% {\end{trivlist}} \newenvironment{example}{\begin{trivlist}\item[]{\bf Example}\ }% {\end{trivlist}} \newenvironment{proof}{\begin{trivlist}\item[]{\bf Proof}\ }% {\end{trivlist}} % Make the page large \addtolength{\textwidth}{1.50in} \addtolength{\textheight}{1.00in} \addtolength{\evensidemargin}{-0.75in} \addtolength{\oddsidemargin}{-0.75in} \addtolength{\topmargin}{-.50in} % \vdashsub{X} makes a turnstyle with subscript "X" % \vdashsup{X} makes a turnstyle with superscript "X" \newdimen\srbdimenA \newcommand{\vdashsupsub}[2]{ \mathop{ \setbox251 = \hbox{$\scriptstyle #1$} \setbox252 = \hbox{$\scriptstyle #2$} \ifdim \wd251<\wd252 \srbdimenA = \wd252 \else \srbdimenA = \wd251 \fi \setbox255 = \hbox {${\srbAvdash \vphantom( \kern -\srbdimenA \kern +.05em} ^{\hbox to\srbdimenA{\hfill \box251\hfill}} _{\hbox to\srbdimenA{\hfill \box252\hfill}}$} \box255 \kern .05em}} \newcommand{\srbAvdash}{\hbox{ \vrule height1.4ex width0.02em \dimen255 = \srbdimenA \advance\dimen255 by 0.1em \vbox{\hrule width\srbdimenA height0.02em \kern .65ex }}} \newcommand{\vdashsup}[1]{\vdashsupsub{{#1}}{\mbox{~}}} \newcommand{\vdashsub}[1]{\vdashsupsub{\mbox{~}}{#1}} \fi % FOR THE SCRIBE: CUSTOMIZE THE ENTRIES BELOW: % Fill in the following information particular to these scribe notes: \usepackage{amsmath} \usepackage{amssymb} \newcommand{\disc}{\mathrm{disc}} \newcommand{\RR}{\mathbb{R}} \newcommand{\ZZ}{\mathbb{Z}} \newcommand{\cv}[1]{\mathbf{1}_{#1}} \newcommand{\abs}[1]{\left\lvert#1\right\rvert} \newcommand{\norm}[1]{\left\lVert#1\right\rVert} \newcommand{\p}[1]{\left(#1\right)} \newcommand{\bra}[1]{\left[#1\right]} \newcommand{\T}[1]{{#1}^{\mathrm{T}}} \newcommand{\IP}{\mathrm{IP}} \newcommand{\sdeg}{\mathrm{sign\text{-}deg}} \newcommand{\edeg}[1][\epsilon]{#1\mathrm{\text{-}deg}} \newcommand{\DTC}{\mathrm{DTC}} \DeclareMathOperator*{\Exp}{\mathbb{E}} \def\scribeone{Yuval Filmus} % Who is the scribe? \def\classdate{Fall, 2014} % Date of the class \def\classnumber{4} % Is this the first, second, ...? % Here are some commands that stay the same for the whole class. \def\classinstructor{Toniann Pitassi} \def\classtitle{Communication Complexity} \def\doctitle{\textup{CS 2429 - Foundations of Communication Complexity}} \def\classid{\textup{Lecture \#\classnumber: \classdate}} % Put your macros for these scribe notes HERE % It is best to use as few as possible. % environments for "theorem", "corollary", "lemma", "fact" "definition" % "homework", "proof", "example", "proposition" % are already defined above. % Start the document \ifx\CompleteCourse\relax \ClassScribeSetupB \else \def\makeatletter{\catcode`\@=11\relax} \def\makeatother{\catcode`\@=12\relax} \makeatletter \def\ps@scribeheadings{\let\@mkboth\@gobbletwo \def\@oddhead{\sl\doctitle \hfill \classid }\def\@oddfoot{\hfil \rm \thepage \hfil}\def\@evenhead{\@oddhead}% \def\@evenfoot{\@oddfoot}\def\sectionmark##1{}\def\subsectionmark##1{}} \makeatother \pagestyle{scribeheadings} \begin{document} \bibliographystyle{siam} \fi \begin{center} \Large\bf\doctitle\\[1em] %\Large\bf\classid\\[1em] {\large\bf Lecturer: \classinstructor}\\[.5em] %{\large\bf Scribe Notes by: \scribeone} \end{center} \vspace*{.4in} \section{The Discrepancy Method --- Cont'd} In the previous lecture we've outlined the discrepancy method, which is a method for getting lower bounds on randomized communication complexity given upper bounds on the discrepancy of the matrix $M_f$ corresponding to the function in question. We showed how to bound the discrepancy using the largest eigenvalue of $M_f$. Today we will first give the BNS lemma which is another way of bounding the discrepancy of $M_f$. We denote the discrepancy of~$f$ (with respect to the uniform distribution) and a rectangle $A\times B$ by $\disc(f, A\times B)$. All our results can be generalized to arbitrary distributions by multiplying each entry of $M_f$ by the probability of the corresponding cell. Recall that Boolean functions can be considered as taking values in either $\{0,1\}$ or $\{+1,-1\}$. In this section, we will use the $\pm 1$ convention when describing the matrices and rectangles. We use the notation $\cv{A}$ for the characteristic vector of~$A$, which contains~$1$ in positions corresponding to the elements of~$A$, and 0's elsewhere. \subsection{The BNS Method} The BNS method is another way to bound the discrepancy, and will furnish us with yet another proof of the upper bound on $\disc(\IP_n)$. The method first appeared in a paper by Babai, Nisan and Szegedy. %\footnote{Szegedy is a Hungarian surname and so the digraph~\emph{sz} should be pronounced as an English~\emph{s} (IPA~$[s]$). When an English~\emph{sh} sound (IPA~$[\int]$) is intended, the monograph~\emph{s} is used, for example Erd\H{o}s.}. The method is given by the following lemma: \begin{lemma}[BNS] The discrepancy of a function~$f:X\times Y \rightarrow \ZZ_2$ can be bounded as follows: \[ \disc(f,A \times B)^2 \leq \Exp_{y,y'} \abs{\Exp_x M_f(x,y) M_f(x,y')}, \] where $x,y,y'$ are chosen independently and uniformly at random, $x$ from $X$ and $y,y'$ from $Y$. \end{lemma} \begin{proof} Recall the definition of discrepancy. $$\disc(f, A \times B)= \sum_{x \in A, y \in B} M_f(x,y)/2^{2n}.$$ The discrepancy can be written using expectations as \[ \disc(f, A \times B) = \abs{\Exp_{x,y} \cv{A}(x) \cv{B}(y) M_f(x,y)}. \] We can recast the Cauchy-Schwarz inequality in the form $\Exp[Z]^2 \leq \Exp[Z^2]$. Thus we can obtain: \begin{align*} \disc(f, A \times B)^2 &= \p{\Exp_x \cv{A}(x) \Exp_y \cv{B}(y) M_f(x,y)}^2 \\ &\leq \Exp_x \p{\cv{A}(x) \Exp_y \cv{B}(y) M_f(x,y)}^2 \\ &\leq \Exp_x \p{\Exp_y \cv{B}(y) M_f(x,y)}^2 \\ &= \Exp_x \p{\Exp_{y,y'} \cv{B}(y) \cv{B}(y') M_f(x,y) M_f(x,y')} \\ &= \Exp_{y,y'} \cv{B}(y) \cv{B}(y') \p{\Exp_x M_f(x,y) M_f(x,y')} \\ &\leq \Exp_{y,y'} \abs{\Exp_x M_f(x,y) M_f(x,y')}. \end{align*} \end{proof} The bound we get does not depend on the sizes of~$A$ and~$B$, and so it is slightly inferior to bounds which do (like Lindsey's lemma). In practice, the difference is usually insignificant (but is the subject of the final question in the first assignment!). We illustrate the method by proving yet again the upper bound on the discrepancy of the inner product function: \begin{lemma} We have $\disc(\IP_n, A \times B) \leq 2^{-n/2}$. \end{lemma} \begin{proof} The matrix corresponding to $\IP_n$ is $H_n$. The rows of~$H_n$ are orthogonal and so \[ \Exp_x H_n(x,y) H_n(x,z) = \begin{cases} 0 & \text{if } y \neq z, \\ 1 & \text{if } y = z. \end{cases} \] Using the BNS bound, \[ \disc(\IP_n,A \times B)^2 \leq \Exp_{y,z} \abs{\Exp_x H_n(x,y) H_n(x,z)} = \Pr[y=z] = 2^{-n}. \] \end{proof} The above theorem can also be proven with respect to an arbitrary distribution $\lambda$. The more general theorem is as follows. \begin{theorem} Let $F$ be a function from $X \times Y$ to $\{-1,1\}$. Then $$ \frac{Disc_{\lambda}(F) ^2}{|X|^2 \times |Y|^2} \leq E_{y,y'} | E_x f(x,y)f(x,y')\lambda(x,y) \lambda(x,y')|.$$ \end{theorem} \section{Degree/Discrepancy Method} The Degree/Discrepancy method, due to Sherstov, is a way to come up with other functions having high randomized communication complexity. The basic idea is to start with some other function (the ``base'' function) which is difficult under some other complexity measure, and to ``lift'' it to a function which is difficult in the randomized communication complexity model. Sherstov's main contribution is using polynomial complexity measures to quantify the difficulty of the base function. \subsection{Polynomial Complexity Measures} We will consider several different complexity measures for the base function. All of them try to capture the notion of being hard to approximate by a polynomial over the real numbers. Consider a Boolean function $f(x_1,\ldots,x_q)$. We will assume that the inputs and outputs are the usual $0/1$ (rather than $\pm 1$). This function can be represented as a real polynomial by following the following steps: \begin{enumerate} \item Present $f$ as a logical formula, e.g. conjunctive normal form. \item Convert the formula to a polynomial using the following rules: \begin{align*} \neg(x) &= 1-x, \\ x \wedge y &= xy, \\ x \vee y &= x+y-xy. \end{align*} \item Use the identity $x^2=x$ to reduce any repeated variables in the monomials. \end{enumerate} The result is some polynomial whose degree is at most~$q$, if $f$ is a $q$-CNF formula. This prompts the following definition: \begin{definition} The \emph{degree} (also \emph{polynomial degree}) of a function $f$, written $\deg(f)$, is the minimal degree of a real polynomial~$P$ such that~$f(x_1,\ldots,x_q) = P(x_1,\ldots,x_q)$ on all Boolean inputs. \end{definition} In general, it is difficult to represent functions exactly by polynomials, and so the fact that a function has high polynomial degree isn't strong enough for our purposes. A rather lenient alternative is the following: \begin{definition} The \emph{sign degree} (sometimes \emph{polynomial threshold degree}) of a function $f$, written $\sdeg(f)$, is the minimal degree of a real polynomial~$P$ such that for all Boolean inputs $x_1,\ldots,x_q$: \begin{itemize} \item If $f(x_1,\ldots,x_q)=1$ then $P(x_1,\ldots,x_q) > 0$. \item If $f(x_1,\ldots,x_q)=0$ then $P(x_1,\ldots,x_q) < 0$. \end{itemize} \end{definition} This definition is so permissive that it is hard to prove lower bounds on the sign degree. Here are two examples of functions for which a lower bound is known: \begin{itemize} \item The parity function on $q$ inputs has the maximal sign degree~$q$. \item The Minsky-Papert ``tribes'' function $\bigvee_{i=1}^m \bigwedge_{j=1}^{4m^2} x_{ij}$ has sign degree~$m=\sqrt[3]{q/4}$. \end{itemize} Lower bounding the sign degree can be difficult simply because a function with high polynomial degree can be sign-represented by a very low degree polynomial. An extreme example is the OR function (the logical inclusive or of all inputs). This function is sign-represented by the linear polynomial $\sum x_i - \frac{1}{2}$, but an exact representation necessitates a degree~$q$ polynomial. This prompts the need for some sort of an interpolation between these two extreme definitions. The following definition generalizes both previous ones: \begin{definition}[$\epsilon$-Approximation Degree] Given a real $0 \leq \epsilon \leq \frac{1}{2}$, the \emph{$\epsilon$-degree} (more officially, \emph{$\epsilon$-approximation degree}) of a function $f$, written $\edeg(f)$, is the minimal degree of a real polynomial~$P$ such that for all Boolean inputs, \[ \abs{f(x_1,\ldots,x_q) - P(x_1,\ldots,x_q)} \leq \epsilon. \] \end{definition} If $\epsilon=0$ this reduces to the regular degree, while if $\epsilon=\frac{1}{2}$ then this (almost) reduces to the sign degree. Clearly the $\epsilon$-degree is monotone decreasing in $\epsilon$, and so for general $0<\epsilon<\frac{1}{2}$ we have \[ 0 \leq \sdeg(f) \leq \edeg(f) \leq \deg(f) \leq q. \] As an example, the OR function, whose sign-degree is~$1$ and whose polynomial degree is~$q$, has $\epsilon$-degree $O(\sqrt{q})$ for $\epsilon=1/8$. Nisan and Szegedy related the $\epsilon$-degree to decision tree complexity, defined as follows: \begin{definition} A \emph{decision tree} for a Boolean function is a binary tree whose inner vertices are labelled by input variables, and whose leaves are labelled by~$0/1$. The computation outlined by the tree proceeds from the root by querying the labelled variable, taking the left branch if the respective variable is~$0$, the right branch if it is~$1$. Upon reaching a leaf, its label is output. The \emph{decision tree complexity} of a function $f$, written $\DTC(f)$, is the depth of the shallowest decision tree which represents it. \end{definition} Using the method outlined above for converting a formula into a real polynomial, one sees that the decision tree complexity upper bounds the polynomial degree. In particular, $\edeg(f) \leq \DTC(f)$. Nisan and Szegedy proved a matching upper bound: \[ \edeg(f) \leq \DTC(f) \leq \edeg(f)^8. \] Formulated differently, we have $\log\edeg(f) = \Theta\p{\log\DTC(f)}$. %\subsection{Looking ahead} %In the next class we will show how to turn a function with high polynomial degree complexity into a function with high randomized communication complexity. For now, we outline the procedure for a base function with high sign degree. The first observation is the following duality theorem, which we formulate for functions in the $\pm 1$ universe. % %\begin{theorem}[Duality of Sign Degree] %The function~$f(x_1,\ldots,x_q)$ satisfies $\sdeg(f) > d$ if and only if there exists a measure~$\mu$ on $\ZZ_2^q$ such that %\[ %\Exp_{x_1,\ldots,x_q \sim \mu}\bra{f(x_1,\ldots,x_q) \chi_S(x_1\ldots,x_q)} = 0 %\text{ for all } |S|\leq d, %\] where $\chi_S$, the Fourier character corresponding to $S$, is defined by %\[\chi_S(x_1,\ldots,x_q) = \prod_{i \in S} x_i.\] %\end{theorem} %Since the Fourier characters corresponding to all subsets of $S$ span all functions which depend only on inputs from~$S$, the condition in the theorem can be reformulated as %\[ \Exp_{x_1,\ldots,x_q \sim \mu}\bra{f(x_1,\ldots,x_q) g(x_1\ldots,x_q)} = 0 \] %for every function $g$ depending on at most $d$ coordinates. %We will prove this theorem, which is an easy application of linear programming duality, during the next lecture. % %\bigskip %Given this theorem, we will proceed according to the following outline: %\begin{enumerate} %\item Start with a base function $f$ with high sign-degree. %\item Lift it to a function $F(X,Y)$ defined as follows. The first player's input $X$ is a vector of length $N = q^{O(1)}$. The second player's input $Y$ is an indexing of all possible ordered choices of~$q$ bits out of~$N$. Thus $|Y|=\log N!/(N-q)!$. Each input $Y$ defines some sequence of non-repeating indices $0 \leq i_1,\ldots,i_q < N$. We define $F(X,Y) = f(x_{i_1},\ldots,x_{i_q})$. %\item Use duality to get a distribution, $\mu$, on the inputs of~$f$ under which~$f$ has zero correlation to all functions depending on few coordinates. %\item Lift this distribution to a distribution, $\lambda$, on $X\times Y$ with similar properties. %\item Use the BNS method to upper bound the discrepancy of~$F$ with %respect to $\lambda$, and deduce a lower bound on the randomized communication complexity. %\end{enumerate} %\end{document} \section{Discrepancy and Duality of Sign Degree} \begin{theorem}[Duality of sign degree] Let $f:\lbrace-1,1\rbrace^n$ $d \geq 0$\\ Then $sign$-$deg(f)$ is at least $d$ if and only if there exists a distribution $\mu$ over $\lbrace-1,1\rbrace^n$ such that\\ $E_{x\sim\mu}\left[ f(x)\cdot\chi_{S}(x)\right] = 0$ $\forall S$, $\vert S \vert < d$ \end{theorem} That is to say, ``f is orthogonal to $\chi_{S}$ for small s'', where $\chi_{S}$ is the parity function over the indices in $S$ \begin{theorem}[Duality of approximation degree] (Sherstov, Shi-Zhu)\\ Fix $\varepsilon \geq 0$. Let $f:\left\lbrace -1,1\right\rbrace ^n \rightarrow \left\lbrace -1, 1 \right\rbrace$, $deg_\varepsilon (f) = d \geq 1$. \\ Then $\exists g: \left\lbrace -1,1\right\rbrace ^n \rightarrow \left\lbrace -1, 1 \right\rbrace$ and a distribution $\mu$ over $\left\lbrace -1,1 \right\rbrace ^n$ such that:\\ \[(1) \quad E_{x \sim \mu} \left[ g(x) \chi_S(x) \right] = 0 \quad \forall S \quad \vert S \vert \leq d \] \[(2) \quad corr_\mu (f,g) > \varepsilon \quad \quad ( corr_\mu (f,g) = E_{x\sim \mu} [f(x)g(x)] ) \] \end{theorem} \begin{proof} {\bf (Duality of sign degree)} This is an instance of the ``Gordon Transposition Lemma''\\ Let $A$ be a matrix of dimension $m\times n$. Then $\exists \vec{u}$ s.t. $\vec{u}^{T}A > 0$ iff $\exists \vec{v} > 0$ s.t. $A\vec{v}=0$\\ We want a polynomial $f'$ which sign-approximates $f$. We look for coefficients $\alpha_{s}$, $\vert S \vert < d$ to produce $f' = \sum_{S} \alpha_{s} \chi_{s}$ \\ Fix $\rho$. If $f(\rho)=1$ $\sum_{S} \alpha_{s} \chi_{s} > 0$, and if $f(\rho)=-1$ $\sum_{S} \alpha_{s} \chi_{s} < 0$. So, $\sum \alpha_{s} \chi_{s} f(\rho) > 0$, that is to say, they match in sign.\\ We construct a matrix with columns representing values for $\rho$ and rows representing values for s, that is, subsets of $1..n$ of size $\leq d$. For each value we fill in $\chi_{s}(\rho) f(\rho)$. Then the rows of our matrix are the values for $\alpha_{s}$, which is $\vec{u}^{T}$ in the above lemma, and $\vec{v}$ is a distribution over our columns. \end{proof} Using duality of sign degree we can prove 2-party communication complexity lower bounds. The outline of the argument is as follows. \begin{itemize} \item[(1)] We start with a base function $f:\lbrace-1,1\rbrace^n$ with large sign degree $d$. For example, $f(x) = \bigvee^{m}_{i=1}\bigwedge^{4m^2}_{j=1}x_{ij}$ has sign-degree $m$, or the parity function, with sign degree $n$.\\ \item[(2)] Use the pattern matrix method to "lift" $f$ to obtain a 2-player communication complexity problem $F(\bar{x},\bar{y})$ $|\bar{x}| = N$ and $|\bar{y}| = \log{N \choose n}$, $N = O(n^k)$. $F(\bar{x},\bar{y}) = f(\bar(x)|_{\bar{y}})$, which is read ``f of $\bar{x}$, restricted to the bits specified by $\bar{y}$'' \\ That is, Alice has $N$ bits ($N$ will be chosen to be polynomial in $n$), and Bob has $\log {N \choose n}$ many bits. We interpret Bob's input as pointing to $n$ locations of Alice's string. They want to compute the function $f$ on these $n$ (consecutive) bits. \item[(3)] By duality of sign degree, there exists a distribution $\mu$ over $\{-1,1\}^n$ such that $f$ is orthogonal to all $\chi_{S}$, $|S| < d$, with respect to $\mu$. Extend $\mu$ to a distribution $\lambda$ over the domain of $F$ in the natural way. Then by orthogonality, the BNS Lemma will imply small discrepancy (discrepancy less than $2^{-d}$) for $F$ with respect to $\lambda$. \end{itemize} Using the above plan, we will prove the following theorem: \begin{theorem} (Sherstov) Let $f$ be boolean over $x_{1}..x_{n}$ with sign degree $\geq d$.\\ Then $disc(F) \leq (\frac{4en^{2}}{Nd})^{\frac{d}{2}}$ where e has its usual meaning as the base of the natural logarithm. \end{theorem} We set $N = \frac{16en^{2}}{d}$ so that $disc \leq 2^{-d}$. See Sherstov, Seperating $AC^0$ from depth-2 majority circuits, and Sherstov, Pattern Matrix Method. %The proof will use the BNS Lemma, proven earlier. %\begin{lemma}(BNS Lemma) %$F(X\times Y) \rightarrow \lbrace -1, 1 \rbrace$ $|X| = 2^N$ $|Y| = 2^N$ \\ %\[disc_{\lambda}(F)^{2} \leq E_{y,y' \in Y} [|E_{x \in X} [F(x,y) F(x,y')]|] = E_{y,y' \in Y} \left| \sum_{x \in X} \lambda(x,y)\lambda(x,y')F(x,y)F(x,y') \right| \]\\ %\end{lemma} \begin{proof} (Proof of Sherstov's theorem) We rename $y,y'$ $V$ and $W$. \medskip \noindent {\bf Extending $\mu$ to $\lambda$:} $\lambda$ is a distribution on $X \times Y$ induced by $\mu$. To obtain $\lambda$ we pick $V \in Y$ uniformly at random. We choose $x|_V$ according to $\mu$. Then we set the rest of the bits of $x$ uniformly at random. So we have: $$\lambda(x,V) = 2^{-N + n} \mu(x|_V)/{N \choose n}.$$ \medskip By the BNS lemma, $$\frac{disc_{\lambda}(F)^2}{|X|^2 \times |Y|^2} \leq E_{V,W} | E_x [f(x|_V)f(x|_W) \lambda(x,V) \lambda(x,W)]|$$ Rewriting in terms of $\mu$ we get $$disc_{\lambda}(F)^2 \leq 4^n E_{V,W} |E_x [f(x|_V)f(x|_W)\mu(x|_V)\mu(x|_W)]|.$$ Let $\Gamma(V,W)$ denote $E_x[ f(x|_V)f(x|_W)\mu(x|_V)\mu(x|_W)]$. \medskip %4^n E_{V,W\sim U} |\Gamma(V,W)|\] where \[\Gamma(V,W) = E_{x\sim U} \left[\mu(x|_V)\mu(x|_W)f(x|_V)f(x|_W)\right]\]\\ {\bf Claim 1} When $|V\cap W| \leq d-1$ then $\Gamma(V,W)=0$. \\ {\bf Claim 2} When $|V\cap W| = i$, $|\Gamma(V,W)| \leq 2^{i-2n}$. \\ By these claims, \[disc_{\lambda}(F)^2 \leq \sum_{k=d}^{n} 2^{k}Pr\left[|V\cap W| = k \right]\] \[ Pr\left[|V\cap W| = k \right] = \frac{ {n \choose k} {N-n \choose n-k} }{ {N \choose n}} \leq \left(\frac{en^2}{Nk}\right)^k\] (The above inequality uses ${n \choose k} \leq (en/k)^k$.) \[disc_{\lambda}(F)^2 \leq \sum_{k=d}^{n} 2^k \left(\frac{en^2}{Nk}\right)^k = \sum_{k=d}^{n} \left(\frac{2en^2}{Nk}\right)^k \leq \left(\frac{4en^2}{Nd}\right)^{d}\]. %The last inequality \end{proof} \begin{proof} of Claim 1 The basic idea here will be that by orthogonality, the expectation is zero. Let $V$ be $x_1 ... x_n$ (for notational convenience). \[\Gamma(V,W) = E_{x} \left[\mu(x_1 ... x_n)f(x_1 ... x_n)\mu(x|_W)f(x|_W)\right]\] \[\Gamma(V,W) = \frac{1}{2^N} \sum_{x_1..x_n} \mu(x_1..x_n)f(x_1..x_n)\sum_{x_{n+1}..x_N}\mu(x|_W)f(x|_W) \] \[\Gamma(V,W) = \frac{1}{2^N} E_{x_1..x_n \sim \mu} f(x_1..x_n)\left[\sum_{x_{n+1}..x_N}\mu(x|_W)f(x|_W)\right] \] $\sum_{x_{n+1}..x_N}\mu(x|_W)f(x|_W)$ depends on $\leq d$ bits, so \[\Gamma(V,W) = 0 \] \end{proof} \begin{proof} of Claim 2 We want to show that if $|V\cap W| = i$, then $|\Gamma(V,W)| \leq 2^{i-2}$. Again for notational convenience we will assume that $V=\{1,2,\ldots,n\}$ and $W=\{1,2,\ldots,i\} \cup \{n+1,n+2,\ldots,n+(n-i)\}$. Then we have: \[|\Gamma(V,W)| \leq E_x[|f(x|_V)\mu(x|_V) f(x|_W)\mu(x|_W)|] \] \[|\Gamma(V,W)| \leq E_{x_1,\ldots,x_{2n-i}} [\mu(x_1,\ldots,x_n)\mu(x_1,\ldots,x_i,x_{n+1},\ldots,x_{2n-i}] \] \[|\Gamma(V,W)| \leq E_{x_1,\ldots,x_n}[\mu(x_1,\ldots,x_n)] \cdot max_{x_1,\ldots,x_i} E_{x_{n+1},\ldots,x_{2n-i}} [\mu(x_1,\ldots,x_i,x_{n+1},\ldots,x_{2n-i})] \] The first quantity above, $E_{x_1,\ldots,x_n}[\mu(x_1,\ldots,x_n]$ is at most $2^{-n}$ because $\mu$ is a probability distribution, and similarly the second expectation in the last equation is at most $2^{-(n-i)}$ again because $\mu$ is a probability distribution. \end{proof} % E_{x_1..x_n}\left[\mu(x_1..x_n)\right]\cdot\max_{x_1..x_n} E_{x_{n+1}..x_{2n-i}}\left[\mu(x_1..x_i x_{n+1}..x_{2n-i})\right] \] %where we assume that $f(x_1..x_i x_{n+1}..x_{2n-i}) = 1$ because we're searching for a maximal value.\\ %$E_{x_1..x_n}\left[\mu(x_1..x_n)\right] = 2^{-n}$ and $ E_{x_{n+1}..x_{2n-i}}\left[\mu(x_1..x_i x_{n+1}..x_{2n-i})\right] \leq 2^{-n-i}$ so %\[|\Gamma(S,T)| = 2^{i-2n}\] %\end{proof} \section{Application to Circuits} In 1989, Allender proved the following theorem, showing that any $AC^0$ function can be computed by quasipolynomial-size depth-3 majority circuits. \begin{theorem}[Allender] Any $AC^0$ function can be computed by a depth-3 majority circuit of quasipolynomial ($O(n^{polylog(n)}$) size. \end{theorem} An open question was whether or not his result could be improved. In particular, is it possible to improve the depth, showing that every function in $AC^0$ be computed by depth-2 majority-of-threshold circuits of quasipolynomial size? A corollary to Sherstov's theorem is a negative resolution to this open problem: \begin{theorem}[Sherstov] $\exists F \in AC^0_3$ (depth 3) whose computation requires majority of exponentially many threshold gates. \end{theorem} It suffices to show an $AC^0$ function with exponentially small discrepancy. We start with the $AC^0_2$ function: \[f=\bigvee_{i=1}^m \bigwedge_{j=1}^{4m^2} e_{ij}\] We construct F(x,y) where $F(x,y) = f(x|_y)$, that is, f of the bits of x specified by y. F(x,y) is in $AC^0_3$: \[F(x,y) = \bigvee_{i=1}^m \bigwedge_{j=1}^{4m^2}\bigvee_{\alpha}\left( y_{ij\alpha_1} \wedge y_{ij\alpha_2} \wedge ... \wedge y_{ij\alpha_q} \wedge x_{ij\alpha}\right) \] because we can swap the order of the $\wedge$'s within the brackets with the last $\bigvee$ and then merge them with the middle $\bigwedge$.\\ By the degree/discrepancy theorem we know that because f requires a high degree polynomial to compute, F(x,y) has low discrepancy. Each threshold gate can be computed by a $O(\log n)$ bit probabilistic CC protocol with $R^{pub}_\epsilon(f) = O(\log n + \log \frac{1}{\epsilon})$.\\ Suppose F has (low) discrepancy $e^{-N^\varepsilon}$. Then any randomized protocol requires $N^\varepsilon$ bits. Also let $F = MAJ(h_1 .. h_S)$ where each $h_i$ is a threshold circuit.\\ The players pick a random $i\in [S]$. They evaluate $h_i$, using $O(\log n)$ bits and output the result. \\ The probability of correctness of the threshold-computing protocol is $1 - \frac{1}{4S}$ if we set $\varepsilon' \sim \frac{1}{S}$.\\ The total cost is $O(\log n) + \log S$ bits. The probability of correctness is $(\frac{1}{2} + \frac{1}{2S}) - \frac{1}{4S} = \frac{1}{2} + \frac{1}{4S}$ on every input.\\ Since we know that F requires $O(N^\varepsilon)$ bits to compute, S must be exponentially large! And so there is no polynomially-sized majority-of-threshold circuit to compute $F \in AC^0_3$. \section{Extensions of Sherstov} \subsection{High approximation degree to high probabilistic communication complexity} First, the above theorem can be generalized to prove lower bounds on 2-party communication complexity of lifted functions where the base function has high $\epsilon$-approximate degree, rather than high sign degree. The idea here is to replace the duality theorem for sign degree by the duality theorm for approximate degree. We follow the same three steps, showing that if $f$ (the base function) has large approximate degree, then there exists a function $g$ that is highly correlated with $f$, and a distribution $\mu$ such that $g$ is orthogonal to all low degree characters with respect to $\mu$. We then lift $g$ to a two-party communication complexity problem $G$, and lift $\mu$ to a distribution $\lambda$ over $G$ to show (using orthogonality and BNS) that $G$ has low discrepancy. Finally, since $f$ is highly correlated with $g$, $F$ is highly correlated with $G$, and thus it follows that $F$ also has high randomized communication complexity. \subsection{NOF lower bounds} The above ideas can also be extended to prove lower bounds in the NOF model as well. The BNS lemma stated above can be generalized straightforwardly to prove a similar lemma in the NOF case. Its generalization for $k=3$ looks like this: $$disc(F)^{2^{2}} \leq E_{y_1, y_1' \in Y_1} E_{y_2,y_2' \in Y_2} | E_{x \in X} f(x,y_1,y_2)f(x,y_1,y_2')f(x,y_1',y_2)f(x,y_1',y_2')|.$$ More generally for arbitrary $k$ we will have a similar expression, but where the LHS is raised to the power $2^{k-1}$. Using this stronger BNS lemma, one can prove a similar general theorem following the basic outline that we presented. Note that for $k=\log n$ players, the bound becomes trivial. It is a longstanding open problem to prove a NOF communication complexity bound for an explicit function (say in $NP$) for more than $\log n$ many players. \end{document}