%Communication Complexity Scribe Notes template \ifx\CompleteCourse\relax \ClassScribeSetupA \else \documentclass[11pt]{article} % Make the page large \addtolength{\textwidth}{1.50in} \addtolength{\textheight}{1.00in} \addtolength{\evensidemargin}{-0.75in} \addtolength{\oddsidemargin}{-0.75in} \addtolength{\topmargin}{-.50in} % \vdashsub{X} makes a turnstyle with subscript "X" % \vdashsup{X} makes a turnstyle with superscript "X" \newdimen\srbdimenA \newcommand{\vdashsupsub}[2]{ \mathop{ \setbox251 = \hbox{$\scriptstyle #1$} \setbox252 = \hbox{$\scriptstyle #2$} \ifdim \wd251<\wd252 \srbdimenA = \wd252 \else \srbdimenA = \wd251 \fi \setbox255 = \hbox {${\srbAvdash \vphantom( \kern -\srbdimenA \kern +.05em} ^{\hbox to\srbdimenA{\hfill \box251\hfill}} _{\hbox to\srbdimenA{\hfill \box252\hfill}}$} \box255 \kern .05em}} \newcommand{\srbAvdash}{\hbox{ \vrule height1.4ex width0.02em \dimen255 = \srbdimenA \advance\dimen255 by 0.1em \vbox{\hrule width\srbdimenA height0.02em \kern .65ex }}} \newcommand{\vdashsup}[1]{\vdashsupsub{{#1}}{\mbox{~}}} \newcommand{\vdashsub}[1]{\vdashsupsub{\mbox{~}}{#1}} \fi % FOR THE SCRIBE: CUSTOMIZE THE ENTRIES BELOW: % Fill in the following information particular to these scribe notes: \def\scribeone{Tasos Zouzias and Toniann Pitassi} % Who is the scribe? \def\classdate{Fall, 2014} % Date of the class \def\classnumber{1} % Is this the first, second, ...? % Here are some commands that stay the same for the whole class. \def\classinstructor{Toniann Pitassi} \def\classtitle{Communication Complexity} \def\doctitle{\textup{CS 2429 - Communication Complexity: Applications and New Directions}} \def\classid{\textup{Lecture \#\classnumber: \classdate}} % Put your macros for these scribe notes HERE % It is best to use as few as possible. % environments for "theorem", "corollary", "lemma", "fact" "definition" % "homework", "proof", "example", "proposition" % are already defined above. \usepackage{amsthm,amsmath,amssymb,graphicx,epsfig,color} \usepackage{enumerate} % Reverse Index (remove before final edition) % Remove the following %\usepackage[active]{srcltx} \newcommand{\eps}{\varepsilon} \newcommand{\ignore}[1]{} \def\C {\mathcal{C}} \def\RR {\mathbb{R}} \renewcommand{\SS}{\mathbb{S}} \def\EE {\mathbb{E}} \newtheorem{definition}{Definition} \newtheorem{theorem}{Theorem} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{question}{Question} \newtheorem{claim}[theorem]{Claim} \newtheorem{conjecture}[theorem]{Conjecture} \newtheorem{observation}[theorem]{Observation} \newtheorem{fact}[theorem]{Fact} \newtheorem{example}{Example} %\newtheorem{algorithm}{Algorithm} \newtheorem{assumption}[theorem]{Assumption} \newtheorem{remark}{Remark} \newtheorem{problem}{Problem} %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% \usepackage[backref=page]{hyperref} \hypersetup{ unicode=false, % non-Latin characters in Acrobat’s bookmarks pdftoolbar=true, % show Acrobat’s toolbar? pdfmenubar=true, % show Acrobat’s menu? pdffitwindow=true, % page fit to window when opened pdftitle={Introduction to Communication Complexity}, % title pdfsubject={Communication Complexity}, % subject of the document pdfcreator={Toniann Pitassi}, % creator of the document pdfproducer={}, % producer of the document pdfkeywords={}, % list of keywords pdfnewwindow=true, % links in new window colorlinks=false, % false: boxed links; true: colored links linkcolor=red, % color of internal links citecolor=green, % color of links to bibliography filecolor=magenta, % color of file links urlcolor=cyan % color of external links } % Start the document \ifx\CompleteCourse\relax \ClassScribeSetupB \else \def\makeatletter{\catcode`\@=11\relax} \def\makeatother{\catcode`\@=12\relax} \makeatletter \def\ps@scribeheadings{\let\@mkboth\@gobbletwo \def\@oddhead{\sl\doctitle \hfill \classid }\def\@oddfoot{\hfil \rm \thepage \hfil}\def\@evenhead{\@oddhead}% \def\@evenfoot{\@oddfoot}\def\sectionmark##1{}\def\subsectionmark##1{}} \makeatother \pagestyle{scribeheadings} \begin{document} \bibliographystyle{siam} \fi \begin{center} \Large\bf\doctitle\\[1em] %\Large\bf\classid\\[1em] {\large\bf Lecturer: \classinstructor}\\[.5em] %{\large\bf Scribe Notes by: \scribeone} \end{center} \vspace*{.4in} % HERE IS WHERE YOUR SCRIBE NOTES SHOULD START % DELETE ALL OF ROB'S TEXT AND ENTER YOUR OWN. \nocite{CC} \section{Introduction} In this course we will define the basic two-party model of communication, as introduced in the seminal 1979 paper by Yao. We will discuss different measures of complexity for the basic model, focusing mostly on deterministic, randomized and nondeterministic complexities of protocols. We will introduce the very important set disjointness function and show that it has near maximal communication complexity. We will also introduce extensions of the basic model to more than two players-- the number-in-hand model, as well as the number-on-forehead model. We will show lower bounds for set disjointness in these models as well, as well as give a general discussion of lower bound methods, and the state-of-the-art in terms of lower bounds. We will then switch to discuss the many fundamental applications of communication complexity for a variety of problems: data structures, streaming, privacy, machine learning, game theory, extended formulations, proof complexity, distributed computing, and circuit lower bounds. We will cover some recent applications in depth, with a particular focus on lower bounds for extended formulations of linear programs. \section{The model} Let $X,Y,Z$ be arbitrary finite sets and let $f:X\times Y \to Z$ be an arbitrary function. There are two players, Alice and Bob, who wish to compute the function $f(x,y)$. The main obstacle is that Alice \emph{only} knows $x$ and Bob \emph{only} knows $y$. Thus, to compute the value $f(x,y)$, they will need to communicate with each other. We are assuming that they both follow a fixed protocol agreed upon beforehand. The protocol consists of the players sending bits to each other until the value of $f$ can be determined. We are only interested in the amount of communication between Alice and Bob, and we wish to ignore the question of the internal computations of each player. Thus, we assume that Alice and Bob have \emph{unlimited} computational power. The cost of a protocol $P$ is the \emph{worst} case cost of $P$ over all inputs $(x,y)$. The complexity of $f$ is the minimum cost of a protocol that computes $f$. Formally how do we specify a protocol? In each step one of the players sends one bit of information to the other player. The bit depends on the input of the player who sends it, and all the previous bits communicated so far. In every step, a protocol specifies: \begin{enumerate} \item Which player sends the next bit; \item Value of this bit (as a function of that players' input, and history so far). \end{enumerate} Without loss of generality, we can assume that the players always alternate, and that the last bit sent is the value $P(x,y)$ output by the protocol. This only increases the length of the protocol by a constant factor. We say that a protocol $P$ computes a function $f$ if for all $x,y$, $f(x,y) = P(x,y)$. Usually we set $X=Y=\{0,1\}^n$, and $Z=\{0,1\}$. In this case, the cost of a protocol, $c(n)$, on inputs $x$, $y$, of length $n$ is the worst case cost of $P$ over all inputs of length $n$, and the communication complexity of $f$ is the minimum cost $c(n)$ over all protocols that compute $f$. Another view of a protocol which may be more convenient is the following: \begin{definition} A protocol $P$ over $X\times Y$ with range $Z$ is a binary tree where each internal node $v$ is labelled either by a function $a_v: X \rightarrow \{0,1\}$ or by a function $b_v: Y \rightarrow \{0,1\}$\footnote{If a node is labelled by $a_v$ intuitively means that Alice is sending a bit at this point, similarly for $b_v$ and Bob.}, and each leaf is labelled with an element of $Z$. The value of the protocol $P$ on input $(x,y)$ is the label of the leaf reached by starting from the root, and traversing the tree. The cost of the protocol $P$ on input $(x,y)$ is the length of the path on input $(x,y)$. The cost of the protocol $P$ is the height of the tree. \end{definition} Next, we give some examples of functions that we will study in the up-coming lectures. \begin{example}[Parity] The parity function of $(x,y)$ has value $1$ if $x,y$ have the same parity. A simple protocol is the following: Alice sends the parity of $x$ (1 if the number of 1's in $x$ is odd, and 0 otherwise). Then Bob replies $1$ if and only if the parity of $y$ is equal to the parity of $x$. \end{example} \begin{example}[Set disjointness]\label{ex:disj} $DISJ(x,y)=1$ iff there exists $i$ such that $x_i=y_i=1$. \end{example} \begin{example}[Equality] Equality function: $EQ(x,y)=1$ iff $x=y$. \end{example} \begin{example}[Inner product] The inner product function is defined as $IP(x,y)= \sum_{i=1}^{n} {x_iy_i} \pmod 2$. \end{example} A simple general protocol : Let any function $f(x,y)$, with $|x|=|y| =n$. Alice sends x. Bob sends $f(x,y)$. The total communication is $n+1$ bits. Therefore, the (deterministic) communication complexity of any boolean function is at most $n+1$. However, for many functions, we can develop much more efficient protocols, i.e., protocols with poly-logarithmic communication bits for specific functions. \section{Randomized vs. Deterministic CC} \begin{definition} For a function $f:X\times Y \to Z$, the (deterministic) communication complexity of $f$, denoted by $D(f)$, is the minimum cost of $P$, over all protocols $P$ that compute $f$. $D(f)$ is a function of $n$, the length of $x$ and $y$. \end{definition} In the probabilistic case, players can toss random bits. There are two models depending on whether the coin tosses are public or private. In the public random string model the players share a common random string, while in the private model each player has his/her own private random string. \begin{definition} Let $P$ be a randomized protocol. \begin{description} \item[Zero-sided error:] $P$ computes a function $f$ with zero-sided error if for every $(x,y)$, \begin{eqnarray*} \Pr [ P(x,y)\ =\ f(x,y)] = 1. \end{eqnarray*} Notice that in this case, the number of bits communicated is a random variable. \item[One-sided error:] $P$ computes a function $f$ (with one sided error $\eps$) if for every $(x,y)$ such that $f(x,y)=0$, \begin{eqnarray*} \Pr [ P(x,y)\ =\ 0] &=& 1, \end{eqnarray*} and for every $(x,y)$ such that $f(x,y)=1$, \begin{eqnarray*} \Pr [ P(x,y)\ =\ 1] &\geq& 1-\eps, \end{eqnarray*} \item[Two-sided error:] $P$ computes a function $f$ (with error $\eps$) if \[\forall\ x\in X,\ y\in Y,\quad \Pr \left[ P(x,y) = f(x,y)\right] \geq 1-\eps,\] \end{description} \end{definition} \begin{definition} Let $f:X\times Y \to \{0,1\}$ be a function. We consider the following complexity measures for $f$: \begin{itemize} \item $R_0(f)$ is the minimum average case cost of a randomized protocol that computes $f$ with zero error. \item For $0<\eps <1/2$, $R_\eps(f)$ is the minimum worst case cost of a randomized protocol that computes $f$ with error $\eps$. \item For $0<\eps <1/2$, $R^1_\eps(f)$ is the minimum worst case cost of a randomized protocol that computes $f$ with one-sided error $\eps$. We define $R^1(f) = R^1_{1/2}(f)$. \end{itemize} \end{definition} \begin{lemma} (Markov) $R^1_{\epsilon}(f) \leq 1/\epsilon R_0(f)$. \end{lemma} \begin{lemma} $R(f) = \Omega(\log D(f))$. \end{lemma} Now, let's give an example for the above two protocols for the equality function. \begin{example}[Equality Revisited] Recall that $EQ(x,y)=1$ iff $x=y$. Let's analyse the randomized communication complexity in the public and private coin protocol for the function $EQ$: \begin{description} \item[Public Coin] Let $x\in X$, $y\in Y$, $X=Y=\{0,1\}^n$ be the input strings, and let $r\in\{0,1\}^n$ be the public coin tosses. The protocol is the following: Alice computes the bit $a=(\sum_{i=1}^{n}x_i r_i)\pmod 2$ and sends it to Bob. Then Bob computes $b=(\sum_{i=1}^{n}y_i r_i)\pmod 2$. The value of the protocol is \[P(x,y,r) = 1\quad \text{iff} \quad \sum_{i=1}^{n}x_i r_i=\sum_{i=1}^{n}y_i r_i\pmod 2.\] Note that the communication is only two bits! Now let's analyse this protocol. If $x=y$, then for all $r$, the protocol is correct, i.e., $P(x,y,r)=1$. If $x\neq y$, then with probability $1/2$ (over the public coin tosses) $P(x,y,r)=1$, i.e., our protocol is wrong. If we repeat the above random experiment $c$ times independently, then the probability that our protocol is wrong on all of the executions is $1/2^c$. This protocol has $O(1)$ probabilistic communication complexity when the error $\epsilon$ is a constant. \item[Private Coin] In this setting, encode the inputs $x=x_0,\ldots, x_{n-1}$ and $y=y_0, \ldots y_{n-1}$ as the coefficients of single variate polynomials of degree at most $n-1$: $$A(z) = \sum_{i=0}^{n-1} x_i z^i,$$ $$B(z) = \sum_{i=0}^{n-1} y_i z^i.$$ Consider some field $F$ of size $q \geq 3n$. If $x=y$ then $A(z)=B(z)$ for all $z \in F$, but if $x \neq y$, then $A(z) \neq B(z)$ for at least 2/3 of the $z \in F$ (by Schwartz-Zippel). Thus, our protocol is as follows. \begin{enumerate} \item Alice samples a randomly chosen $z \in F$ and the value $A(z)$, and sends Bob $z$ and $A(z)$. \item Bob sends $1$ if and only if $ B(z) = A(z)$. \end{enumerate} Thus Bob computes the right answer with probabilisty at least 2/3. Note that the communication is only $O(log n)$ bits. \end{description} \end{example} \subsection{Newman's Theorem} The above example gives two different randomized protocols for equality; with public coins the protocol has constant cost, and with private coins the protocol has cost $O(\log n)$. Furthermore it is known that the equality function requires $\Omega(\log n)$ bits in the private coin model, and thus there can be an additive $O(\log n)$ savings in the public coin model. The following theorem due to Newman states that this gap is as large as possible, since any public coin protocol can be transformed into a private coin protocol with a small penality in the error and a small additive penalty in the communication complexity. \begin{theorem} Let $f: \{0,1\}^n \times \{0,1\}^n \rightarrow \{0,1\}$ be a function. For every $\delta>0$ and every $\epsilon>0$, $$R_{\epsilon + \delta}(f) \leq R^{pub}_{\epsilon}(f) + O(\log n + \log \delta^{-1})$$ \end{theorem} \begin{proof} We will prove that any public coin protocol $P$ with error $\epsilon$ can be transformed into another public coin protocol, $P'$ such that: (i) the communication complexity of $P'$ is the same as that of $P$; (ii) $P'$ uses only $O(\log n + \log \delta^{-1})$ random bits; and (iii) the error of $P'$ is at most $\epsilon + \delta$. The theorem follows since $P'$ can be easily converted to a private coin protocol with the desired parameters: first Alice will privately flip that many random coins, and then send them to Bob, and then they proceed to follow the protocol $P'$. Let $Z(x,y,r)$ be a random variable that is equal to 1 if $P(x,y,r)$ outputs an incorrect answer, and 0 otherwise. Because $P$ has error $\epsilon$, $E_r [Z(x,y,r)] \leq \epsilon$ for every $x,y$. Let $r_1,\ldots, r_t$ be random strings, where we will soon set $t=O(n/\delta^2)$, and define $P_{r_1,\ldots,r_t}(x,y)$ as follows: Alice and Bob choose $i \leq t$ at random, and then run $P(x,y,r_i)$. We will prove that there exist strings $r_1,\ldots,r_t$ such that $E_i[Z(x,y,r_i)] \leq \epsilon + \delta$ for all $(x,y)$. For this choice of strings, the protocol $P_{r_1,\ldots,r_t}$ will be our desired protocol, $P'$. We use the probabilistic method to show the existence of $r_1,\ldots,r_t$. Choose $r_1,\ldots,r_t$ at random, and consider a particular input pair $(x,y)$. The probability that $E_i[Z(x,y,r_i)] > \epsilon + \delta$ is exactly the probability that $1/t \sum_{i=1}^t Z(x,y,r_i) > \epsilon + \delta$. By Chernoff inequality, since $E_r[Z(x,y,r)] \leq \epsilon$, $$Pr_{r_1,\ldots,r_t} [ 1/t \sum_{i=1}^t Z(x,y,r_i) - \epsilon) > \delta] \leq 2e^{-2\delta^2t}.$$ By choosing $t=O(n/\delta^2)$, this is smaller than $2^{-2n}$. Thus for a random choice of $r_1,\ldots,r_t$, the probabilty that there exists a bad $(x,y)$ such that $E_i[Z(x,y,r_i)] > \epsilon + \delta$ is smaller than $2^{2n} 2^{-2n} =1$ (by the union bound). Thus there exists $r_1,\ldots,r_t$ such that for every $(x,y)$ the error of $P_{r_1,\ldots,r_t}$ on $(x,y)$ is at most $\epsilon + \delta$. It is easy to check that the number of random bits used by the protocol $P_{r_1,\ldots,r_t}$ is $\log t = O(\log n + \log \delta^{-1})$, and that the communication complexity of $P_{r_1,\ldots,r_t}$ is the same as that of $P$. \end{proof} \section{Nondeterministic/co-Nondeterministic CC} In the non-deterministic model of communication complexity, players share \emph{nondeterministic} bits $z$. Now a protocol is a function of $x,y,z$, and we say that a protocol $P$ computes a function $f$ if for all $x,y$: \begin{eqnarray*} f(x,y)=1 &\implies& \exists z\ P(x,y,z)=1\\ f(x,y)=0 &\implies& \forall z\ P(x,y,z)=0. \end{eqnarray*} The communication complexity in this model is defined as the maximum length of $z$ plus the number of bits exchanged over all $x,y$. Similarly, exchanging the position of the existential and for all quantifier we can define co-nondeterministic CC. The basic example is set disjointness (see Example~\ref{ex:disj}). If the players share $\log n$ nondeterministic bits, they guess $i$ and check if $x_i=y_i = 1$. Define $N^1(f)$ to be the nondeterministic communication complexity of $f$, and similarly, let $N^0(f)$ denote the co-nondeterministic communication complexity of $f$. \section{Communication Complexity Classes} We can define the following communication complexity classes: \[P^{CC},\ RP^{CC}, BPP^{CC}, NP^{CC}, PP^{CC} \] as the set of functions $f(x,y)$ that can be solved with poly-logarithmic communication complexity. $RP^{CC}$ are those communication comlexity functions that can be solved in time $polylog(n)$ by a randomized, 1-sided error protocol. That is, on "yes" instances, the protocol is correct with probability 1, and on "no" instances the protocol errs with probability at most 1/4. $BPP^{CC}$ are those functions that can be solved in time $polylog(n)$ by a randomized, 1-sided error protocol. On "yes" instances, the protocol is correct with probability $3/4$, and on "no" instances the protocol is also correct with probability $3/4$. Note that the error is 2-sided and bounded away from $1/2$. $PP^{CC}$ are those functions that can be solved in time $polylog(n)$ by a randomized protocol with unbounded error. That is, on all instances, the protocol is correct with probability greater than $1/2$. Finally, $ZPP^{CC}$ are those functions that can be solved in {\it average case time} $polylog(n)$ by a randomized protocol with zero error. Remark: Unlike the computational complexity classes where NO non-trivial relationship is known, here we know almost everything, i.e., \[P^{CC}\subsetneq RP^{CC}\subsetneq BPP^{CC}\subsetneq NP^{CC}.\] \section{Applications} Communication complexity arguments have found numerous application to a large number of different areas. Applications include the following. \begin{enumerate} \item Bisection width of networks \item VLSI \item Decision tree lower bounds \item Data structures -- cell probe model and dynamic data structures \item Boolean circuit complexity. This includes Depth 2 threshold circuits, and the circuit class $ACC$. \item Turing machine time-space trade-offs \item Streaming algorithms \item Game theory (truthfulness vs. accuracy) \item Differential privacy \item Proof complexity \end{enumerate} %\section{Course Outline} %The main topics that we will discuss in this course can be summarized as follows: % %\begin{itemize} % \item $2$ player protocol, upper bounds, lower bounds methods (Fooling set method, rank method, discrepancy method); % \item Public/private coin protocols; % \item The number-on-forehead model (NOF). Upper bounds and Lower bounds. % \item Applications. %\end{itemize} \bibliography{refs} %\bibliographystyle{plain} \end{document}