%Communication Complexity Scribe Notes template \ifx\CompleteCourse\relax \ClassScribeSetupA \else \documentclass[11pt]{article} \usepackage{synttree} \usepackage{amssymb} \newtheorem{theorem}{Theorem} \newtheorem{lemma}[theorem]{Lemma} \newtheorem{corollary}[theorem]{Corollary} \newtheorem{proposition}[theorem]{Proposition} \newtheorem{homework}{Homework} \newenvironment{definition}{\begin{trivlist}\item[]{\bf Definition}\ }% {\end{trivlist}} \newenvironment{fact}{\begin{trivlist}\item[]{\bf Fact}\ }% {\end{trivlist}} \newenvironment{example}{\begin{trivlist}\item[]{\bf Example}\ }% {\end{trivlist}} \newenvironment{proof}{\begin{trivlist}\item[]{\bf Proof}\ }% {\end{trivlist}} % Make the page large \addtolength{\textwidth}{1.50in} \addtolength{\textheight}{1.00in} \addtolength{\evensidemargin}{-0.75in} \addtolength{\oddsidemargin}{-0.75in} \addtolength{\topmargin}{-.50in} % \vdashsub{X} makes a turnstyle with subscript "X" % \vdashsup{X} makes a turnstyle with superscript "X" \newdimen\srbdimenA \newcommand{\vdashsupsub}[2]{ \mathop{ \setbox251 = \hbox{$\scriptstyle #1$} \setbox252 = \hbox{$\scriptstyle #2$} \ifdim \wd251<\wd252 \srbdimenA = \wd252 \else \srbdimenA = \wd251 \fi \setbox255 = \hbox {${\srbAvdash \vphantom( \kern -\srbdimenA \kern +.05em} ^{\hbox to\srbdimenA{\hfill \box251\hfill}} _{\hbox to\srbdimenA{\hfill \box252\hfill}}$} \box255 \kern .05em}} \newcommand{\srbAvdash}{\hbox{ \vrule height1.4ex width0.02em \dimen255 = \srbdimenA \advance\dimen255 by 0.1em \vbox{\hrule width\srbdimenA height0.02em \kern .65ex }}} \newcommand{\vdashsup}[1]{\vdashsupsub{{#1}}{\mbox{~}}} \newcommand{\vdashsub}[1]{\vdashsupsub{\mbox{~}}{#1}} \fi % FOR THE SCRIBE: CUSTOMIZE THE ENTRIES BELOW: % Fill in the following information particular to these scribe notes: \def\scribeone{Lei Huang} % Who is the scribe? \def\classdate{Fall, 2014} % Date of the class \def\classnumber{5} % Is this the first, second, ...? % Here are some commands that stay the same for the whole class. \def\classinstructor{Toniann Pitassi} \def\classtitle{Communication Complexity} \def\doctitle{\textup{CS 2429 - Foundations of Communication Complexity}} \def\classid{\textup{Lecture \#\classnumber: \classdate}} % Put your macros for these scribe notes HERE % It is best to use as few as possible. % environments for "theorem", "corollary", "lemma", "fact" "definition" % "homework", "proof", "example", "proposition" % are already defined above. % Start the document \ifx\CompleteCourse\relax \ClassScribeSetupB \else \def\makeatletter{\catcode`\@=11\relax} \def\makeatother{\catcode`\@=12\relax} \makeatletter \def\ps@scribeheadings{\let\@mkboth\@gobbletwo \def\@oddhead{\sl\doctitle \hfill \classid }\def\@oddfoot{\hfil \rm \thepage \hfil}\def\@evenhead{\@oddhead}% \def\@evenfoot{\@oddfoot}\def\sectionmark##1{}\def\subsectionmark##1{}} \makeatother \pagestyle{scribeheadings} \begin{document} \bibliographystyle{siam} \fi \begin{center} \Large\bf\doctitle\\[1em] %\Large\bf\classid\\[1em] {\large\bf Lecturer: \classinstructor}\\[.5em] %{\large\bf Scribe Notes by: \scribeone} \end{center} % HERE IS WHERE YOUR SCRIBE NOTES SHOULD START % DELETE ALL OF ROB'S TEXT AND ENTER YOUR OWN. \section{Applications of Communication Complexity: Extended Formulations of Linear Programs} Linear programming is a very powerful tool for attacking hard combinatorial optimization problems. Methods such as the ellipsoid algorithm have shown that linear programming is solvable in polynomial time. Linear programming also plays a central role in the design of approximation algorithms. In fact, it is known that linear programming is P-complete, and this implies that if $NP=P$ then for every problem in $NP$, given an instance, it is possible (in polytime) to solve it via a polynomial-sized LP. A large class of linear programs were identified by Yannakakis, and referred to as {\it extended} {\it formulations}. We emphasize that extended formulations of LPs to not capture all LPs for solving a given NP-hard problems, but nonetheless, they capture a large and useful family of LPs. We will define extended formulations, and then prove that lower bounds on extended formulations follow from communication complexity lower bounds. \section{Definitions and Background} For a matrix $A$, let $A_j$ denote the $j^{th}$ column and $A^j$ the $j^{th}$ row. For any combinatorial optimization problem, we can encode its set of possible solutions as a set of points $X \subseteq \{0,1\}^n$ such that optimizing an instance of the problem becomes the problem of optimizing a linear objective function, $f(X)$ over the convex hull of these points. This convex hull, $P = conv(X)$ defines a polytope in $n$ dimensional space, whose {\it vertices} are the feasible solutions to the problem and whose {\it facets} (or sides) correspond to the constraints of the problem. A {\it face} of the polytope $P$ is the intersection of $P$ with any closed half space $H$ such that none of the points of $P$ lie on the boundary of $H$. A {\it vertex} of a polytope is a 0-dimensional face, and a {\it facet} is an $(n-1)$ dimensional face of polytope. A {\it feasible} {\it solution} for an optimization problem with constraints $Ax \leq b$ is any point $x \in R^n$ which satisfies all of the constraints of the problem. The {\it size} of a linear program is the number of constraints it contains. \subsection{The permutahedron polytope} Examples. Consider the permutahedron consisting of the convex hull of all permutations of $[1,..,n]$. It is not too hard to see that it is defined by exponentially many constraints. (For all $S \subseteq [n]$ we have $\sum_{i \in S} x_i \geq 1 + 2 + \ldots + |S| = |S|(|S|+1)/2$. However, it is possible to introduce new variables, and to write a new set of constraints over the original variables plus the new variables that is polynomial size, and such that the projection of this higher dimensional polytope down to the original variables is the permutahedron! That is, it is possible to rewrite the constraints (using more variables) in order to obtain a polynomial size {\it extended} {\it formulation} for the permutahedron LP. To do this, Goemans showed that we can take any switching network that sorts $x_1,\ldots,x_n$. A switching network has $n$ inputs, $x_1, \ldots, x_n$. It is visualized as a set of n horizontal wires. It also has a set of ordered comparator gates that connect pairs of wires. A comparator gate from wire $i$ to wire $j$ is drawn as a vertical arrow originating at wire $i$ and pointing to wire $j$. There are two outputs to a comparator gate, the OR output (the start of the arrow) computes the OR of the two inputs, and the AND output (the end of the arrow) computes the AND of the two inputs. The effect of this gate is to sort $i$ and $j$. Goeman's showed that if we have a comparator network for $x_1,\ldots,x_n$ with $m$ gates, then we can introduce $2m$ new variables, $x_{n+1}, \ldots, x_{n+2m}$, corresponding to all possible subfunctions computed by the network, and a set of inequalities over the variables $x_1 ,\ldots, x_{n+2m}$ such that the projection of this polytope back to the original variables is the permutahedron. The inequalities express the fact that the output gates are $1,2,..,n$, and that each intermediate gate is computing correctly. If the circuit has $k$ comparators, we have \begin{itemize} \item[(1)] for all $i \in [n]$, $x_{2k+i} = i$ \item[(2)] for any comparator gate with inputs $x_i$, $x_j$ and outputs $x_k = min(x_i,x_j)$, and $x_l = max(x_i,x_j)$ we have $x_i + x_j = x_k + x_l$, $x_k \leq x_i$, $x_k \leq x_j$. \end{itemize} It is not hard to see (although we will not prove it here) that $Proj_n(Q) = P$ and the size of $Q$ is polynomial. Intuitively, if it is a legal comparator circuit, then when we feed it some permutation of $[1,..,n]$, it will output the values in sorted order, and conversely, if the circuit outputs the values in sorted order and all of the intermediate variables are defined correctly, then the input must be a permutation of $[1,..,n]$. (DRAW PICTURE) \subsection{Extended Formulations} \begin{definition} An extended formulation (EF) of a polytope $P \subset R^n$ defined by $Ax \leq b$ is a linear system: $$Ex + Fy =g, y \geq 0$$ in the variables $(x,y) \in R^{n+r}$ where $E,F$ are real matrices with $n,r$ columns respectively and such that $x \in P$ (satisfies $Ax \leq b$) if and only if there exists $y$ such that $Ex + Fy =g, y \geq 0$ holds. \end{definition} The {\it size} of an extended formulation is the number of $y$ variables, or equivalently the number of inequalities. The extension complexity of a polytope $P$, $xc(P)$, is the minimum number of inequalities in any extended formulation for $P$. We will now define the {\it slack} {\it matrix} corresponding to a polytope $P$. By using extended formulations, we can sometimes reduce the number of facets exponentially. When this can be done, we can run a standard LP algorithm to get a polynomial time algorithm. This is one of the most common approaches attempting to show that $P$ equals $NP$. For example, the TSP polytope is the set of vectors corresponding to tours of $K_n$. A polysize extended formulation for the TSP polytope implies $P=NP$! We note that EFs do not characterize *all* linear programs for a given problem. The restriction is that the polytope must be independent of the instance, so the instance only shows up in the objective function. Thus lower bounds for EFs rules out a large family of polytime LP algorithms but does not rule out all of them. (Since linear programming is P-complete, ruling out all LPs would essentially be showing that $P \neq NP$.) \subsection{Brief History} \begin{itemize} \item[(1)] Yannakakis in 1990 showed that any symmetric EF for TSP or matching has exponential size. He also established the connection between EFs and communication complexity. He also established the equivalence between EF and nonnegative rank. \item[(2)] Fiorini-et-al, 2012 showed that any EF for clique and TSP has exponential size. This is a reduction to set disjointness. \item[(3)] Braun-et-al, 2012 extended the EF setup for approximation algorithms, and showed that any EF for approximating clique within $n^{1/2 - \epsilon}$ has exponential size. \item[(4)] Braverman, Moitra 2013, and Braun-Pakutta, 2013 showed that any EF approximating clique within $n^{1-\epsilon}$ has exponential size. These papers use information complexity techniques; the latter paper uses the notion of common information. \item[(5)] Chan-et-al, 2013 showed that any EF that approximates Maxcut within $(2-\epsilon)$ has quasipolynomial size. This paper is a reduction to Sherali-Adams lower bounds via Fourier analysis. This is the only paper that I am aware of that looks at instances that are "natural". They also extend the EF setup to the SDP setting and prove lower bounds for symmetric EFs of SDPs. \item[(6)] Rothvoss, 2013 showed that any EF for perfect matching has exponential size. This is particularly interesting since we do have LPs for solving perfect matching. This is a corruption lower bound. \end{itemize} All of the above results with the exception of (5) use communication complexity. \section{Lower Bounds on Extension Complexity} \subsection{Yannakakis' Factorization Theorem} Yannakakis shows that the notion of EF which is a geometric parameter is equivalent to nonnegative rank of the associated slack matrix, which is algebraic parameter. \begin{definition} Let $P$ be a polytope defined by facets $Ax \leq b$, The slack matrix, $S_{Ax \leq b}$ for $P$ is a matrix with $v$ rows and $f$ facets, where $v$ is the number of vertices of $P$, and $f$ is the number of facets of $P$. The entry $[i,j]$ is equal to $b_i - A_i x_j$, or the distance from the ith facet to the jth vertex. \end{definition} \begin{definition} The nonnegative rank, $rank_+(M)$ of a matrix $M$ is the smallest $r \in N$ such that $M$ can be expressed as $M=FV$ where $F,V$ are nonnegative matrices with intermediate dimension $r$. \end{definition} In Yannakakis' seminal paper, he gave a factorization theorem which is the backbone for most of our extended formulation lower bounds. It shows that the nonnegative rank of the slack matrix for a linear program is a lower bound on the extension complexity. \begin{theorem} Let $P$ be a polytope defined by $Ax \leq b$. If the slack matrix for $P$ has nonnegative rank $r$, then $xc(P) = \theta(n+r)$. \end{theorem} We note that this is actually an equivalence. If $M$ has nonnegative rank $r$, then we can write $M$ as $M_1 + \ldots + M_r$ where each $M_i$ is a rank one nonnegative matrix. Nonnegative rank $r$ implies that there exist $r$ nonnegative functions $f_1,\ldots, f_r$, $f_i$ mapping cliques to $R$ such that for each $G$, $val(G)$ can be written as a nonnegative linear combination of $f_1,\ldots,f_r$. \subsection{The Clique Polytope} The clique polytope (for graphs with $n$ vertices) is the convex hull of the following set of $2^n$ vectors: all vectors $v \in \{0,1\}^{n^2}$ that describe a clique on some subset $S \subseteq [n]$ (and have no other edges). Given a graph $G=(V,E)$ over $V \subseteq [n]$, $w(G)$ is defined as follows. \begin{itemize} \item[(1)] $e_{i,i}=1$ if $i \in V$ and 0 otherwise \item[(2)] $e_{i,j} =0$ if $i,j \in V$ and $(i,j) \in E$; \item[(3)] $e_{i,j}=-1$ if $i,j \in V$ and $i,j) \not \in E$; \item[(4)] otherwise $e_{i,j}=0$. \end{itemize} It is not too hard to see that for each graph $G$, $ \leq cliquenum(G')$ and the vector $x$ achieving equality corresponds to a maximal clique in $G$. For a graph $G$, the linear program (with exponentially many constraints) for clique has variables $x_{i,j}$, $i,j \in [n]$. We want to maximize $$ subject to the following constraints: For each graph $G'$, $ \leq cliquenum(G')$. Note that the $x$ achieving the max value will be a maximal clique in $G$. Let $Q^{all}$ be the polytope defined by the above inequalities. Clearly $P \subseteq Q^{all}$. (Actually $Q^{all} = P$. Why?) We want to show that there is no polysize EF for $Q^{all}$. Given $Q^{all}$ defined by $Ax \leq b$ as described above, an EF is $Ex + Fy =b'$, $y \geq 0$ such that the projection to $x$ is $Q^{all}$. By the factorization theorem, it suffices to prove lower bounds on the nonnegative rank of $SM(P,Q^{all})$. \section{Nonnegative Rank and Communication Complexity} Let $M$ be our slack matrix, and suppose that it has rank $r$. Then it can be written as the sum of $r$ rank one, nonnegative matrices, $M_1 + \ldots + M_r$. The support of each $M_i$ is a combinatorial rectangle. Thus the nonnegative rank of $M$ is at least the number of rectangles needed to cover the support of $S$. Thus, it suffices to show that $M$ has high nondeterministic communication complexity. Now consider a simple subset of all graphs, $SIMPLE = \{G_a ~|~ a \subset [n]\}$ where $G_a = (V_a, E_a)$, $V_a =a$ and $E_a = \emptyset$. That is, the vertices of $G_a$ are the vertices in $a$, and $G_a$ has no edges. Let $Q'$ be the set of inequalities $ \leq cliquenum(G_a)$ for all $G_a \in SIMPLE$. Note that the clique number of $G_a$ is 1 for all $G_a \in SIMPLE$. We will show that UDISJ is embedded in $SM(P,Q')$ and therefore embedded in $SM(P,Q^{all})$. Let $Clique_b$ be a graph consisting of a clique on all vertices in $b$ and no other edges. Consider $G_a \in SIMPLE$ and $Clique(b)$. That is, $G_a$ has no edges and the vertex set is $a$, and $Clique(b)$ is a clique over the vertices in $b$. If $a \cap b$ is empty, then $ =0$ and if $a \cap b$ has size $1$, then $=1$. Thus, we see that UDISJ is embedded in $SM(P,Q')$ and therefore the nonnegative For any pair $(a,b)$ with the promise that either $a,b$ are disjoint or they intersect in exactly one element, \end{document}