\documentclass[twoside,11pt]{article}

\title{A programming language for scientific computation}
\author{
\begin{tabular}{cccc}
Yong Man Ra & Domiano Lee & Tiantian Zhou & Hanhua Feng\thanks{
Last modification: \$Id: wp.tex,v 1.9 2003/02/15 17:25:38 hanhua Exp $.$\$
}
\\
\multicolumn{2}{c}{\{yr86,cl588\}@columbia.edu} & 
\multicolumn{2}{c}{\{tiantian,hanhua\}@cs.columbia.edu}
\end{tabular}
}

\usepackage{graphicx}

\setlength{\textwidth}{6.7in}
\setlength{\textheight}{8.5in}
\setlength{\oddsidemargin}{-0.2in}
\setlength{\evensidemargin}{-0.4in}
\setlength{\topmargin}{0in}

\begin{document}
\maketitle

In this project, we are planning to design and implement a language for
scientific computation, which can perform matrix and vector operations by
arithmetic operators such as Matlab\cite{matlab} and IDL\cite{idl} did.

\section{Background}

For the past fifty years, the demands for high-performance scientific
computations have been pushing various computer technologies.  Among
them, programming languages that are more efficient, more convenient,
and more natural to scientific computations have also been developed.
Unfortunately, their progress is more or less lagged behind other
computer technologies.

Fortran is one of the first-generation programming languages, which has
created many irreplaceable scientific packages.  Although there were
many other generations of Fortran standards, the backward compatibility
constantly hindered its development.  Fortran can hardly keep up
with the development of modern programming language and software
engineering techniques, modern computer hardware architecture, or even
the needs for mathematical efficiency and conveniences.  Meanwhile,
the progress of computer science in recent years is essentially
dominated by the industry of information technologies, where the
market of scientific computations is too small to be seen.
Universities no longer implement Fortran so that the population of Fortran
programmer becomes less and less.

Scientific investigators have to attempt other possible alternatives.  When they choose a
programming language for their projects, they would take these issues
into account: (1) suitable and efficient for scientific computation,
(2) easy to learn (better be the language they have learned in
class), and (3) popularity, so that their programs would not easily
get out-dated, particularly for lack of maintainers.  Unfortunately,
these requisites can be hardly satisfied simultaneously.  Some people
choose to use those popular languages that are not designated for
scientific computations, and have to do lots of unnecessary and
tedious work in order to implement relatively simple mathematical
formulas, while other people use non-standardized languages, which are included
in some popular commercial software packages.  The languages in Matlab
and IDL are historically satisfactory, but the down-side is that the
programs written by these languages would be affected by the
economical status of their vendors, which is in turn doomed probably
by the market size of these software packages.

Under this circumstance we present our programming language, as a
small trial in design and implementation of programing languages for
scientific computations.

\section{Related works}

Matlab by Mathworks is probably the most popular commercial language
(or rather to say, software) of this kind nowadays. Another choice is
IDL, which is developed by Research Systems Inc., a Kodak
company. There are some other related softwares such as
Maple\cite{maple}, Mathcad\cite{mathcad}, and
Mathematica\cite{mathematica}.  Some of them can do (or focus on)
symbolic computations.

Invented by Kenneth E. Iverson, who received the Turing Award of 1980
for this reason, APL is a language trying to imitate standard
mathematical notations, using hundreds of different ``primitives''
which are hard to be typed by an ordinary keyboard.

Many open-sourced numerical tools are also available.  GNU
Octave\cite{octave} is a general numerical interactive language,
originally developed for solving chemical reactor design problems.
Scilab\cite{scilab} is another free numerical tool similar to Matlab.

There are a lot of libraries written in C++, trying to make use of
object-oriented and operator overloading techniques to implement
direct matrix/vector operations by arithmetic operators \cite{oon}.
Some of them go even further: by using Blitz++\cite{blitz} the C++
compiler reportedly generates much better binary code than a Fortran
compiler can.  Another project is POOMA\cite{pooma}, which is
focused on parallel computing.  Java does not support operator
overloading, so it is not possible to do matrix/vector operations by
arithmetic operators.  However, investigators are trying to add
operator overloading into Java as well as some other features, in
order to make the language more suitable to high-performance
scientific computing.  Titanium\cite{titanium} is such a project led
by computer scientists in UC Berkeley.

Besides those basic matrix/vector operations, there are many advanced
frequently-used matrix algorithm in linear algebra and matrix theory.
Unfortunately there are little choices of implementations: even Matlab
and IDL cannot build their own.  Matlab currently uses
LAPACK\cite{lapack} and IDL uses algorithms in Numerical Recipe
(NR)\cite{nr}.  Both LAPACK (and its relatives) and NR were originally
written in Fortran.  LAPACK can be translated to C by f2c\cite{f2c},
and Numerical Recipe in C was probably translated from Fortran by hand.
Algorithms in NR is said to be weak in efficiency and precision
\cite{nrprob} in part because those algorithms were originally implemented
with single-precision float numbers.

\section{Goal}

The main goal of our language is to implement and achieve a simple and ease-of-use
programming language for numerical computations.  The programmer of
our language can do matrix operations directly by arithmetic operators
such as '$+$', '$-$' and '$*$', as well as other frequently-used
symbols.

\subsection{Portability}

Our language will run on various platforms.  For portability we will
use ANTLR\cite{antlr} as the syntax recognizer, which is running on a
Java virtual machine(VM).  The compiler of our language will generate
Java codes that can be compiled by Java compiler in JDK and run on a
Java VM.  The basic matrix operations will be performed by classes in
javax.vecmath package.

\subsection{Efficiency}

Efficiency is always one of the most important evaluations of a programming
language for scientific computing.  There are two aspects of
efficiency: the efficiency of running time, and the efficiency of
coding.  Using external numerical packages, the running time
efficiency is dominated by those external packages.  However, our
language will be designed to reduce the programmer's work.

\subsection{Ease-of-use}

The language will be simple and quick-to-start.  Only basic and
necessary language features would be included in the language
specification.  Meanwhile, most of the needs in numerical computations
would be fulfilled.

\section{Main language features}

In this section we describe some of main features in our language.

\subsection{Elemental data types}

We support three data types: integer, float and string.

\subsection{Basic matrix operations}

Basic matrix and vector operations should be supported, e.g.,
addition, subtraction, multiplication, transpose and inversion.

\subsection{Matrix slicing}

The slicing operation is to get vector from a row or a column of a
matrix.  If the sliced vector is l-value, the corresponding row or
column of the original matrix should be modified.  

\subsection{Program flow control}

The main statements of program flow control should be implemented,
including if-then-else, while, break, continue, and for.  In addition,
programs in our language can also define and invoke subroutines
(probably subroutines in other files).

\subsection{Internal functions}

The minimum set of internal functions includes print, input, load and
save.

\subsection{Interface to Java numerical packages}

If we can find good Java numerical packages, we will add a calling
interface to our language.

\subsection{Sample syntax}

This sample code gives an idea of how we would like to conduct loop, 
matrix declaration, multiplication, and the use of function.
Also, syntax structure may subject to minor changes when implementing.

\begin{verbatim}
function LS(Matrix A, vector b){
        return invert(A) * b;
}

for (i = 3 : 5) {
        A = [1, 2; 3, 4];
        b = [1, i];
        x = A * b;
        y = LS(A, b);
        print(x);
        print(y);
}
\end{verbatim}

% \section{Architecture}

% The following figure illustrates the structure of supporting
% environment of our language.

\section{Related issues}

In this section we will talk about some issues that we would meet in
the future.  Other issues are worth discussing, although they
might not be included in our goal.

\subsection{Interpretive versus translative}

There are two approaches toward the goal of our project.  We can
either scan and then immediately execute our language (interpretive),
or generate an output file, either in source code of some other
language or in executable binary code.  Matlab and IDL are both
interpretive.  We plan to generate a Java source code output rather
than to interpret our language.

\subsection{Zero-based index versus one-based index}

Human beings have been counting and numbering objects starting from
one for thousands of years.  The first element of a vector
$\textbf{x}$ is marked as $x_1$.  As a result, Fortran starts its
index from one.  However, computer scientists love arrays starting
at zero, which is more natural in computer systems.  As a result, C
and its descendants start their indexes from zero.  In signal or image
processing, it is also a better choice to number the first point of
a time series or the first pixel of a image with zero.  Originally
designed for matrix operations, Matlab numbers the first element with
one.  Originally designed for image processing, IDL numbers the first
element with zero.

We think the use of one-based index system is basically just to agree
with concepts in human brain.  Decrementing by one in all indexes in
linear algebra would introduce no harm.  Our language would use a
zero-based index system.

\subsection{Arrays with number of dimensions greater than 2}

In mathematics, people usually deal with vectors and matrices, and
high-dimensional arrays are rarely encountered.  However,
high-dimensional arrays are useful in many other scientific field.
For example, in functional magnetic resonance imaging(fMRI), a
4-dimensional array is needed to store the dynamic spatial and
temporal signals of a human brain.

Originally Fortran supports only one-dimensional array, and Matlab
supports only vectors and matrices before the version 5.0 came out.
Currently, both Matlab and IDL support high-dimensional
arrays.

Due to lack of supporting classes and libraries in the current JDK, we
probably will not implement high-dimensional arrays.

\subsection{Numerics and object-oriented programming}

Intuitively, numerical algorithms have little to do with object-oriented
techniques.  In most of numerical routines, even structures are rarely
used, much less to say classes.  Meanwhile, it is very natural to
implement vectors and matrices by classes in C++, not to mention the
pure class-based language Java.  However, if vectors and matrices have
been internally implemented as language features, object-oriented
features would immediately become redundant for most numerical needs.

We probably would not implement any object-oriented features.

\subsection{Numerics and advanced object-oriented features}

In mid-80's, template has been introduced into C++, along with
so-called generic programming (GP) techniques allegedly "stolen" from
the Smalltalk language.  Java designers have refused to introduce
template into Java for years, although there are rumors saying that generics are
one of the new features proposed for Java 1.5 (Tiger).

Most functionalities implemented by template can also be implemented
without template.  In Java, one can use classes and interfaces to
implement these functionalities.

A template acts somehow like a macro in C++, therefore it has better
performance than those implementations in classes and interfaces, with
a price of slow compilation and huge executive files.

In ISO C++/98, there are many strong template features, such as
expression template and template functions within template classes.
The immediate consequence of these rarely used functionalities is that
many C++ compilers failed to comply this standard.  Programmers have
to spend more time to deal with syntax problems if they want to make
use of these advanced features.  Anyway, there are certain reasons for
these features to exist: these features have been used in Blitz++ in
order to gain much better performance by source-code level
CPU-cache-line optimization, of which both the C++ compiler and the
users of matrix classes are unaware \cite{veldhuizen}.

In class, Prof. Aho mentioned three important concepts in
object-oriented techniques: abstraction, morphism, and late-binding.
In our opinion, the generic programming feature in C++ is unsuccessful
because it introduces some problem with ``late-type-binding''.  This is not
to say the program should decide which type should be used in the run
time, but to say that it is hard for programmers to build up a complete
template structure before trying to use it.  Many syntax errors
cannot be found before instantiation of the template class with some
specified types.  A template class is probably running well for type
$A$, but would have syntax errors for type $B$.  A better language
design is that, before the definition of a template class, by
introducing classes of types, the type arguments can be first
described and constrained, then problems and errors can be found as
early as possible, rather than to be discovered at the time when the
program does ``late-type-binding''.

It is too time-consuming to implement expression-based CPU-cache-line
optimization in our project, and it is not suitable for generating
object code in Java because we have to use Java classes to do basic
matrix operations.

\subsection{Java as an educational language}

One reason to use the Java language is its portability.  The same code can
run on different platforms without any changes.  However, Java is not
really as much strong in its portability as it has claimed.  In
order to run Java applications, users usually need to install Java
runtime environment (JRE) for a Java VM, which is unfortunately not a
part of most operating systems.  Moreover, Java has trouble to
implement its basic classes and libraries by itself.  The situation of
Java is a little similar to his younger bother and competitor C\#:
both of them would be largely dominated and affected by their
``Owners'', Sun Microsystems and Microsoft.  There are several other
implementations by various companies, but they may drop these projects
at any time.  The open-sourced project GCJ\cite{gcj} still has a long
way to go.

The future of Java is unknown but yet not bad: it is an ideal
object-oriented language and now replacing the old Pascal language and
becoming the standard educational language in computer science.  As an
educational project, Java is probably our best choice.

\subsection{Parallel(Network) computing}

Many algorithms cannot be used because of lacking computational
capability.  By using multiple processors and/or multiple computers,
parallel algorithms can obtain solutions in shorter time.  Meanwhile
so much computational capability on the network is wasted because the
parallel algorithms are hard to be implemented in reality.  There are
almost no true parallel or networking features in current popular and
general languages: Java does nothing but encapsulating network APIs in
classes and some remote-loading features in Java Applets.  Although Java makes this
task a little easier, programmers still need to deal with race conditions themselves even if they just want to use multiple processors in a single computer,  On the other hand, many programming languages
designed for scientific computations would take advantage of multiple
processors.  By observation we found Matlab can make use of multiple
processors in its supporting functions, although it is not applicable
for user routines.

We are thinking of two approaches.  The first one is to automatically
parallelize by checking the variable dependencies.  The second is to
introduce a network object model. This issue is probably far beyond
what we can attain in this project.

\begin{thebibliography}{00}
\bibitem{matlab}
Matlab, http://http://www.mathworks.com/

\bibitem{idl}
IDL, http://www.rsinc.com/idl/index.asp

\bibitem{maple}
Maple, http://www.maplesoft.com/

\bibitem{mathcad}
Mathcad, http://www.mathsoft.com/products/mathcad/

\bibitem{mathematica}
Mathematica, http://www.wolfram.com/products/mathematica/

\bibitem{octave}
GNU Octave, http://www.octave.org/

\bibitem{scilab}
Scilab, http://www.scilab.org/

\bibitem{oon}
Object-Oriented Numerics Page, http://www.oonumerics.org/oon/

\bibitem{blitz}
Blitz++, http://www.oonumerics.org/blitz/

\bibitem{pooma}
POOMA, http://www.codesourcery.com/pooma/pooma

\bibitem{titanium}
Titanium, http://www.cs.berkeley.edu/Research/Projects/titanium/

\bibitem{lapack}
http://www.netlib.org/lapack/

\bibitem{nr} 
William Press, Saul Teukolsky, William Vetterling and Brain Flannery,
Numerical Recipes in Fortran 77, 2nd Edition, Cambridge University
Press, 1992, http://www.nr.com

\bibitem{f2c}
f2c, http://www.netlib.org/f2c/

\bibitem{nrprob}
Various\ authors, 
http://www.starlink.rl.ac.uk/star/docs/sc13.htx/N-a2b3c1.html , 
http://www.lysator.liu.se/c/num-recipes-in-c.html , and
http://www.nr.com/bug-rebutt.html

\bibitem{antlr}
Antlr, http://www.antlr.org/

\bibitem{veldhuizen} 
Todd Veldhuizen, Techniques for Scientific C++, Indiana University
Computer Science Technical Report No. 542, 2000.,
http://osl.iu.edu/~tveldhui/papers/techniques/

\bibitem{gcj}
GCJ, http://gcc.gnu.org/java/

\end{thebibliography}

\end{document}
