charmxi: Pass in original filename when preprocessing .ci files

[charm.git] / doc / faq / install.tex

\section{Installation and Usage}

\subsubsection{How do I get Charm++?}

See our \htmladdnormallink{download}{http://charm.cs.uiuc.edu/download/} page.

\subsubsection{Should I use the GIT version of Charm++?}

The developers of Charm++ routinely use the latest GIT versions, and most of the
time this is the best case. Occasionally something breaks, but the GIT version
will likely contain bug fixes not found in the releases.

\subsubsection{How do I compile Charm++?}

Run the interactive build script {\tt ./build} with no extra arguments If this fails,
email \htmladdnormallink{charm AT cs.uiuc.edu}{mailto:charm AT cs.uiuc.edu} with the
problem. Include the build line used (this is saved automatically in
{\tt smart-build.log})

If you have a very unusual machine configuration, you will have to run
{\tt ./build\ --help} to list all possible build options. You will then choose the closest
architecture, and then you may have to modify the associated conf-mach.sh and
conv-mach.h files in src/arch to point to your desired compilers and options. If
you develop a significantly different platform, send the modified files to
\htmladdnormallink{charm AT cs.uiuc.edu}{mailto:charm AT cs.uiuc.edu} so we can include it
in the distribution.

\subsubsection{How do I compile AMPI?}

Run the interactive build script {\tt ./build} and choose the option for building
``Charm++, AMPI, ParFUM, FEM and other libraries''.

\subsubsection{Can I remove part of charm tree after compilation to free disk space?}

Yes.  Keep src, bin, lib, lib\_so, include, tmp.  You will not need tests, examples, doc, contrib for normal usage once you have verified that your build is functional.

\subsubsection{If the interactive script fails, how do I compile Charm++?}

%<p>First, on a typical Linux machine, unpack the tarball, cd into "charm",
%and type <tt>./build AMPI net-linux -g</tt>.

%<p>Next, if your machine is similar to one of the machines below, 
%use the listed "build" command:
%<table border="1">
%<tr><th>Computer</th><th>Type</th><th>Charm Build</th>

%<tr>
%       <td>UIUC CSE <a href="http://www.cse.uiuc.edu/turing/">Turing</a> Cluster</td>
%       <td>Myrinet Linux Cluster. <br>
%               640 2GHz G5 processors as 2-way nodes.</td>
%       <td>
%               Debug: ./build AMPI net-ppc-darwin -g<br>
%               Production: ./build AMPI net-ppc-darwin gm -O3<br>
%               To run a job, use "rjc" to find nodes, then "charmrun".
%       </td>
%</tr>
%<tr>
%       <td>PSC <a href="http://www.psc.edu/machines/tcs/lemieux.html">Lemieux</a></td>
%       <td>HP/Compaq AlphaServer. <br> 
%               3,000 64-bit Alpha processors as 750 4-way SMP nodes. 3TB total RAM.
%       </td>
%       <td>
%               Use: ./build AMPI elan-axp cxx -O<br>
%               To run a short job, use "charmrun".
%               To run a long job, prepare a batch script and use "qsub".
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IBMp690/">Copper</a></td>
%       <td>IBM SP (pSeries 690). <br> 
%               Use up to 32 1.3GHz 64-bit IBM Power4 processors as a single SMP node.
%       </td>
%       <td>
%               32-bit Mode: ./build AMPI mpi-sp -O<br>
%               64-bit Mode: ./build AMPI mpi-sp mpcc64 -O<br>
%               To run an interactive job, use "charmrun".
%               To run a long job, write a batch script and use "llsubmit".
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/XeonCluster/">Tungsten</a></td>
%       <td>Myrinet Linux Cluster. <br>
%               2,560 3.2 GHz Intel Xeon processors as 2-way nodes with Myrinet, 3 GB RAM per node.
%       </td>
%       <td>
%               Use: ./build AMPI net-linux gm -O , or<br>
%                    ./build AMPI net-linux gm icc ifort -O (Intel compilers) , or<br>
%                    ./build AMPI mpi-linux cmpi -O (Champion MPI)<br>
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/TGIA64LinuxCluster/">Mercury</a></td>
%       <td>Myrinet Linux Cluster. <br>
%               1,744 1.3/1.5 GHz Intel Itanium-2 processors as 2-way nodes with Myrinet, 4 GB RAM per node.
%       </td>
%       <td>
%               Use: ./build AMPI net-linux-ia64 gm -O , or<br>
%                    ./build AMPI mpi-linux-ia64 gm -nobs -O (MPICH over GM)<br>
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/SGIAltix/">Cobalt</a></td>
%       <td>SGI Altix 3700 <br>
%               1,024 1.6 GHz Intel Itanium-2 processors as two 512-processor shared memory systems, 1,024 GB RAM in one system and 2,048 GB RAM in the other
%       </td>
%       <td>
%               Use: ./build AMPI mpi-linux-ia64 ifort mpt icc -O <br>
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IA32LinuxCluster/">Platinum</a></td>
%       <td>Myrinet Linux Cluster with VMI. <br>
%               968 1 GHz 32-bit Intel Pentium III processors as 2-way nodes with Myrinet.
%       </td>
%       <td>
%               Debug: ./build AMPI net-linux gm icc -O<br>
%               Production: ./build AMPI mpi-linux vmi icc -O<br>
%               To run an interactive job with net version, use "charmrun". To run with mpi version, use vmirun. Checkout NCSA's webpage for how to write a job script.<br>
%       </td>
%</tr>
%<tr>
%       <td>NCSA <a href="http://www.ncsa.uiuc.edu/UserInfo/Resources/Hardware/IA32LinuxCluster/">Titanium</a></td>
%       <td>Myrinet Itanium Linux Cluster. <br>
%               256 800 MHz 64-bit Intel Itanium processors as 2-way nodes with Myrinet.
%       </td>
%       <td>
%               Use: ./build AMPI net-linux-ia64 gm ecc -O , or<br>
%                    ./build AMPI mpi-linux-ia64 vmi ecc -O<br>
%               To run an interactive job with net version, use "charmrun". To run with mpi version, use vmirun.<br>
%       </td>
%</tr>
%<tr>
%       <td><a href=http://www.hpcx.ac.uk/">HPCx</td>
%       <td>IBM eServer 575 LPARs with HPC Federation interconnect. <br>
%               96 1.5 GHz 64-bit Power5</td>
%       <td>Use: ./build AMPI lapi -O3 -qstrict
%               "charmrun" will allocate jobs to the load leveler. A prototype file is needed for the accounting</td>
%</tr>
%<tr>
%       <td>Architecture Cluster</a></td>
%       <td>Fast Ethernet Linux Cluster. <br>
%               140 1.7 GHz 32-bit Athlon MP 2000 processors as 2-way nodes.
%       </td>
%       <td>
%               Debug: ./build AMPI net-linux -g<br>
%               Production: ./build AMPI net-linux -O<br>
%               To run interactive use "frun", "fsub" for batch submission.<br> 
%       </td>
%</tr>

%</table>

%<br>
%For a computer not in the list above, you can decide which build options to use based
%on the following rules of thumb.
%<ul>
%<li>The compiler defaults to gcc or mpicc on most platforms.  
%If you'd like to use another compiler, you can usually add an 
%option like "icc", "pgcc", "xlc", "xlc64", or many others--
%see the README file or charm/src/arch/common for a complete list.
%For platform specific options, one can run build command with "help" option, for example: <br>
%./build charm++ net-linux help

%<li>Under Linux, if you or your communication layer uses pthreads,
%and you use Charm threads (such as AMPI), and your Glibc is less than
%version 3.2, you'll get a horrible crash unless you add the "smp" option.
%This version links in a special version of pthreads that works with
%our user-level threads.

%<li>On the net- version, if you're having trouble with rsh/ssh or just
%want to spawn processes on your own machine, add the "local" option, 
%like <tt>./build AMPI net-linux local -g</tt>

%<li>Run "./build --help" to display detailed help page for supported build options

%</ul>

%See the README file for extensive details.
%<br>
%<br>

%<li>
%<b>How do I set up my linux box for parallel runs?</b>
%or <b>What does "rsh> protocol failure in circuit setup" mean?</b>
%</li>

%<br>Charmrun normally uses rsh to start the individual programs in a parallel
%run. If you can't execute "rsh localhost echo Hello", you need to set up rsh.

%<p>Rsh is normally started by the inetd service.  For pre-RedHat 7 Linux systems,
%you ask inetd to start rsh by adding, as root, this line to /etc/inetd.conf:
%<pre>
%shell  stream  tcp     nowait.1000     root    /usr/sbin/tcpd  in.rshd
%</pre>
%You'll then need to restart inetd using "/etc/rc.d/init.d/inetd restart".

%<p>On a modern Linux system you probably don't have an /etc/inetd.conf file,
%because your system uses xinetd.  With xinetd, you enable rsh by adding,
%as root, a file called "rsh" to the directory /etc/xinetd.d/ containing:
%<pre>
%# default: on
%# description: The rshd server is the server for the rcmd(3) routine and, \
%#      consequently, for the rsh(1) program.  The server provides \
%#      remote execution facilities with authentication based on \
%#      privileged port numbers from trusted hosts.
%service shell
%{
%       socket_type             = stream
%       wait                    = no
%       user                    = root
%       only_from               = 127.0.0.1 ...your subnet address here.../24
%       cps                     = 1000 30
%       log_on_success          += USERID
%       log_on_failure          += USERID
%       server                  = /usr/sbin/in.rshd
%}
%</pre>

%Afer adding this file, you'll need to restart xinetd using "/etc/rc.d/init.d/xinetd restart".

%<p>Finally, no matter what kind of inetd you have, you probably also want a
%/etc/hosts.equiv file.  This file contains a list of hosts you want to be able to
%rsh from without using a password.  Since Charm never uses passwords with rsh,
%you want to add all the machines you'll be starting Charm jobs from.
%For example, if I start jobs on this machine from localhost and foo.bar.edu,
%my /etc/hosts.equiv file would contain:
%<pre>
%localhost
%foo.bar.edu
%</pre>

See Appendix V of the Charm manual for \htmladdnormallink{Installation and Usage}{http://charm.cs.illinois.edu/manuals/html/charm++/V.html}.


\subsubsection{How do I specify the processors I want to use?}

On machines where MPI has already been wired into the job system, use the --mpiexec flag and -np arguments.

For the net versions, you need to write a nodelist file which lists
all the machine hostnames available for parallel runs.
\begin{alltt}
group main
  host foo1
  host foo2 ++cpus 4
  host foo3.bar.edu
\end{alltt}

%<p>For the net SCYLD version, you don't need nodelist file,
%<tt>charmrun
%+p n</tt> will automatically find the first n available nodes.

For the MPI version, you need to set up an MPI configuration for available
machines as for normal MPI applications.

\subsubsection{How do I use {\em ssh} instead of the deprecated {\em rsh}?}

You need to set up your {\tt .ssh/authorized\_keys} file
correctly. Setup no-password logins using ssh by putting the correct host
key (ssh-keygen) in the file {\tt .ssh/authorized\_keys}.

Finally, in the {\tt .nodelist} file,
you specify the shell to use for remote execution of a program using
the keyword {\em ++shell}.
\begin{alltt}
group main ++shell ssh
  host foo1
  host foo2
  host foo3
\end{alltt}

\subsubsection{Can I use the serial library X from a Charm program?}

Yes.  Some of the known working serial libraries include:
\begin{itemize}
\item The Tcl/Tk interpreter (in NAMD)
\item The Python interpreter (in Cosmo prototype)
\item OpenGL graphics (in graphics demos)
\item Metis mesh partitioning (included with charm)
\item ATLAS, BLAS, LAPACK, ESSL, FFTW, MASSV, ACML, MKL, BOOST
\end{itemize}

In general, any serial library should work fine with Charm++.

\subsubsection{How do I get the command-line switches available for a specific
program?}

Try \begin{alltt}./charmrun ./pgm --help\end{alltt} to see a list of parameters
at the command line. The charmrun arguments are documented in the
\htmladdnormallink{Installation and Usage Manual}{http://charm.cs.uiuc.edu/manuals/html/install/manual.html}
the arguments for the installed libraries are listed in the library manuals.

%<b>Could you tell me how to run speedshop?</b></li>

%<br><b>speedshop</b> is the performance analysis tool on Origin2000. In
%order to use it, one need not specify any special compilation or linking
%options. It can also be used for parallel programs. Here is what to do
%to use speedshop.
%<p>Suppose the name of the executable is <tt>pgm</tt>, and it is an MPI
%program (or a Charm++ program on mpi-origin version). You run:
%<pre>mpirun -np 4 ssrun -ideal pgm &lt;pgm-options></pre>
%<tt>ssrun</tt> is the command that instruments the executable (original
%executable is not modified), and runs it. The
%<tt>-ideal</tt> option specifies
%what data needs to be collected. It is called <b>experiment</b> in speedshop
%terminology.
%<p>This will produce a few files called
%<tt>pgm.ideal.*</tt>. To be precise,
%when <tt>-np</tt> is 4, it will produce 5 files: one of them is
%<tt>pgm.ideal.m*</tt>,
%and the rest are
%<tt>pgm.ideal.f*</tt>. * is the PID of each process. the
%m* file corresponds to the master program of MPI (and also the lazy thread),
%so it does not contain any user data. The user processes generate <tt>pgm.ideal.f*</tt>
%files.
%<p>Now run prof on it: <tt>prof [-gprof] pgm.ideal.f*</tt>
%<p>This will print the profiling statistics to stdout. The
%<tt>-gprof</tt>
%option asks prof to generate gprof style output.
%<p>There are various interesting experiements one can run with speedshop.
%See speedshop(1) for more details.
%<br>&nbsp;</ol>

\subsubsection{What should I do if my program hangs while gathering
  CPU topology information at startup?}

This is an indication that your cluster's DNS server is not responding
properly. Ideally, the DNS resolver configured to serve your cluster
nodes should be able to rapidly map their hostnames to their IP
addresses. As an immediate workaround, you can run your program with
the {\tt +skip\_cpu\_topology} flag, at the possible cost of reduced
performance. Another workaround is installing and running {\tt nscd},
the ``name service caching daemon'', on your cluster nodes; this may
add some noise on your systems and hence reduce performance. A third
workaround is adding the addresses and names of all cluster nodes in
each node's {\tt /etc/hosts} file; this poses maintainability problems
for ongoing system administration.