doc/bigsim/install.tex

   1 \section{BigSim Emulator Installation and Usage}
   2 \label{install}
   3
   4 \subsection{Installing Charm++ and BigSim}
   5
   6     The BigSim Emulator is distributed as part of the Charm++ standard
   7 distribution.  One needs to download Charm++ and compile the BigSim Simulator.
   8 This process should begin with downloading Charm++ from the website:
   9 http://charm.cs.uiuc.edu.
  10
  11 Please refer to ``Charm++ Installation and Usage Manual'' and also the file
  12 README in the source code for detailed instructions on how to compile Charm++.
  13 In short, the ``build'' script is the main tool for compiling \charmpp{}.  One
  14 needs to provide {\em target} and {\em platform} selections:
  15 \begin{verbatim} ./build <target> <platform> [options ...] [charmc-options ...] \end{verbatim}
  16
  17 For example, to compile on a 64-bit Linux machine, one would type:
  18 \begin{verbatim}
  19 ./build charm++ net-linux-x86_64 -O2
  20 \end{verbatim}
  21
  22 \noindent which builds essential \charmpp{} kernel using UDP sockets as the
  23 communication method; alternatively, it is possible to build the Charm++ kernel
  24 on MPI using:
  25 \begin{verbatim}
  26 ./build charm++ mpi-linux-x86_64 -O2
  27 \end{verbatim}
  28
  29 For other platforms, net-linux-x86\_64 should be replaced by whatever platform is being used.
  30 See the charm/README file for a complete list of supported platforms.
  31
  32 \subsubsection{Building Only the BigSim Emulator}
  33
  34 The BigSim Emulator is implemented on top of Converse in Charm++.  To compile
  35 the BigSim Emulator, one can compile Emulator libraries directly on top of
  36 normal \charmpp{} using ``bgampi'' as the compilation target, like
  37 \begin{verbatim}
  38 ./build bgampi net-linux-x86_64 -O2
  39 \end{verbatim}
  40
  41 With Emulator libraries, one can write BigSim applications using its
  42 low level machine API (defined in ~\ref{bgemulator}).
  43
  44 \subsubsection{Building Charm++ or AMPI on the BigSim Emulator}
  45
  46 In order to build Charm++ or AMPI on top of BigSim Emulator (which itself is
  47 implemented on top of Converse), a special build option ``bigemulator'' needs
  48 to be specified:
  49 \begin{verbatim}
  50 ./build bgampi net-linux-x86_64 bigemulator -O2
  51 \end{verbatim}
  52
  53 The ``bgampi'' option is the compilation {\em target} that tells ``build'' to
  54 compile BigSim Emulator libraries in addition to \charmpp{} kernel libraries.
  55 The ``bigemulator'' option is a build {\em option} to platform ``net-linux'',
  56 which tells ``build'' to build Charm++ on top of the BigSim Emulator.
  57
  58 The above ``build" command creates a directory named
  59 ``net-linux-x86\_64-bigemulator" under charm, which contains all the header
  60 files and libraries needed for compiling a user application.
  61 With this version of Charm++, one can run normal Charm++ and AMPI application
  62 on top of the emulator (in a virtualized environment).
  63
  64 \subsection{Compiling BigSim Applications}
  65
  66 \charmpp{} provides a compiler script {\tt charmc} to compile all programs.  As
  67 will be described in this subsection, there are three methods to write a BigSim
  68 application: (a) using the low level machine API, (b) using \charmpp{} or (c)
  69 using AMPI. Methods (b) and~(c) are essentially used to obtain traces from the
  70 BigSim Emulator, such that one can use those traces in a post-mortem simulation
  71 as explained in Section~\ref{bignetsim}.
  72
  73 \subsubsection{Writing a BigSim application using low level machine API}
  74 The original goal of the low level machine API was to mimic the BlueGene/C low
  75 level programming API. It is defined in section~\ref{bgemulator}. Writing a
  76 program in the low level machine API, one just needs to link \charmpp{}'s
  77 BigSim emulator libraries, which provide the emulation of the machine API using
  78 Converse as the communication layer.
  79
  80 In order to link against the BigSim library, one must specify
  81 \texttt{-language bigsim} as an argument to the {\tt charmc} command, for
  82 example:
  83
  84 \begin{verbatim}
  85 charmc -o hello hello.C -language bigsim
  86 \end{verbatim}
  87
  88 Sample applications in low level machine API can be found in the directory
  89 charm/examples/bigsim/emulator/.
  90
  91 \subsubsection{Writing a BigSim application in Charm++}
  92
  93 One can write a normal \charmpp{} application which can automatically run on
  94 the BigSim Emulator after compilation. \charmpp{} implements an object-based
  95 message-driven execution model. In \charmpp{} applications, there are
  96 collections of C++ objects, which communicate by remotely invoking methods on
  97 other objects via messages.
  98
  99 To compile a program written in \charmpp{} on the BigSim Emulator, one
 100 specifies \texttt{-language charm++} as an argument to the {\tt charmc}
 101 command:
 102 \begin{verbatim}
 103 charmc -o hello hello.C -language charm++
 104 \end{verbatim}
 105 This will link both \charmpp{} runtime libraries and BigSim Emulator libraries.
 106
 107 Sample applications in \charmpp{} can be found in the directory
 108 charm/examples/bigsim, specifically charm/examples/bigsim/emulator/littleMD.
 109
 110 \subsubsection{Writing a BigSim application in MPI}
 111
 112 One can also write an MPI application for the BigSim Emulator.  Adaptive MPI,
 113 or AMPI, is implemented on top of Charm++, supporting dynamic load balancing
 114 and multithreading for MPI applications. Those are based on the user-level
 115 migrating threads and load balancing capabilities provided by the \charmpp{}
 116 framework. This allows legacy MPI programs to run on top of BigSim \charmpp{}
 117 and take advantage of the \charmpp{}'s virtualization and adaptive load
 118 balancing capability.
 119
 120 Currently, AMPI implements most features in the MPI version 1.0, with a few
 121 extensions for migrating threads and asynchronous reduction.
 122
 123 To compile an AMPI application for the BigSim Emulator, one needs to link
 124 against the AMPI library as well as the BigSim \charmpp{} runtime libraries by
 125 specifying \texttt{-language ampi} as an argument to the {\tt charmc} command:
 126 \begin{verbatim}
 127 charmc -o hello hello.C -language ampi
 128 \end{verbatim}
 129
 130 Sample applications in AMPI can be found in the directory charm/examples/ampi,
 131 specifically \\
 132 charm/examples/ampi/pingpong.
 133
 134 \subsection{Running a BigSim Application}
 135
 136 To run a parallel BigSim application, \charmpp{} provides a utility program
 137 called {\tt charmrun} that starts the parallel execution.  For detailed
 138 description on how to run a \charmpp{} application, refer to the file
 139 charm/README in the source code distribution.
 140
 141 To run a BigSim application, one needs to specify the following parameters to
 142 {\tt charmrun} to define the simulated machine size:
 143
 144 \begin{enumerate}
 145
 146 \item {\tt +vp}: define the number of processors of the hypothetical (future)
 147 system
 148 \item {\tt +x, +y} and {\tt +z}:  optionally define the size of the machine in
 149 three dimensions, these define the number of nodes along each dimension of the
 150 machine (assuming a torus/mesh topology);
 151 \item {\tt +wth} and {\tt +cth}:  For one node, these two parameters define the
 152 number of worker processors~({\tt +wth}) and the number of communication
 153 processors~({\tt +cth}).
 154 \item {\tt +bgwalltime}: used only in simulation mode, when specified, use
 155 wallclock measurement of the time taken on the simulating machine to estimate
 156 the time it takes to run on the target machine.
 157 \item {\tt +bgcounter}:  used only in simulation mode, when specified, use the
 158 performance counter to estimate the time on target machine. This is currently
 159 only supported when perfex is installed, like Origin2000.
 160 \item {\tt +bglog}: generate BigSim trace log files, which can be used with BigNetSim.
 161 \item {\tt +bgcorrect}: starts the simulation mode to predict performance.
 162 Without this option, a program simply runs on the emulator without doing any
 163 performance prediction.  Note: this option is obsolete, and no longer
 164 maintained, use +bglog to generate trace logs, and use BigNetSim for
 165 performance prediction.  \end{enumerate}
 166
 167 For example, to simulate a parallel machine of size 64K as 40x40x40, with one
 168 worker processor and one communication processor on each node, and use 100 real
 169 processors to run the simulation, the command to be issued should be:
 170 \begin{verbatim}
 171 ./charmrun +p100 ./hello +x40 +y40 +z40 +cth1 +wth1
 172 \end{verbatim}
 173
 174 To run an AMPI program, one may also want to specify the number of virtual
 175 processors to run the MPI code by using {\tt +vp}. As an example,
 176 \begin{verbatim}
 177 ./charmrun +p100 ./hello +x40 +y40 +z40 +cth1 +wth1 +vp 128000
 178 \end{verbatim}
 179 \noindent starts the simulation of a machine   of size 40x40x40 with one worker
 180 processor in each node, running 128000 MPI tasks (2 MPI tasks on each node),
 181 using 100 real processors to run the simulation. In this case, {\tt
 182 MPI\_Comm\_size()} returns 128000 for {\tt MPI\_COMM\_WORLD}.  If the {\tt +vp}
 183 option is not specified, the number of virtual processors will be equal to the
 184 number of worker processors of the simulated machine, in this case 64000.
 185