doc/charm++/shrinkexpand.tex

   1 \experimental{}
   2 This feature enables a \charmpp{} application to dynamically shrink and expand
   3 the number of processors that it is running on during the execution.
   4 It internally uses three other features of \charmpp{}:
   5 CCS (Converse Client Server) interface, load balancing, and checkpoint restart.
   6 \begin{itemize}
   7 \item The CCS interface is used to send and receive the shrink and expand commands.
   8 These commands can be internal (i.e. application decides to shrink or expand
   9 itself) or external via a client. The runtime listens for the commands on
  10 a port specified at launch time.
  11 \item The load balancing framework is used to evacuate the tasks before a shrink
  12 operation and distribute the load equally over the new set of processors
  13 after an expand operation.
  14 \item The in-memory checkpoint restart mechanism is used to restart the application
  15 with the new processor count quickly and without leaving residual processes behind.
  16 \end{itemize}
  17
  18 An example program with a CCS client to send shrink/expand commands
  19 can be found in \texttt{examples/charm++/shrink\_expand}
  20 in the charm distribution.
  21
  22 To enable shrink expand, \charmpp{} needs to be built with the \texttt{--enable-shrinkexpand}
  23 option:
  24 \begin{alltt}
  25         ./build charm++ netlrts-linux-x86_64 --enable-shrinkexpand
  26 \end{alltt}
  27
  28 An example application launch command needs to include a load balancer, a
  29 nodelist file that contains all of the nodes that are going to be used, and a
  30 port number to listen the shrink/expand commands:
  31 \begin{alltt}
  32         ./charmrun +p4 ./jacobi2d 200 20 +balancer GreedyLB ++nodelist ./mynodelistfile ++server ++server-port 1234
  33 \end{alltt}
  34
  35 The CCS client to send shrink/expand commands needs to specify the hostname,
  36 port number, the old(current) number of processor and the new(future) number of
  37 processors:
  38 \begin{alltt}
  39         ./client <hostname> <port> <oldprocs> <newprocs>
  40         (./client valor 1234 4 8 //This will increase from 4 to 8 processors.)
  41 \end{alltt}
  42
  43 To make a \charm{} application malleable, first, pup routines for
  44 all of the constructs in the application need to be written. This includes
  45 writing a pup routine for the \texttt{mainchare} and marking it migratable:
  46 \begin{alltt}
  47         mainchare [migratable]  Main { ... }
  48 \end{alltt}
  49
  50 Second, the \texttt{AtSync()} and \texttt{ResumeFromSync()} functions need to be
  51 implemented in the usual way of doing load balancing (See Section~\ref{lbStrategy}
  52 for more info on load balancing).
  53 Shrink/expand will happen at the next load balancing step after the receipt
  54 of the shrink/expand command.
  55
  56 \textbf{NOTE:} If you want to shrink your application, for example, from two physical nodes to one
  57 node where each node has eight cores, then you should have eight entries in the nodelist file for
  58 each node, one per processor. Otherwise, the application will shrink in a way that will use four
  59 cores from each node, whereas what you likely want is to use eight cores on only one of the
  60 physical nodes after shrinking.  For example, instead of having a nodelist like this:
  61 \begin{alltt}
  62         host a
  63         host b
  64 \end{alltt}
  65
  66 the nodelist should be like this:
  67 \begin{alltt}
  68         host a
  69         host a
  70         host a
  71         host a
  72         host a
  73         host a
  74         host a
  75         host a
  76         host b
  77         host b
  78         host b
  79         host b
  80         host b
  81         host b
  82         host b
  83         host b
  84 \end{alltt}
  85
  86 \textbf{Warning: this is an experimental feature and not supported in all charm
  87 builds and applications.}
  88 Currently, it is tested on \texttt{netlrts-\{linux/darwin\}-x86\_64} builds.
  89 Support for other \charmpp{} builds and AMPI applications are under development.
  90 It is only tested with \texttt{RefineLB} and \texttt{GreedyLB} load balancing
  91 strategies; use other strategies with caution.