doc/charm++/alltoall.tex

   1 \section{All-to-All}
   2 All-to-All is a frequently encountered pattern of communication in
   3 parallel programs where each processing element sends a message to
   4 every other processing element. Variations on this pattern are also
   5 common. A processing element may want to send multiple messages to the
   6 same destination over time, for example, and not every pair of
   7 processors may need to communicate. In Charm++ we classify these
   8 scenarios under a single API with the aim of improving any type of
   9 Many-to-Many communication pattern.
  10
  11 Note that we are currently extending support for All-to-All
  12 communication in Charm++ and so the API may change in the
  13 future.
  14
  15 \subsection{MeshStreamer}
  16
  17 MeshStreamer optimizes the case of All-to-All and Many-to-Many
  18 communication on regular 2D and 3D machine topologies. Messages sent
  19 using MeshStreamer are routed along the dimensions of the specified
  20 topology and aggregated at intermediate destinations. When using it,
  21 the first step is to create a MeshStreamer group.
  22
  23 \begin{alltt}
  24 MeshStreamer(int totalBufferCapacity, int numRows,
  25              int numColumns, int numPlanes,
  26              const CProxy_MeshStreamerClient<dtype> &clientProxy,
  27              int yieldFlag = 0, int progressPeriodInMs = -1);
  28 \end{alltt}
  29
  30 The constructor takes as input a reference to a MeshStreamerClient
  31 proxy. The user should pass in the proxy for the group which will
  32 receive the data sent using MeshStreamer. To do so, this group should
  33 inherit from the MeshStreamerClient group. Note that MeshStreamer and
  34 MeshStreamerClient are templated. The templated parameter specifies
  35 the type of data units which will be communicated.
  36
  37 The totalBufferCapacity parameter for the MeshStreamer constructor
  38 specifies the buffering limit of the library. When the collective
  39 number of items buffered by the local instance of the group reaches
  40 the specified limit, the library sends a message along each dimension
  41 to the destination for which it has buffered the most messages.
  42
  43 MeshStreamer employs a virtual topology to route messages. The
  44 topology is specified by the user. When a regular mesh partition is
  45 avilable for execution, performance will be much better if the
  46 dimensions of the virtual topology submitted by the user correspond to
  47 the physical dimensions of the machine topology. The Charm++ Topology
  48 Manager can be used to produce this information for the user at run
  49 time.
  50
  51 The insertData function, best used when called on the local instance
  52 of the MeshStreamer group, hands over individual units of data for
  53 transmission by the library.
  54
  55 \begin{alltt}
  56 void insertData(dtype &dataItem, const int destinationPe);
  57 \end{alltt}
  58
  59 To receive items, the user needs to define a process function, which
  60 is a pure virtual function of MeshStreamerClient.
  61
  62 \begin{alltt}
  63 virtual void process(dtype &data)=0;
  64 \end{alltt}
  65
  66 MeshStreamer aggregates items into messages which are sent out when
  67 internal buffers fill up or periodic time limits are reached. The
  68 message arriving at the destination index of the MeshStreamer group
  69 may contain items from various group indices. The receiveCombinedData
  70 function loops over the received items and calls the process function
  71 for each item. The user may choose to redefine this function to
  72 specify an alternate message processing behavior.
  73
  74 \begin{alltt}
  75 virtual void receiveCombinedData(MeshStreamerMessage<dtype> *msg);
  76 \end{alltt}
  77