3 Copyright (C) 1989-2000 Regents of the University of Illinois
8 Charm++ is a message-passing parallel language and runtime system.
9 It is implemented as a set of libraries for C++, is efficient,
10 and is portable to a wide variety of parallel machines.
11 Source code is provided, and non-commercial use is free.
14 GETTING THE LATEST CHARM SOURCE
15 ===============================
17 You can use our anonymous cvs server to checkout the latest charm++ source code.
18 (It may not be the latest stable version though)
19 What you need to do is as following:
21 1. login the cvs server:
23 cvs -d :pserver:checkout@charm.cs.uiuc.edu:/cvsroot login
25 when CVS password is prompted, just press enter.
30 You should get latest charm source tree.
31 3. logout the cvs server:
39 First, you need to decide which version of charm++ to use. The "build"
40 script in charm source directory takes several command line options to
41 compile Charm++. The command line syntax is:
43 build <target> <version> [options ...] [--basedir=dir] [--libdir=dir] [--incdir=dir] [charmc-options ...]
45 for detailed help messages, use -h or --help to the build script, i.e.
50 <target> specifies the parts of Charm++ to compile. The most often used
51 <target> is "charm++", which will compile the key Charm++ executables and
52 runtime libraries. Other common targets are "AMPI" and "FEM".
53 <versions> defines the CPU, OS and Communication layer of the machines. See
54 "How to choose a <version>" below for details.
58 <options> defines more detailed information of the compilations, including
59 compilers, features to support, etc. See "How to choose <option>"
61 [--libdir=dir] specify additional lib paths for building Charm++.
62 [--incdir=dir] specify additional include paths for building Charm++.
63 [--basedir=dir] a shortcut to specify additional include and lib paths for
64 building Charm++, the include path is dir/include and lib path
68 Running build script, a directory of the name of combination of version and
69 options like "<version>-<option1>-<option2>-..." will be created and
70 the build script will compile Charm++ under this directory.
72 For example, on an ordinary Linux PC:
74 ./build charm++ net-linux
76 will build charm++ in the directory: net-linux/. The communication
77 defaults to UDP packets and the compiler to gcc.
79 For a more complex example, on a Scyld workstation with the Intel C++
80 compiler, where you want the communication to happen over TCP sockets:
82 ./build charm++ net-linux scyld icc tcp
84 will build charm++ in the directory: net-linux-scyld-tcp-icc/.
86 You can specify multiple options, however you can use at most one compiler
87 option. The sequence of the options are not important given in build script,
88 only one directory name will be generated, following the rules:
89 1. compiler option will be at the end;
90 2. other options are sorted alphabetically.
92 **** How to choose a <version> ****
94 Here is the table for choosing correct version. The default setting of compiler
95 in Charm version is gcc/g++. However, one can use <options> to specify other
96 compilers. See the detailed explanation of the <options> below.
97 (Note: this isn't a complete list. Run ./build for a complete listing)
99 Charm version OS Communication Default Compiler
100 ------------- --------- -------------- --------------------
101 net-linux PC Linux UDP/Myrinet GNU compiler
102 net-sol Solaris UDP GNU compiler
103 net-win32 Win32 UDP MS Visual C++
104 net-cygwin Win32/cygwin UDP GNU compiler
105 mpi-sp IBM A/IX MPI A/IX xlC Compiler
106 mpi-origin Origin2000 MPI SGI C++ compiler
108 net-ppc-darwin MacOS X UDP GNU C++ compiler
109 net-linux-ia64 IA64 Linux UDP/Myrinet GNU compiler
110 net-linux-amd64 Opteron Linux UDP GNU compiler
111 net-irix IRIX UDP GNU compiler
112 net-axp Alpha UDP GNU compiler
113 net-hp HP-UX UDP GNU compiler
114 mpi-linux PC Linux MPI GNU compiler
115 mpi-ppc-darwin MacOS X MPI GNU C++ compiler
116 mpi-linux-ia64 IA64 Linux MPI GNU compiler
117 mpi-linux-amd64 Opteron Linux MPI GNU compiler
118 mpi-axp Alpha MPI GNU compiler
119 mpi-linux-axp Alpha Linux MPI GNU compiler
120 origin2000 Origin2000 shared-mem SGI C++ compiler
121 t3e Cray T3E shared-mem Cray C++ compiler
124 To choose <version>, your choice is determined by three options:
126 1.) The way a parallel program written in Charm++ will communicate:
128 "net-" Charm++ communicates using the regular TCP/IP stack
129 (UDP packets), which works everywhere but is fairly slow. Use this
130 option for networks of workstations, clusters, or single-machine
131 development and testing.
133 "mpi-" Charm++ communicates using MPI calls. Use this for
134 machines with a good MPI implementation (such as the Origin 2000).
136 "exemplar", "ncube-2", "paragon-red", "sp3", and "t3e" Charm++
137 communicates using direct calls to the machine's communication primitives.
139 "sim-" and "uth-" are not actively maintained. These are
140 single-processor versions: "uth-" simulates processors as user-level
141 threads; "sim-" switches between processors and counts communications.
144 2.) Your operating system:
147 "win32" MS Windows with MS Visual C++ compiler
148 "cygwin" MS Windows with Cygnus' Cygwin Unix layer
150 "origin" SGI Origin 2000 IRIX
153 "rs6k" IBM R/S 6000 A/IX
155 "hp" Hewlett-Packard HP-UX
156 "axp" DEC Alpha DECUNIX
159 3.) Some operating systems have other options, such as:
160 "-x86" For Solaris, use PC hardware (instead of Sun).
161 "-axp" For Linux, use Alpha hardware (instead of PC).
162 "-ia64" Use Itanium(tm) instructions (instead of x86).
163 "-amd64" Use AMD Opteron instructions (instead of x86).
165 Your Charm++ version is made by concatenating the options, e.g.:
167 "net-linux" Charm++ for a network of Linux workstations, compiled
169 "mpi-origin" Charm++ for SGI Origin 2000, compiled using SGI CC.
172 **** How to choose <options> ****
174 <version> above defines the most important OS, CPU and Communication of
175 your machine, and most of time, it use the GNU gcc as default compiler.
176 To use different compiler or demand additional special feature support, you
177 need to choose <options> from the following list:
179 * gcc3 - GNU GCC/G++ version 3
180 * acc - HP aC++ compiler
181 * cc - For Sun WorkShop C++ compilers;
182 * cc64 - For 64 bits Sun WorkShop C++ or IBM xlC compilers;
183 * cxx - DIGITAL C++ compiler;
184 * kcc - KAI C++ compiler;
185 * pgcc - Portland Group's C++ compiler;
186 * icc - Intel C/C++ compiler for Linux IA32
187 * ecc - Intel C/C++ compiler for Linux IA64
188 * mpcc - SUN Solaris C++ compiler for MPI.
190 * scyld - support Beowulf Scyld based on bproc;
191 * clustermatic - for Clustermatic Beowulf cluster based on bproc;
192 * gm - support MyriCom's Myrinet GM library;
193 * vmi - support NCSA's VMI library;
195 * tcp - for net- version, default communication is via UDP. Using option
196 tcp will switch to TCP. TCP version of CHarm++ is usually slower
197 than UDP, but it is more reliable.
198 * smp - Enable direct SMP support. An "smp" version communicates using
199 shared memory within a machine; but normal message passing across
200 machines. Because of locking, "smp" may slightly impact non-SMP
201 performance. Try your application to decide if enabling smp mode
202 improves performance.
204 * bluegene - compile Charm++ as running on Blue Gene emulator.
205 * help - show supported options for a version. For example, for net-linux,
207 > ./build charm++ net-linux help
209 supported options: gcc3 gm icc kcc pgcc scyld smp bluegene tcp
215 If you have downloaded a binary version of Charm++, you can skip
216 this step-- Charm++ should already be compiled. For win32/win64 systems,
217 see README.win; for Cygwin version, see README.cygwin; for net- version,
220 Once you have decided on a version, unpack Charm++, cd into charm,
223 > ./build <target> <version> <opts>
226 "charm++" The basic Charm++ language.
227 "AMPI" An implementation of MPI on top of Charm++
228 "FEM" A Finite-Element framework on top of Charm++
230 <version> is described above
232 <opts> are build-time options (such as the compiler or "smp"),
233 or command line options passed to the charmc compile script.
234 Common compile time options such as -g, -O, -Ipath, -Lpath, -llib are
237 For example, on a Linux machine, you would run
238 > ./build charm++ net-linux -O
240 This will construct a net-linux directory, link over all
241 the Charm++ source code into net-linux/tmp, build the entire
242 Charm++ runtime system in net-linux/tmp, and link sample programs
245 Several #define's control the compilation of Charm++. Some of these
246 #define's can be found in src/<version>/conv-mach.h. #define's can
247 also be specified on the command line, using the -D option. For
249 > ./build charm++ net-linux -O -DCMK_OPTIMIZE=1
251 CMK_OPTIMIZE: Turn on optimizations in Charm++/Converse. This disables most of
252 the run-time checking performed by Converse and Charm++ runtime. This option
253 should be used only after the program has been debugged. Also, this option
254 disables Converse/Charm++ tracing mechanisms such as projections and summary.
256 When Charm++ is built successfully, the diretory structure under the
257 target directory will look like:
261 --- bin/ # all executables
263 --- doc/ # documentations
265 --- include/ # header files
269 --- lib_so/ # dynamic libraries
271 --- examples/ # all example programs
273 --- tests/ # all test programs
275 --- tmp/ # Charm++ build directory
280 To make a sample program, cd into pgms/charm++/queens/.
281 This program solves the N-queens problem-- find how many ways there
282 are to arrange N queens on an NxN chess board such that none may
285 To build the program, type make. You should get an
286 executable named "pgm".
292 Following the previous example, to run the program on two processors, type
294 > ./charmrun ./pgm 12 6 +p2
296 This should run for a few seconds, and print out:
297 There are 14200 Solutions to 12 queens. Finish time=4.030000
299 Charmrun is used to provide a uniform interface to run charm programs.
300 On some platforms, charmrun is just a shell script which calls the
301 platform-specific start program, such as mpirun on mpi versions.
303 For net- version, charmrun is an executable which invokes rsh or ssh to start
304 node programs on remote machines. You should set up a ~/.nodelist that
305 enumerates all the machines you want to run jobs on, otherwise it will
306 create a default ~/.nodelist for you that contains only localhost. Here is a
307 typical .nodelist file:
309 group main ++shell /bin/ssh
312 The default remote shell program is rsh, but you can define different remote
313 shell you like to start remote processes in the ++shell option. You should
314 also make sure that you can rsh or ssh to these machines without password
315 authentication. Just type following command to verify:
316 > rsh <machinename> date
317 If this gives you current date immediately, your running environment with this
318 node has been setup correctly.
320 Now, for test running purpose, net- version charmrun comes with an easy-to-use
321 "++local" options. No remote shell invocation is needed in this case. It starts
322 node programs right on your local machine. This could be useful if you just
323 want to run program on only one machine, for example, your laptop. This
324 can save you all the hassle of setting up rsh/ssh or charmd daemons.
325 To use this option, just type:
327 > ./charmrun ++local ./pgm 12 100 +p2
329 However, for best performance, you should launch one node program per processor.
331 For more detailed information, please check the "INSTALLATION MANUAL" and "RUN MANUAL"
335 Build Charm++ in Dynamic libraries
336 =============================
338 In order to compile Charm++ into dynamic libraries, one need to specify
339 "-build-shared" option as one of the Charm compiler script "charmc"
340 at link time. For example, to compile Charm++ under net-linux/tmp, run
342 make charm++ OPTS='-O -build-shared'
344 Charm++'s dynamic libraries are compiled into lib_so/ directory.
345 Typically, they are with ".so" suffix.
347 Note, "-build-shared" option is automatically turned on when building
348 Charm++ using "build" script. So you don't need to pass "-build-shared"
351 One can compile a Charm++ applicaiton linking against Charm++ dynamic
352 libraries, run charmc with "-charm-shared" as one of the link options.
355 charmc -o pgm pgm.o -charm-shared
357 You can then run the program as usual.
358 Note, linking against Charm++ dynamic libraries produces much smaller size
359 binaries and takes much less linking time.
364 The Charm++ web page, with documentation, more programs,
365 and the latest version of Charm++, is at
366 http://charm.cs.uiuc.edu/
368 The Charm++ mailing list, for questions, comments, suggestions,
369 improvements, or bug reports is
376 Charm++ was created and is maintained by the Parallel Programming Lab,
377 in the Computer Science department at the University of Illinois at
378 Urbana-Champaign. Our managing professor is Dr. L.V. Kale; students
379 have included (in rough time order) Wennie Shu, Kevin Nomura, Wayne
380 Fenton, Balkrishna Ramkumar, Vikram Saletore, Amitabh B. Sinha, Manish
381 Gupta, Attila Gursoy, Balkrishna Ramkumar, Amitabh B. Sinha, Nimish
382 Shah, Sanjeev Krishnan, Jayant DeSouza, Parthasarathy Ramachandran,
383 Jeff Wright, Michael Lang, Jackie Wang, Fang Hu, Michael Denardo,
384 Joshua Yelon, Narain Jagathesan, Zehra Sura, Krishnan Varadarajan,
385 Sameer Paranjpye, Milind Bhandarkar, Robert Brunner and Jayant Desouza.
386 Current developers include Terry Wilmarth, Gengbin Zheng, Orion Lawlor,
387 Karthik Mahesh, and Neelam Saboo.