| blob | 415c2c17dd37b3f0fb5694f32d8361c41aacb464 |
1 This file describes the most significant changes. For more detail, use
2 'git log' on a clone of the charm repository.
4 ================================================================================
5 What's new in Charm++ 6.8.1
6 ================================================================================
8 This is a backwards-compatible patch/bug-fix release. Roughly 100 bug
9 fixes, improvements, and cleanups have been applied across the entire
10 system. Notable changes are described below:
12 General System Improvements
14 - Enable network- and node-topology-aware trees for group and chare
15 array reductions and broadcasts
17 - Add a message receive 'fast path' for quicker array element lookup
19 - Feature #1434: Optimize degenerate CkLoop cases
21 - Fix a rare race condition in Quiescence Detection that could allow
22 it to fire prematurely (bug #1658)
23 * Thanks to Nikhil Jain (LLNL) and Karthik Senthil for isolating
24 this in the Quicksilver proxy application
26 - Fix various LB bugs
27 * Fix RefineSwapLB to properly handle non-migratable objects
28 * GreedyRefine: improvements for concurrent=false and HybridLB integration
29 * Bug #1649: NullLB shouldnt wait for LB period
31 - Fix Projections tracing bug #1437: CkLoop work traces to the
32 previous entry on the PE rather than to the caller
34 - Modify [aggregate] entry method (TRAM) support to only deliver
35 PE-local messages inline for [inline]-annotated methods. This avoids
36 the potential for excessively deep recursion that could overrun
37 thread stacks.
39 - Fix various compilation warnings
41 Platform Support
43 - Improve experimental support for PAMI network layer on POWER8 Linux platforms
44 * Thanks to Sameer Kumar of IBM for contributing these patches
46 - Add an experimental 'ofi' network layer to run on Intel Omni-Path
47 hardware using libfabric
48 * Note that support for '+partitions' (also used as '+replicas' by
49 NAMD) is known to be broken, currently (bug #1675)
50 * Thanks to Yohann Burette and Mikhail Shiryaev of Intel for
51 contributing this new network layer
53 - The GNI network layer (used on Cray XC/XK/XE systems) now respects
54 the ++quiet command line argument during startup
56 AMPI Improvements
58 - Support for MPI_IN_PLACE in all collectives and for persistent requests
60 - Improved Alltoall(v,w) implementations
62 - AMPI now passes all MPICH-3.2 tests for groups, virtual topologies, and infos
64 - Fixed Isomalloc to not leave behind mapped memory when migrating off a PE
66 ================================================================================
67 What's new in Charm++ 6.8.0
68 ================================================================================
70 Over 900 bug fixes, improvements, and cleanups have been applied
71 across the entire system. Major changes are described below:
73 Charm++ Features
75 - Calls to entry methods taking a single fixed-size parameter can now
76 automatically be aggregated and routed through the TRAM library by
77 marking them with the [aggregate] attribute.
79 - Calls to parameter-marshalled entry methods with large array
80 arguments can ask for asynchronous zero-copy send behavior with an
81 `nocopy' tag in the parameter's declaration.
83 - The runtime system now integrates an OpenMP runtime library so that
84 code using OpenMP parallelism will dispatch work to idle worker
85 threads within the Charm++ process.
87 - Applications can ask the runtime system to perform automatic
88 high-level end-of-run performance analysis by linking with the
89 `-tracemode perfReport' option.
91 - Added a new dynamic remapping/load-balancing strategy,
92 GreedyRefineLB, that offers high result quality and well bounded
93 execution time.
95 - Improved and expanded topology-aware spanning tree generation
96 strategies, including support for runs on a torus with holes, such
97 as Blue Waters and other Cray XE/XK systems.
99 - Charm++ programs can now define their own main() function, rather
100 than using a generated implementation from a mainmodule/mainchare
101 combination. This extends the existing Charm++/MPI interoperation
102 feature.
104 - Improvements to Sections:
106 * Array sections API has been simplified, with array sections being
107 automatically delegated to CkMulticastMgr (the most efficient implementation
108 in Charm++). Changes are reflected in Chapter 14 of the manual.
110 * Group sections can now be delegated to CkMulticastMgr (improved performance
111 compared to default implementation). Note that they have to be manually
112 delegated. Documentation is in Chapter 14 of Charm++ manual.
114 * Group section reductions are now supported for delegated sections
115 via CkMulticastMgr.
117 * Improved performance of section creation in CkMulticastMgr.
119 * CkMulticastMgr uses the improved spanning tree strategies. See above.
121 - GPU manager now creates one instance per OS process and scales the
122 pre-allocated memory pool size according to the GPU memory size and
123 number of GPU manager instances on a physical node.
125 - Several GPU Manager API changes including:
127 * Replaced references to global variables in the GPU manager API with calls to
128 functions.
130 * The user is no longer required to specify a bufferID in dataInfo struct.
132 * Replaced calls to kernelSelect with direct invocation of functions passed
133 via the work request object (allows CUDA to be built with all programs).
135 - Added support for malleable jobs that can dynamically shrink and
136 expand the set of compute nodes hosting Charm++ processes.
138 - Greatly expanded and improved reduction operations:
140 * Added built-in reductions for all logical and bitwise operations
141 on integer and boolean input.
143 * Reductions over groups and chare arrays that apply commutative,
144 associative operations (e.g. MIN, MAX, SUM, AND, OR, XOR) are now
145 processed in a streaming fashion. This reduces the memory footprint of
146 reductions. User-defined reductions can opt into this mode as well.
148 * Added a new `Tuple' reducer that allows combining multiple reductions
149 of different input data and operations from a common set of source
150 objects to a single target callback.
152 * Added a new `Summary Statistics' reducer that provides count, mean,
153 and standard deviation using a numerically-stable streaming algorithm.
155 - Added a `++quiet' option to suppress charmrun and charm++ non-error
156 messages at startup.
158 - Calls to chare array element entry methods with the [inline] tag now
159 avoid copying their arguments when the called method takes its
160 parameters by const&, offering a substantial reduction in overhead in
161 those cases.
163 - Synchronous entry methods that block until completion (marked with
164 the [sync] attribute) can now return any type that defines a PUP
165 method, rather than only message types.
167 - Static (non-generated) header files are now warning-free for
168 gcc -Wall -Wextra -pedantic.
170 - Deprecated setReductionClient and CkSetReductionClient in favor of
171 explicitly passing callbacks to contribute calls.
173 - On C++ standard library implementations with support for
174 std::is_constructible (e.g. GCC libstdc++ >4.5), chare array
175 elements only need to define a constructor taking CkMigrateMessage*
176 if it will actually be migrated.
178 - The PUP serialization framework gained support for some C++11
179 library classes, including unique_ptr and unordered_map, when the
180 underlying types have PUP operators.
182 AMPI Features
184 - More efficient implementations of message matching infrastructure, multiple
185 completion routines, and all varieties of reductions and gathers.
187 - Support for user-defined non-commutative reductions, MPI_BOTTOM, cancelling
188 receive requests, MPI_THREAD_FUNNELED, PSCW synchronization for RMA, and more.
190 - Fixes to AMPI's extensions for load balancing and to Isomalloc on SMP builds.
192 - More robust derived datatype support, optimizations for truly contiguous types.
194 - ROMIO is now built on AMPI and linked in by ampicc by default.
196 - A version of HDF5 v1.10.1 that builds and runs on AMPI with virtualization
197 is now available at https://charm.cs.illinois.edu/gerrit/#/admin/projects/hdf5-ampi
199 - Improved support for performance analysis and visualization with Projections.
201 Platforms and Portability
203 - The runtime system code now requires compiler support for C++11
204 R-value references and move constructors. This is not expected to be
205 incompatible with any currently supported compilers.
207 - The next feature release (anticipated to be 6.9.0 or 7.0) will require
208 full C++11 support from the compiler and standard library.
210 - Added support for IBM POWER8 systems with the PAMI communication API,
211 such as development/test platforms for the upcoming Sierra and Summit
212 supercomputers at LLNL and ORNL. Contributed by Sameer Kumar of IBM.
214 - Mac OS (darwin) builds now default to the modern libc++ standard
215 library instead of the older libstdc++.
217 - Blue Gene/Q build targets have been added for the `bgclang' compiler.
219 - Charm++ can now be built on Cray's CCE 8.5.4+.
221 - Charm++ will now build without custom configuration on Arch Linux
223 - Charmrun can automatically detect rank and node count from
224 Slurm/srun environment variables.
226 - Many obsolete architecture, network, and compiler support files have
227 been removed. These include:
228 * IBM Blue Gene/P
229 * Sony/Toshiba/IBM Cell (including PlayStation 3)
230 * Cray XT
231 * Intel IA-64 (Itanium)
232 * Intel x86-32 for Windows, Mac OS X (darwin), and Solaris
233 * Cygwin for Windows
234 * Older IBM AIX/POWER configurations
235 * GCC 3 and KAI compilers
236 * Sun/Oracle Solaris
238 ================================================================================
239 What's new in Charm++ 6.7.1
240 ================================================================================
242 Changes in this release are primarily bug fixes for 6.7.0. The major exception
243 is AMPI, which has seen changes to its extension APIs and now complies with more
244 of the MPI standard. A brief list of changes follows:
246 Charm++ Bug Fixes
248 - Startup and exit sequences are more robust
250 - Error and warning messages are generally more informative
252 - CkMulticast's set and concat reducers work correctly
254 AMPI Features
256 - AMPI's extensions have been renamed to use the prefix AMPI_ instead of MPI_
257 and to generally follow MPI's naming conventions
259 - AMPI_Migrate(MPI_Info) is now used for dynamic load balancing and all fault
260 tolerance schemes (see the AMPI manual)
262 - AMPI officially supports MPI-2.2, and also implements the non-blocking
263 collectives and neighborhood collectives from MPI-3.1
265 Platforms and Portability
267 - Cray regularpages build target has been fixed
269 - Clang compiler target for BlueGene/Q systems added
271 - Comm. thread tracing for SMP mode added
273 - AMPI's compiler wrappers are easier to use with autoconf and cmake
275 ================================================================================
276 What's new in Charm++ 6.7.0
277 ================================================================================
279 Over 120 bugs fixed, spanning areas across the entire system
281 Charm++ Features
283 - New API for efficient formula-based distributed sparse array creation
285 - CkLoop is now built by default
287 - CBase_Foo::pup need not be called from Foo::pup in user code anymore - runtime
288 code handles this automatically
290 - Error reporting and recovery in .ci files is greatly improved, providing more
291 precise line numbers and often column information
293 - Many data races occurring under shared-memory builds (smp, multicore) were
294 fixed, facilitating use of tools like ThreadSanitizer and Helgrind
296 AMPI Enhancements
298 - Further MPI standard compliance in AMPI allows users to build and run
299 Hypre-2.10.1 on AMPI with virtualization, migration, etc.
301 - Improved AMPI Fortran2003 PUP interface 'apup', similar to C++'s STL PUP
303 Platforms and Portability
305 - Compiling Charm++ now requires support for C++11 variadic templates. In GCC,
306 this became available with version 4.3, released in 2008
308 - New machine target for multicore Linux ARM7: multicore-linux-arm7
310 - Preliminary support for POWER8 processors, in preparation for the upcoming
311 Summit and Sierra supercomputers
313 - The charmrun process launcher is now much more robust in the face of slow
314 or rate-limited connections to compute nodes
316 - PXSHM now auto-detects the node size, so the '+nodesize' is no longer needed
318 - Out-of-tree builds are now supported
320 Deprecations
322 - CommLib has been removed.
324 - CmiBool has been dropped in favor of C++'s bool
327 ================================================================================
328 What's new in Charm++ 6.6.1
329 ================================================================================
331 Changes in this release are primarily bug fixes for 6.6.0. A concise list of
332 affected components follows:
334 - CkIO
336 - Reductions with syncFT
338 - mpicxx based MPI builds
340 - Increased support for macros in CI file
342 - GNI + RDMA related communication
344 - MPI_STATUSES_IGNORE support for AMPIF
346 - Restart on different node count with chkpt
348 - Immediate msgs on multicore builds
350 ================================================================================
351 What's new in Charm++ 6.6.0
352 ================================================================================
354 - Machine target files for Cray XC systems ('gni-crayxc') have been added
356 - Interoperability with MPI code using native communication interfaces on Blue
357 Gene Q (PAMI) and Cray XE/XK/XC (uGNI) systems, in addition to the universal
358 MPI communication interface
360 - Support for partitioned jobs on all machine types, including TCP/IP and IB
361 Verbs networks using 'netlrts' and 'verbs' machine layers
363 - A substantially improved version of our asynchronous library, CkIO, for
364 parallel output of large files
366 - Narrowing the circumstances in which the runtime system will send
367 overhead-inducing ReductionStarting messages
369 - A new fully distributed load balancing strategy, DistributedLB, that produces
370 high quality results with very low latency
372 - An API for applications to feed custom per-object data to specialized load
373 balancing strategies (e.g. physical simulation coordinates)
375 - SMP builds on LRTS-based machine layers (pamilrts, gni, mpi, netlrts, verbs)
376 support tracing messages through communication threads
378 - Thread affinity mapping with +pemap now supports Intel's Hyperthreading more
379 conveniently
381 - After restarting from a checkpoint, thread affinity will use new
382 +pemap/+commap arguments
384 - Queue order randomization options were added to assist in debugging race
385 conditions in application and runtime code
387 - The full runtime code and associated libraries can now compile under the C11
388 and C++11/14 standards.
390 - Numerous bug fixes, performance enhancements, and smaller improvements in the
391 provided runtime facilities
393 - Deprecations
394 * The long-unsupported FEM library has been deprecated in favor of ParFUM
395 * The CmiBool typedefs have been deleted, as C++ bool has long been universal
396 * Future versions of the runtime system and libraries will require some degree
397 of support for C++11 features from compilers
399 ================================================================================
400 What's new in Charm++ 6.5.0
401 ================================================================================
403 - The Charm++ manual has been thoroughly revised to improve its organization,
404 comprehensiveness, and clarity, with many additional example code snippets
405 throughout.
407 - The runtime system now includes the 'Metabalancer', which can provide
408 substantial performance improvements for applications that exhibit dynamic
409 load imbalance. It provides two primary benefits. First, it automatically
410 optimizes the frequency of load balancer invocation, to avoid work stoppage
411 when it will provide too little benefit. Second, calls to AtSync() are made
412 less synchronous, to further reduce overhead when the load balancer doesn't
413 need to run. To activate the Metabalancer, pass the option +MetaLB at
414 runtime. To get the full benefits, calls to AtSync() should be made at every
415 iteration, rather than at some arbitrary longer interval as was previously
416 common.
418 - Many feature additions and usability improvements have been made in the
419 interface translator that generates code from .ci files:
420 * Charmxi now provides much better error reports, including more accurate
421 line numbers and clearer reasons for failure, including some semantic
422 problems that would otherwise appear when compiling the C++ code or even at
423 runtime.
424 * A new SDAG construct 'case' has been added that defines a disjunction over a
425 set of 'when' clauses: only one 'when' out of a set will ever be triggered.
426 * Entry method templates are now supported. An example program can be found
427 in tests/charm++/method_templates/.
428 * SDAG keyword "atomic" has been deprecated in favor of the newly supported
429 keyword "serial". The two are synonymous, but "atomic" is now provided only
430 for backward compatibility.
431 * It is no longer necessary to call __sdag_init() in chares that contain SDAG
432 code - the generated code does this automatically. The function is left as
433 a no-op for compatibility, but may be removed in a future version.
434 * Code generated from .ci files is now primarily in .def.h files, with only
435 declarations in .decl.h. This improves debugging, speeds compilation,
436 provides clearer compiler output, and enables more complete encapsulation,
437 especially in SDAG code.
438 * Mainchare constructors are expected to take CkArgMsg*, and always have
439 been. However, charmxi would allow declarations with no argument, and
440 assume the message. This is now deprecated, and generates a warning.
442 - Projections tracing has been extended and improved in various ways
443 * The trace module can generate a record of network topology of the nodes in
444 a run for certain platforms (including Cray), which Projections can
445 visualize.
446 * If the gzip library (libz) is available when Charm++ is compiled, traces
447 are compressed by default.
448 * If traces were flushed as a results of filled buffers during the run, a
449 warning will be printed at exit to indicate that the user should be wary of
450 interference that may have resulted.
451 * In SMP builds, it is now possible to trace message progression through the
452 communication threads. This is disabled by default to avoid overhead and
453 potential misleading interpretation.
455 - Array elements can be block-mapped at the SMP node level instead of at the
456 per-PE level (option "+useNodeBlkMapping").
458 - AMPI can now privatize global and static variables using TLS. This is
459 supported in C and C++ with __thread markings on the variable declarations
460 and definitions, and in Fortran with a patched version of the gfortran
461 compiler. To activate this feature, append '-tls' to the '-thread' option's
462 argument when you link your AMPI program.
464 - Charm can now be built to only support message priorities of a specific data
465 type. This enables an optimized message queue within the the runtime
466 system. Typical applications with medium sized compute grains may not benefit
467 noticeably when switching to the new scheduler. However, this may permit
468 further optimizations in later releases.
470 The new queue is enabled by specifying the data type of the message
471 priorities while building charm using --with-prio-type=dtype. Here, dtype can
472 be one of char, short, int, long, float, double and bitvec. Specifying bitvec
473 will permit arbitrary-length bitvector priorities, and is the current default
474 mode of operation. However, we may change this in a future release.
476 - Converse now provides a complete set of wrappers for
477 fopen/fread/fwrite/fclose to handle EINTR, which is not uncommon on the
478 increasingly-popular Lustre. They are named CmiF{open,read,write,close}, and
479 are available from C and C++ code.
481 - The utility class 'CkEntryOptions' now permits method chaining for cleaner
482 usage. This applies to all its set methods (setPriority, setQueueing,
483 setGroupDepID). Example usage can be found in examples/charm++/prio/pgm.C.
485 - When creating groups or chare arrays that depend on the previous construction
486 of another such entity on the local PE, it is now possible to declare that
487 dependence to the runtime. Creation messages whose dependence is not yet
488 satisfied will be buffered until it is.
490 - For any given chare class Foo and entry method Bar, the supporting class's
491 member CkIndex_Foo::Bar() is used to lookup/specify the entry method
492 index. This release adds a newer API for such members where the argument is a
493 function pointer of the same signature as the entry method. Those new
494 functions are used like CkIndex_Foo::idx_Bar(&Foo::Bar). This permits entry
495 point index lookup without instantiating temporary variables just to feed the
496 CkIndex_Foo::Bar() methods. In cases where Foo::Bar is overloaded, &Foo::Bar
497 must be cast to the desired type to disambiguate it.
499 - CkReduction::reducerType now have PUP methods defined; and can hence be
500 passed as parameter-marshalled arguments to entry methods.
502 - The runtime option +stacksize for controlling the allocation of user-level
503 threads' stacks now accepts shorthanded annotation such as 1M.
505 - The -optimize flag to the charmc compiler wrapper now passes more aggressive
506 options to the various underlying compilers than the previous '-O'.
508 - The charmc compiler wrapper now provides a flag -use-new-std to enable
509 support for C11 and C++11 where available. To use this in application code,
510 the runtime system must have been built with that flag as well.
512 - When using, CmiMemoryUsage(), the runtime can be instructed not to use the
513 underlying mallinfo() library call, which can be inaccurate in settings where
514 usage exceeds INT_MAX. This is accomplished by setting the environment
515 variable "MEMORYUSAGE_NO_MALLINFO".
517 - Experimental Features
518 * Initial implementation of a fast message-logging protocol. Use option
519 'mlogft' to build it.
520 * Message compression support for persistent message on Gemini machine layer.
521 * Node-level inter-PE loop/task parallelization is now supported through
522 CkLoop
523 * New temperature/CPU frequency aware load balancer
524 * Support interoperation of Charm++ and native MPI code through dynamically
525 switching control between the two
526 * API in centralized load balancers to get and set PE speed
527 * A new scheme for optimization of double in-memory checkpoint/restart.
528 * Message combining library for improved fine-grained communication
529 performance
530 * Support for partitioning of allocated nodes into subsets that run
531 independent Charm++ instances but can interact with each other.
533 Platform-Specific Changes
534 -------------------------
536 - Cray XE/XK
537 * The gemini_gni network layer has been heavily tuned and optimized,
538 providing substantial improvements in performance, scalability, and
539 stability.
540 * The gemini_gni-crayxe machine layer supports a 'hugepages' option at build
541 time, rather than requiring manual configuration file editing.
542 * Persistent message optimizations can be used to reduce latency and
543 overheads
544 * Experimental support for 'urgent' sends, which are sent ahead of any other
545 outgoing messages queued for transmission.
547 - IBM Blue Gene Q: Experimental machine-layer support for the native PAMI
548 interface and MPI, with and without SMP support. This supports many new
549 systems, including LLNL's Sequoia, ALCF's Mira, and FZ Juelich's Juqueen.
551 There are three network-layer implementations for these systems: 'mpi',
552 'pami', and 'pamilrts'. The 'mpi' layer is stable, but its performance and
553 scalability suffers from the additional overhead of using MPI rather than
554 driving the interconnect directly. The 'pami' layer is well tested for NAMD,
555 but has shown instability for other applications. It is likely to be replaced
556 by the 'pamilrts' layer, which is more generally stable and seems to provide
557 the same performance, in the next release.
559 In addition to the common 'smp' option to build the runtime system with
560 shared memory support, there is an 'async' option which sometimes provides
561 better performance on SMP builds. This option passes tests on 'pamilrts', but
562 is still experimental.
564 Note: Applications that have large number of messages may crash in default
565 setup due to overflow in the low-level FIFOs. Environment variables
566 MUSPI_INJFIFOSIZE and PAMI_RGETINJFIFOSIZE can be set to avoid application
567 failures due to large number of small and large messages respectively. The
568 default value of these variable is 65536 which is sufficient for 1000
569 messages in flight.
571 - Infiniband Verbs: Better support for more flavors of ibverbs libraries
573 - MPI Network Layer
574 * Experimental rendezvous protocol for better performance above some MPI
575 implementations.
576 * Some tuning parameters ("+dynCapSend" and "+dynCapRecv") are now
577 configurable at job launch, rather than Charm++ compilation.
579 - PGI C++: Disable automatic 'using namespace std;'
581 - Charm++ now supports ARM, both non-smp and smp.
583 - Mac OS X: Compilation options to build and link correctly on newer versions
586 ================================================================================
587 What's new in Charm++ 6.4.0
588 ================================================================================
590 --------------------------------------------------------------------------------
591 Platform Support
592 --------------------------------------------------------------------------------
594 - Cray XE and XK systems using the Gemini network via either MPI
595 (mpi-crayxe) or the native uGNI (gemini_gni-crayxe)
597 - IBM Blue Gene Q, using MPI (mpi-bluegeneq) or PAMI (pami-bluegeneq)
599 - Clang, Cray, and Fujitsu compilers
601 - MPI-based machine layers can now run on >64k PEs
603 --------------------------------------------------------------------------------
604 General Changes
605 --------------------------------------------------------------------------------
607 - Added a new [reductiontarget] attribute to enable
608 parameter-marshaled recipients of reduction messages
610 - Enabled pipelining of large messages in CkMulticast by default
612 - New load balancers added:
613 * TreeMatch
614 * Zoltan
615 * Scotch graph partitioning based: ScotchLB and Refine and Topo variants
616 * RefineSwap
618 - Load balancing improvements:
620 * Allow reduced load database size using floats instead of doubles
621 * Improved hierarchical balancer
622 * Periodic balancing adapts its interval dynamically
623 * User code can request a callback when migration is complete
624 * More balancers properly consider object migratability and PE
625 availability and speed
626 * Instrumentation records multicasts
628 - Chare arrays support options that can enable some optimizations
630 - New 'completion detection' library for parallel process termination
631 detection, when the need for modularity excludes full quiescence
632 detection
634 - New 'mesh streamer' library for fine-grain many-to-many collectives,
635 handling message bundling and network topology
637 - Memory pooling allocator performance and resource usage improved
638 substantially
640 - AMPI: More routines support MPI_IN_PLACE, and those that don't check
641 for it
643 ================================================================================
644 What's new in Charm++ 6.2.1 (since 6.2.0)
645 ================================================================================
647 --------------------------------------------------------------------------------
648 New Supported Platforms:
649 --------------------------------------------------------------------------------
651 POWER7 with LAPI on Linux
653 Infiniband on PowerPC
655 --------------------------------------------------------------------------------
656 General Changes
657 --------------------------------------------------------------------------------
659 - Better support for multicasts on groups
660 - Topology information gathering has been optimized
661 - Converse (seed) load balancers have many new optimizations applied
662 - CPU affinity can be set more easily using +pemap and +commap options
663 instead of the older +coremap
664 - HybridLB (hierarchical balancing for very large core-count systems)
665 has been substantially improved
666 - Load balancing infrastructure has further optimizations and bug fixes
667 - Object mappings can be read from a file, to allow offline
668 topology-aware placement
669 - Projections logs can be spread across multiple directories, speeding
670 up output when dealing with thousands of cores (+trace-subdirs N
671 will divide log files evenly among N subdirectories of the trace
672 root, named PROGNAME.projdir.K)
673 - AMPI now implements MPI_Issend
674 - AMPI's MPI_Alltoall uses a flooding algorithm more agressively,
675 versus pairwise exchange
676 - Virtualized ARMCI support has been extended to cover the functions
677 needed by CAF
679 --------------------------------------------------------------------------------
680 Architecture-specific changes
681 --------------------------------------------------------------------------------
683 - LAPI SMP has many new optimizations applied
685 - Net builds support the use of clusters' mpiexec systems for job
686 launch, via the ++mpiexec option to charmrun
688 ================================================================================
689 What's new in Charm++ 6.2.0 (since 6.1)
690 ================================================================================
692 --------------------------------------------------------------------------------
693 New Supported Platforms:
694 --------------------------------------------------------------------------------
696 64-bit MIPS, such as SiCortex, using mpi-linux-mips64
698 Windows HPC cluster, using mpi-win32/mpi-win64
700 Mac OSX 10.6, Snow Leopard (32-bit and 64-bit).
702 --------------------------------------------------------------------------------
703 General Changes
704 --------------------------------------------------------------------------------
706 Runtime support
707 - Smarter build/configure scripts
708 - A new interface for model-based load balancing
709 - new CPU topology API
710 - a general implementation of CmiMemoryUsage()
711 - Bug fix: Quiescence detection (QD) works with immediate messages
712 - New reduction functions implemented in Converse
713 - CCS (Converse Client-Server) can deliver message to more than one processor
714 - Added a memory-aware adaptive scheduler, which can be optionally
715 compiled in to charm
716 - Added preliminary support for automatic message prioritization
717 (disabled by default)
719 Charm++
720 - Cross-array and cross-group sections
721 - Structured Dagger (SDAG): Support templated arguments properly
722 - Plain chares support checkpoint/restart (both in-memory and disk-based)
723 - Conditional packing of messages and parameters in SMP scenario
724 - Changes to the CkArrayIndex class hierarchy
725 -- sizeof() all CkArrayIndex* classes is now the same
726 -- Codes using custom array indices have to use placement-new to construct
727 their custom index. Refer example code: examples/charm++/hello/fancyarray/
728 -- *** Backward Incompatibility ***
729 CkArrayIndex[4D/5D/6D]::index are now of type int (instead of short)
730 However the data is stored as shorts. Access by casting
731 CkArrayIndexND::data() appropriately
732 -- *** Deprecated ***
733 The direct use of public data member
734 CkArrayIndexND::index (N=1..6) is deprecated. We reserve the right to
735 change/remove this variable in future releases of Charm++.
736 Instead, please access the indices via member function:
737 int CkArrayIndexND::data()
739 Adaptive MPI (AMPI)
740 - Compilers renamed to avoid collision with host MPI (ampicc, ampiCC,
741 ampif77, ampif90)
742 - Improved MPI standard conformance, and documentation of non-conformance
743 * Bug fixes in: MPI_Ssend, MPI_Cart_shift, MPI_Get_count
744 * Support MPI_IN_PLACE in MPI_(All)Reduce
745 * Define various missing constants
746 - Return the received message's tag in response to a non-blocking
747 wildcard receive, to support SuperLU
748 - Improved tracing for BigSim
750 Multiphase Shared Arrays (MSA)
751 - Typed handles to enforce phases
752 - Split-phase synchronization to enable message-driven execution
753 - 3D arrays
755 TCharm
756 - Automatic tracing of API calls for simulation and analysis
758 Debugging
759 - Wider support for architectures other than net- (in particular MPI layers)
760 - Improved support for large scale debugging (better scalability)
761 - Enhanced record/replay stability to handle various events, and to
762 signal unexpected messages
763 - New detailed record/replay: The full content of messages can be
764 recorded, and a single processor can be re-executed outside of the
765 parallel application
767 Performance analysis
768 - Tracing of nested entry methods
770 Automatic Performance Tuning
771 - Created an automatic tuning framework [still for experimental use only]
773 CkMulticast
774 - Network-topology / node aware spanning trees used internally for and
775 lower bytes on the network and improved performance in multicasts and
776 reductions delegated to this library
778 Comlib
779 - Improved OneTimeMulticastStrategy classes
781 BigSim
782 - Out-of-core support, with prefetching capability
783 - Detailed tracing of MPI calls
784 - Detailed record/replay support at emulation time, capable of
785 replaying any emulated processor after obtained recorded logs.
787 --------------------------------------------------------------------------------
788 Architecture-specific changes
789 --------------------------------------------------------------------------------
791 Net-*
792 - Can run jobs with more than 1024 PEs
794 Net-Linux
795 - New charmrun option ++no-va-randomization to disable address space
796 randomization (ASLR). This is most useful for running AMPI with
797 isomalloc
799 MPI
800 - Default to using ampicxx instead of mpiCC
802 MPI-SMP
803 - The +p option now has the same semantics as in other smp builds
805 Power 7
806 - Support for VSX in SIMD abstraction API
808 Blue Gene/L
809 - Compilers and options have been updated to the latest ones
811 Blue Gene/P
812 - Added routines for measuring performance counters on BG/P.
813 - Updated to support latest DCMF driver version. On ANL's Intrepid, you may
814 need to set BGP_INSTALL=/bgsys/drivers/V1R4M1_460_2009-091110P/ppc in your
815 environment. This is the default on ANL's Surveyor.
817 Cray XT
818 - cputopology information is now available on XT3/4/5
820 Infiniband (ibverbs)
821 - Bug fix: plug memory leaks that caused failures in long runs
822 - Optimized to reduce startup delays
824 LAPI
825 - Support for SMP (experimental)
828 ================================================================================
829 Note that changes from 5.9, 6.0, and 6.1 are not documented here. A partial list
830 can be found on the charm download page, or by reading through version control
831 logs.
833 ================================================================================
834 What's New since Charm++ 5.4 release 1
835 ================================================================================
837 --------------------------------------------------------------------------------
838 New Supported Platforms:
839 --------------------------------------------------------------------------------
840 1. Charm++ ported to IA64 Itanium running Win2K and Linux, Charm++ also support
841 Intel C/C++ compilers;
843 2. Charm++ ported to Power Macintosh powerpc running Darwin;
845 3. Charm++ ported to Myrinet networking with GM API;
847 --------------------------------------------------------------------------------
848 Summary of New Features:
849 --------------------------------------------------------------------------------
850 1. Structure Dagger
851 Structured Dagger is a coordination language built on top of CHARM++.
852 Structured Dagger allows easy expression of dependences among messages and
853 computations and also among computations within the same object using
854 when-blocks and various structured constructs.
856 2. Entry functions support parameter marshalling
857 Now you can declare and invoke remote entry functions using parameter
858 marshalling instead of defining messages.
860 3. Easier running - standalone mode
861 For net-* version running locally, you can now run Charm programs without
862 charmrun. Running a node program directly from command line is now the
863 same as "charmrun +p1 <program>"; for SMP version, you can also specify
864 multiple (local) processors, as in "program +p2".
867 --------------------------------------------------------------------------------
868 Summary of Changes:
869 --------------------------------------------------------------------------------
870 1. "build" changed for compilation of Charm++
871 To build Charm++ from scratch, we now take additional command line options
872 to compile with addon features and using different compilers other than gcc.
873 For example, to build Linux IA64 with Myrinet support, type command:
874 ./build net-linux-ia64 gm
877 ******* Old Change histories *******
880 ================================================================================
881 What's New in Charm++ 5.4 release 1 since 5.0
882 ================================================================================
884 --------------------------------------------------------------------------------
885 New Supported Platforms:
886 --------------------------------------------------------------------------------
888 1. Win9x/2000/NT: with Visual C++ or Cygwin gcc/g++, you can compile and run
889 Charm++ programs on all Win32 platforms.
891 2. Scyld Beowulf: Charm++ has been ported to the Linux-based Scyld Beowulf
892 operating system. For more information on Scyld, see <http://www.scyld.com>
894 3. MPI with VMI: Charm++ has been ported to NCSA's Virtual Machine Interface,
895 which is an efficient messaging library for heterogeneous cluster
896 communication.
899 --------------------------------------------------------------------------------
900 Summary of New Features:
901 --------------------------------------------------------------------------------
902 1. Dynamic Load balancing:
903 Chare migration is supported in the new release. Migration-based dynamic
904 load balancing framework with various load balancing strategies library has
905 been added.
907 2. Chare Array
908 Charm++ array is supported. You can now create an array of Chare objects
909 and use array index to refer the Charm++ array elements. A reduction
910 library on top of Chare array has been implemented and included.
912 3. Projections
913 Projections, a Java application for Charm++ program performance analysis and
914 visualization, has been included and distributed in the new release. Two
915 trace modes are available: trace-projections and trace-summary. Trace-summary
916 is a light-weight trace library compared to trace-projections.
918 4. AMPI
919 AMPI is a load-balancing based library for porting legacy MPI applications
920 to Charm++. With few changes in the original MPI code to AMPI, the new
921 legacy MPI application on Charm++ will gain from Charm++'s adptive
922 load balancing ability.
924 5. Easier invocation
925 "Charmrun" is now available on all platforms, with a uniform command line
926 syntax. You can forget the difference between net-* versions and MPI versions,
927 and run charm++ application with this same charmrun command syntax.
928 ++local option is added in charmrun for net-* version, it provides
929 simple local use of Charm and no longer require the ability to
930 "rsh localhost" or a nodelist file in order to run charm only on the local
931 machine. This is especially attractive when you run Charm++ on Windows.
933 6. New libraries:
934 Many new libraries have been added in this release. They include:
935 1) master-slave library: for writing manager-worker paradigm programs.
936 2) receiver library: provide asynchronous communication mode for chare array.
937 3) f90charm: provides Fortran90 bindings for Charm++ Array.
938 4) BlueGene: a Charm++/Converse emulator for IBM proposed Blue Gene.
940 --------------------------------------------------------------------------------
941 Summary of Changes:
942 --------------------------------------------------------------------------------
943 1. message declaration syntax in .ci file:
944 The message declaration syntax for packed/varsize messages has been changed.
945 The packed/varsize keywords are eliminated, and you can specify the actual
946 actual varsize arrays in the interface file and have the translator generate
947 alloc, pack and unpack.
950 Here is the detailed list of Changes:
952 --------------------------------------------------------------------------------
953 Major Features:
954 --------------------------------------------------------------------------------
956 10/06/1999 rbrunner Added migration-based dynamic load balancing
957 framework.
958 11/15/1999 olawlor Added reduction support foe Charm++ arrays
959 02/06/2000 milind Added AMPI, an implementation of MPI with
960 dynamic load balancing
961 02/18/2000 paranjpy New platforms supported: net-win32, and net-win32-smp
962 04/04/2000 olawlor Added arbitrarily indexed Charm++ arrays.
963 Also, added translator support for new arrays.
964 04/15/2000 olawlor Added "puppers" for packing and unpacking
965 objects.
966 06/14/2000 milind Added the threaded FEM framework.
968 --------------------------------------------------------------------------------
969 Minor Features:
970 --------------------------------------------------------------------------------
972 10/09/1999 rbrunner Added packlib, a library for C and C++ to
973 pack-unpack data to/from Charm++ messages.
974 10/13/1999 gzheng New LB strategy: RefineLB
975 10/13/1999 paranjpy New LB Strategy: Heap
976 10/14/1999 milind New LB Strategy: Metis
977 10/19/1999 olawlor New test program for testing LB strategies.
978 10/21/1999 gzheng New trace mode: trace-summary
979 10/28/1999 milind New supported platform: net-sol-x86
980 10/29/1999 milind Added runtime checks for ChareID assignment.
981 11/10/1999 rbrunner Added Neighborhood base strategy for LB
982 framework.
983 11/15/1999 olawlor conv-host now reads in a startup file
984 ~/.conv-hostrc
985 11/15/1999 olawlor New test program for testing array reductions.
986 11/16/1999 rbrunner Added processor-speed checking functions to
987 LB framework
988 11/19/1999 milind Mapped SIGUSR to a Ccd condtion handler
989 11/22/1999 rbrunner New LB strategy: WSLB
990 11/29/1999 ruiliu Modified Metis LB strategy to deal with
991 different processor speeds
992 12/16/1999 rbrunner New LB strategy: GreedyRef
993 12/16/1999 rbrunner New LB strategy: RandRef
994 12/21/1999 skumar2 New LB strategy: CommLB
995 01/03/2000 rbrunner New LB strategy: RecBisectBfLB
996 01/08/2000 skumar2 New LB strategy: Comm1LB, with varying processor
997 speeds
998 01/18/2000 milind Modified SM library syntax, and added a test
999 program for SM.
1000 01/19/2000 gzheng Added irecv, a library to simplify conversion
1001 of message-passing programs to Charm++
1002 02/20/2000 olawlor Added preliminary broadcast support to Charm++
1003 arrays.
1004 02/23/2000 paranjpy Added converse-level quiescence detection
1005 03/02/2000 milind Added ++server-port option to pre-specify
1006 CCS port.
1007 03/10/2000 wilmarth Random seed-based load balancer now uses
1008 bit-vector for active PEs.
1009 03/21/2000 gzheng Added support for marking user-defined events
1010 in trace-summary.
1011 03/28/2000 wilmarth Added CMK_TRUECRASH. Very helpful for
1012 post-mortem debugging of Charm++ programs on
1013 net-* versions.
1014 03/31/2000 jdesouza Added Fortran90 support to the Charm++
1015 interface translator.
1016 03/09/2000 milind Added support for -LANG and -rpath options
1017 in charmc for Origin2000.
1018 04/28/2000 milind Added prioritized converse threads.
1019 05/01/2000 milind Added test programs for TeMPO, AMPI and irecv.
1020 05/04/2000 milind New supported platform: mpi-sp.
1021 05/04/2000 gzheng Added irecv pingpong program.
1022 05/17/2000 olawlor Each chare, group and array element now has to
1023 have migration constructor.
1024 05/24/2000 milind Added Jacobi3D programs for irecv and AMPI both.
1025 05/24/2000 milind Made migratable an optional attribute of
1026 chares, groups, and nodegroups.
1027 Arrays are by default migratable.
1028 05/29/2000 paranjpy Added pup methods to arrays, reductions etc
1029 internal objects.
1030 06/13/2000 milind Made CtvInitialize idempotent. That is, it
1031 can be called by any number of threads now,
1032 only the first one will actually do
1033 CtvInitialize.
1034 06/20/2000 milind Added a simple test program for the FEM
1035 framework.
1036 07/06/2000 milind Imported Metis 4.0 sources in the CVS tree.
1037 Also added code to make metis libraries and
1038 executables to Makefile.
1039 07/07/2000 milind Added more meaningfull error messages using
1040 perror in addition to a cryptic error codes in
1041 net-* versions.
1042 07/10/2000 milind fem and femf are now recognized as "languages"
1043 by charmc.
1044 07/10/2000 saboo Added the derived datatypes library.
1045 07/13/2000 milind Added +idle_timeout functionality. It takes a
1046 commandline parameter denoting milliseconds of
1047 maximum consecutive idle time allowed per
1048 processor.
1049 07/14/2000 milind Added group multicast. Added
1050 CkSendMsgBranchMulti, CldEnqueueMulti, and
1051 translator changes to support it.
1052 07/14/2000 milind SUPER_INSTALL now takes "-*" arguments prior
1053 to the target, that will be passed to make as
1054 "makeflags". This makes it easy to suppress
1055 make's output of commands etc (with the -s
1056 flag). As a result of this, several Makefiles
1057 have been massaged.
1058 07/18/2000 milind Added support for using "dbx" on suns as
1059 debugger.
1060 07/19/2000 milind Added ability to tracemode projections which
1061 produces binary trace files. Use flag
1062 +binary-trace on the command line.
1063 07/26/2000 milind Separated AMPI from TeMPO.
1064 07/28/2000 milind Added test programs to test reduce, alltoall
1065 and allreduce functionality of AMPI.
1066 08/02/2000 milind Added an option to let the user specify which
1067 "xterm" to use. For example, on some systems
1068 (CDE), only dtterm is installed. So, by
1069 putting ++xterm dtterm on the conv-host
1070 commandline, one can use dtterm when ++in-xterm
1071 option is specified on conv-host commandline.
1072 08/14/2000 milind FEM Framework: Added capabilities to handle
1073 esoteric meshes to standalone offline programs.
1074 Makefile now produces gmap and fgmap programs,
1075 which are used for this purpose. They convert
1076 the mesh to a graph before partitioning it
1077 using Metis.
1078 08/24/2000 milind Added the 2D crack propagation program as a
1079 test program for FEM framework.
1080 08/25/2000 milind Initial implementation of isomalloc-based
1081 threads. This implementation uses a fixed
1082 stack size for all threads (can be set at
1083 runtime.)
1084 08/26/2000 milind Added a macro CtvAccessOther that lets you
1085 get/set a Ctv variable of any thread. It
1086 should be invoked as CtvAccessOther(thread,
1087 varname); Added CthGetData function to each of
1088 the threads implementation. This function is
1089 used in the CtvAccessOther macro.
1090 08/27/2000 milind FEM Framework: Separated mesh to graph
1091 conversion capability into a separate program.
1092 This way, the generated graph can be partitioned
1093 repeatedly.
1094 09/04/2000 milind Added the class static readonly variables to
1095 ci file syntax.
1096 09/05/2000 milind FEM Framework: A very fast O(n) algorithm for
1097 mesh2graph , uses more memory, but the tradeoff
1098 was worth it. Coded by Karthik Mahesh, minor
1099 optimizations by Milind.
1100 09/05/2000 milind Added a barebones charm kernel scheduling
1101 overhead measurement program.
1102 09/15/2000 milind Added pup support for AMPI and FEM framework.
1103 09/20/2000 olawlor Added capability to have an array of base type
1104 where individual element could be of derived
1105 types.
1106 10/03/2000 gzheng New supported platform: net-linux-axp
1107 10/05/2000 skumar2 Added program littleMD to the test suite.
1108 10/07/2000 skumar2 New job scheduler (Faucets projects).
1109 10/15/2000 milind Improved support for Fortran90 in charmc.
1110 11/04/2000 jdesouza Made the Faucets scheduler multi-threaded.
1111 11/05/2000 olawlor FEM Framework: supports multiple element types,
1112 mesh re-assembly, etc.
1113 11/15/2000 gzheng New platform support: net-cygwin
1114 11/18/2000 gzheng conv-host no longer needs /bin/csh to start
1115 remote program. set
1116 CMK_CONV_HOST_CSH_UNAVAILABLE to 1 to use
1117 /bin/sh instead.
1118 11/25/2000 milind Finished experimental implementation of
1119 converse-threads based on co-operative pthreads.
1120 11/25/2000 milind Added a benchmark suite of all pingpongs in
1121 Charm++.
1122 11/28/2000 milind Removed deletion of _idx at the end of every
1123 send or doneInserting call. Instead now it is
1124 in the destructor of the proxy. This allows us
1125 to cache proxies, when proxy creation becomes
1126 a bottleneck.
1127 11/28/2000 olawlor Added "seek blocks" to puppers. This should
1128 allow out-of-order pup'ing without the ugliness
1129 of getBuf; and in a way that works with all
1130 PUP::ers.
1131 11/29/2000 olawlor Simplified and regularized command-line-argument
1132 handling.
1133 11/29/2000 milind AMPI: Added multiple-communicators capability.
1134 12/05/2000 gzheng Now /bin/sh is default shell to fork node
1135 program on remote machines.
1136 12/13/2000 olawlor Added charmrun wrapper for poe on mpi-sp.
1137 12/14/2000 milind Added bluegene emulator sources and test
1138 programs. Added "bluegene" as a language known
1139 to charmc. Makefile now has a target called
1140 bluegene. Added preliminary bluegene
1141 documentation. (copied from Arun's webpage.)
1142 12/15/2000 gzheng f90charm addition to Makefile and charmc. Also,
1143 added fixed size arrays support to f90charm. A
1144 test program f90charm/hello is checked in.
1145 12/17/2000 milind Added rtest test program. Contributed by jim to
1146 test Converse message transmission.
1147 12/20/2000 olawlor Added charmconfig script. Enables automatic
1148 determination of C++ compiler properties,
1149 replacing the verbose and error-prone
1150 conv-mach.h entries for CMK_BOOL,
1151 CMK_STL_USE_DOT_H, CMK_CPP_CAST_OK, ...
1152 12/20/2000 olawlor Charm++ Arrays optimizations: Key and object
1153 now variable-length fields, instead of pointers.
1154 This extra flexibility lets us save many
1155 dynamic allocations in the array framework.
1156 12/20/2000 olawlor Added PUP::able support-- dynamic type
1157 identification, allocation, and deletion.
1158 Allows you to write: p(objPtr); and
1159 objPointer will be properly identified,
1160 allocated, packed, and deallocated (depending
1161 on the PUP::er). Requires you to register any
1162 such classes with DECLARE_PUPable and
1163 DEFINE_PUPable.
1164 12/20/2000 olawlor Arrays optimizations: Made CkArrayIndex
1165 fixed-size. This significantly improves
1166 messaging speed (7 us instead of 10 us
1167 roundtrip). Move spring cleaning check into a
1168 CcdCallFnAfter, which gains more speed (down to
1169 4 us roundtrip).
1170 12/20/2000 olawlor More optimizations: Minor speed tweaks--
1171 conv-ccs.c uses hashtable for handler lookup;
1172 conv-conds skips timer test until needed;
1173 convcore.c scheduler loop optmizations (no
1174 superfluous EndIdle calls); threads.c
1175 CMK_OPTIMIZE-> no mprotect.
1176 12/20/2000 olawlor More Optimizations: Minor speed tweaks-- ck.C
1177 groups cldEnqueue skip; init.h defines
1178 CkLocalBranch inline; and supporting changes.
1179 12/22/2000 gzheng IA64 support for Converse user level threads.
1180 01/02/2001 olawlor CCS: Minor update-- enabled CcsProbe, cleaned
1181 up superflous debug messages in server, added
1182 Java interface (originally written for
1183 AppSpecter).
1184 01/09/2001 gzheng charmconfig converted to autoconf style, need
1185 to change configure.in and conv-autoconfig.h.in,
1186 and run autoconf to get configure and copy to
1187 charmconfig. added fortran subroutine name
1188 test and get libpthread.a
1189 01/10/2001 milind Added telnet method of getting libpthread.a
1190 from charm webserver.
1191 01/11/2001 olawlor Moved projections files here from
1192 CVSROOT/projections-java. Added fast Java
1193 versions of the .log file input routines in
1194 LogReader, LogLoader, LogAnalyzer, and
1195 UsageCalc. Added "U.java" user interface
1196 utility file, allowing times to be input in
1197 seconds, milliseconds, or microseconds,
1198 instead of just microseconds.
1199 01/15/2001 gzheng add +trace-root to specify the directory to
1200 put log files in. this is need in Scyld cluster
1201 where there is no NFS mounting and no i/o
1202 access to home directory sharing on nodes.
1203 01/15/2001 milind Made AMPI into a f90 module instead of
1204 'ampif.h' inclusion. AMPI f90 bindings are
1205 now more inclusive. Fixed argc,argv handling
1206 bugs in ArgsInfo message. Fixed a bug in pup
1207 that caused thread not to be sized, but was
1208 packed nevertheless. Moved irecv to waitall
1209 instead of at in ampi_start. Made
1210 AMPI_COMM_WORLD to be 0, because it clashed
1211 with wildcard(-1). AMPI_COMM_UNIVERSE is now
1212 handled properly in the AMPI module.
1213 C/C++ data members are NOT visible to
1214 Fortran 90.
1215 01/18/2001 gzheng New supported platform: net-linux-scyld
1216 01/20/2001 olawlor Moved array index field from CMessage_* to the
1217 Ck envelope itself. This is the right thing
1218 to do, because any message may be sent to/from
1219 an array element. To reduce the wasted space
1220 in a message, a union is used to overlay the
1221 fields for the various possible message types.
1222 01/29/2001 olawlor Freed charmrun on net-* version from using
1223 remote shell to fork off processes. One can now
1224 use a daemon provided in the distribution.
1225 02/07/2001 olawlor Added debugging support to puppers.
1226 02/13/2000 gzheng Added ++local option to charmrun to start node
1227 program locally without any daemon; fix the
1228 hang program if you type wrong pgm name in
1229 scyld version, and redirect all output to
1230 /dev/null, otherwise all node program can send
1231 its output to console in scyld. Also implemented ++local in net-win32 version.
1232 02/26/2000 milind Changed the varsize syntax. Now one can specify
1233 actual varsize arrays in the interface file
1234 and have the translator generate alloc, pack
1235 and unpack.
1237 --------------------------------------------------------------------------------
1238 Bug Fixes:
1239 --------------------------------------------------------------------------------
1241 10/29/1999 milind Replaced jmemcpy by memcpy in net versions, as
1242 it was causing a bit to flip (bug reported
1243 by jim.)
1244 10/29/1999 milind Fixed multiline macros in all header files.
1245 02/05/2000 milind Fixed linking errors by getting the order of
1246 libraries right from the charmc command-line.
1247 02/18/2000 paranjpy Fixed Charm++ initialization bug on SMPs.
1248 02/21/2000 milind Fixed a context-switching bug in mipspro version
1249 of QuickThreads.
1250 02/25/2000 milind Charm++ interface translator was segfaulting
1251 on interface file errors. Fixed that. Also,
1252 added linenumbers to error messages.
1253 03/02/2000 milind Made CCS work on SMPs.
1254 03/07/2000 milind Made ConverseInit consistent with the manual on
1255 Origin2000 version.
1256 04/18/2000 milind Fixed a bug in CkWaitFuture, which was caching
1257 a variable locally, while it was changed by
1258 another thread.
1259 05/04/2000 paranjpy Fixed argv deletion bug on net-win32-smp.
1260 06/08/2000 milind sp3 version: changed optimization flags, which
1261 where power2 processor-specific.
1262 06/20/2000 milind mpi-* versions: Fixed ConverseExit since it was
1263 not obeying the following statement in the MPI
1264 standard: The user must ensure that all pending
1265 communications involving a process completes
1266 before the process calls MPI_FINALIZE.
1267 07/05/2000 milind Fixed a nasty bug in charmc in the -cp option.
1268 It used to append the name provided to -o flag
1269 to the directory provided to the -cp flag.
1270 Thus, -o ../pgm -cp ../bin options meant that
1271 the pgm would be copied to ../bin/.., which is
1272 not the expected behavior. This fix correctly
1273 copies pgm to ../bin.
1274 07/07/2000 milind Removed variable arg_myhome, as it was not
1275 being used anywhere, and also, setting it was
1276 causing problems of env var HOME was not set.
1277 07/27/2000 milind thishandle for the arrayelement was not being
1278 correctly set. Bug was reported by Neelam.
1279 08/26/2000 milind Origin2000: Changed the page alignment to
1280 reflect the mmap alignment. The mmap man page
1281 specifically states that it is not the same as
1282 page size.
1283 09/02/2000 milind Fixed a bug in code generated for threaded
1284 (void) entry methods of array elements. The
1285 dummy message that is passed to that method in
1286 a thread has to be deleted before calling the
1287 object method, because upon object method's
1288 return, the thread might have migrated.
1289 09/03/2000 olawlor Minor fix-fixes: 1.) Change to LBObjid hash
1290 function would fail for >4-int object indices.
1291 Replaced with proper function, which also
1292 preserves the 1-int case. 2.) Array element
1293 sends must go via the message queue to prevent
1294 stack build-up for deep single-processor call
1295 chains. These might happen, e.g., in a driver
1296 element calling itself for the main time loop.
1297 Messages are now properly noted as sent, then
1298 wait through the queue for delivery. This
1299 entailed minor reorganization of the message
1300 delivery subsystem.
1301 09/21/2000 olawlor Tiny SMP thread fix-- registrations of a
1302 thread-private variable now reserve space on
1303 calls after the first. This wastes space for
1304 multiple CthInitialize's-- it's a quick hack to
1305 get threads working again on SMP versions.
1306 10/16/2000 olawlor A few CCS fixes: -Added split-phase reply
1307 (delay reply indefinitely) -Cleaned up error
1308 handling -Pass user data as "void *" instead of
1309 "char *"
1310 11/03/2000 wilmarth Removed 0 size array allocation in Charm++
1311 quiescence detection.
1312 11/20/2000 gzheng Rewrote part of Fiber thread, including a bug
1313 fix for a the non thread-safe function, and a
1314 different fiber free strategy.
1315 11/29/2000 gzheng The LB init procedure tried to allocate
1316 65536*160 as initial size, which is 10M memory
1317 for communication table, which is too big.
1318 Cut it down to roughly 1M, and it can expand
1319 in later code.
1320 12/05/2000 gzheng In many cases, conv-host exits without print
1321 out the error message from remote shell. try
1322 to fix it by calling sync to flush the pipe
1323 before exit 1.
1324 12/10/2000 milind net-linux: Made static linking the default
1325 option because dynamic linking runtime causes
1326 isomalloc threads to crash.
1327 12/18/2000 milind Increased portability of isomalloc threads by
1328 removing dependence on alloca.
1329 12/28/2000 milind Fixed ctrl-getone abort bug on SMP.
1330 12/28/2000 milind Made _groupTable a pointer on which a
1331 constructor is explicitly called. Since it
1332 was a Cpv variable, its constructor was not
1333 called by default in case of an SMP version.
1334 12/29/2000 olawlor Prevent infinite copy constructor recursion on
1335 Origin2000.
1336 01/10/2001 olawlor Added "explicit" keyword to remove ambiguity
1337 for KCC, which was confused by the private
1338 PUP::er(int) "cast" constructor and the operator
1339 |(PUP::er &p,T &t) into rejecting all operator|
1340 (int,int) as ambiguous.
1341 2001/01/17 gzheng fix the charmconfig bug on paragon-red: the
1342 failure testing of fortran won't stop the
1343 compilation.
1344 01/20/2001 olawlor Arrays reduction: Fixed bug-- reduction may end
1345 because all contributors migrate away.
1346 01/29/2001 olawlor Fix heap-corrupting bug-- call ->init() on
1347 nodeGroupTable, which sets the "pending"
1348 message queue to NULL. This prevents a nasty
1349 delete-unitialized-data bug later on. Also
1350 delayed queue creation until messages actually
1351 arrive.
1353 --------------------------------------------------------------------------------
1354 Documentation Changes:
1355 --------------------------------------------------------------------------------
1357 01/31/2000 milind Installation manual: Fixed bugs pointed out by
1358 quantum TA
1359 02/28/2000 wilmarth Added a new look Charm++ manual.
1360 06/20/2000 milind Added pdflatex support to generate PDF versions
1361 of manuals from LaTeX sources.
1362 12/05/2000 milind Added Orion's FEM manual. Converted from HTML.
1363 12/10/2000 milind Added pplmanual.sty for all manuals.
1364 12/17/2000 milind Added master-slave library documentation to
1365 convext manual.
1366 12/21/2000 saboo Added DDT documentation.
1367 01/02/2001 olawlor Updated for new CCS version.
1369 --------------------------------------------------------------------------------
1370 Other Changes:
1371 --------------------------------------------------------------------------------
1373 10/24/1999 olawlor charmc is changed to Bourne shell script
1374 instead of csh. All conv-mach.csh are
1375 replaced by conv-mach.sh.
1376 10/25/1999 olawlor SUPER_INSTALL is converted to use bourne shell.
1377 10/28/1999 milind All Makefiles now take OPTS commandline
1378 arguments.
1379 01/16/2000 olawlor Simplified Charm++ interface translator.
1380 02/23/2000 ruiliu Changed rand() calls from all over the codes
1381 to the new Converse random number generator.
1382 02/26/2000 milind Simplified the converse scheduler loop by
1383 combining the maxmsgs and poll modes.
1384 08/31/2000 milind Imported system documentation into the CVS tree.
1385 Also added super_install target for docs with
1386 necessary Makefile modifications.
1387 09/08/2000 olawlor Made soft links use relative pathnames instead
1388 of absolute. This lets you move a charm++
1389 installation without having to recompile
1390 anything.
1391 09/11/2000 olawlor Grouped commonly needed code in the new util
1392 directory. Also, added pup_c a C wrapper for
1393 puppers.
1394 09/11/2000 olawlor Slightly reorganized header structure. Now no
1395 headers should need to be listed twice (once in
1396 ALLHEADERS, again in CKHEADERS). Now headers
1397 are soft-linked instead of copied. This makes
1398 development much easier. Added support for the
1399 new Common/util directory.
1400 09/21/2000 olawlor Major reorganization of net-* codes. Now all
1401 the TCP socket routines are in separate files.
1402 Also combined windoes NT code with unix codes.
1403 09/21/2000 olawlor Major rewrite of CCS-- underlying protocol is
1404 now binary (send/recv binary data everywhere);
1405 conv-host forwards requests to nodes; and
1406 source has been significantly re-arranged.
1407 (especially if NODE_0_IS_CONVHOST).
1408 11/22/2000 milind Removed IDL translator from distribution.
1409 12/01/2000 olawlor Renamed conv-host charmrun; added test for
1410 script conv-host. Also added charmrun for most
1411 other machines.
1412 12/17/2000 milind Moved List related data structures into
1413 cklists.h in util. Removed most of the redundant
1414 list implementations.
1415 12/20/2000 gzheng SUPER_INSTALL: format the output of list of
1416 versions and make the help page fit into one
1417 page of xterm.
1418 12/24/2000 milind Added test-{charm,converse,ampi,fem} targets to
1419 super_install.
1420 12/28/2000 milind net-sol-smp now uses pthreads.
1421 01/29/2001 olawlor Merged windowsNT and unix build procedures by
1422 basing the Windows build on cygwin. Added
1423 scripts to deal with unix and windows
1424 differences.