Docs: Review f90charm

[charm.git] / doc / f90charm / manual.rst

==============================
Fortran90 Bindings for Charm++
==============================

.. contents::
   :depth: 3

Charm++ is a parallel object language based on C++. The f90charm module
is to provide Fortran90 programs a f90 interface to Charm++. Using the
F90Charm interface, users can write Fortran90 programs in a fashion
similar to Charm++, which allows creation of parallel object arrays
(Chare Arrays) and sending messages between them.

To interface Fortran90 to Charm++ and thus obtain a parallel version of
your program you need to do the following things:

#. Write a Charm Interface file (extension .ci)

#. Write your F90 program with f90charmmain() as main program;

#. Write implementations of Chare entry methods in the F90 program;

#. Compile and Link with Charm’s Fortran library

#. Run it!

Overview
========

Here we suppose you already know most concepts in Charm++ and have
done some Charm++ programming.
Unlike in C++, we don’t have classes in Fortran90. Thus, a Chare in
F90Charm is represented as a Fortran type structure. Here is an
example:

.. code-block:: fortran

         ! ## Just replace Hello throughout with your chare's name. ##
         ! ## and add your chare's personal data below where indicated ##
         ! ## Everything else remains the same ##
         MODULE HelloMod

         TYPE Hello
         ! ## your chare's data goes here, the integer below is an example ##
         integer data
         END TYPE

         TYPE HelloPtr
         TYPE (Hello), POINTER :: obj
         integer*8 aid
         END TYPE

         END MODULE

You can think of this module as a Chare declaration. Type [Hello]
defines arbitrary user program data and HelloPtr defines the Chare
pointer which the Fortran program will use later to communicate with the
F90Charm runtime library. The [aid] is the handle of the array returned
by the F90Charm library, and the user shouldn’t change it.

In F90Charm as in Charm++, you need to write a .ci interface file so
that the Charm translator will generate helper functions. The syntax of
.ci files is the same as in Charm++, however, for F90Charm, there are
certain constraints. First, you don’t need to declare the main chare as
in Charm++. Second, F90Charm currently only supports up to 3D Chare
arrays, and you cannot define Chare, Group, and NodeGroup types. Third,
there are no message declarations in .ci files, all the entry functions
must be declared in the parameter marshalling fashion of Charm++.
Essentially, users can declare in .ci files readonly variables and 1-3D
chare arrays with parameter marshalled entry methods.

It is the programmer’s responsibility to write the implementation of
Chare entry methods. The decl and def files generated by Charm++’s
translator define the interface functions programmer need to write.

For each Chare defined in the .ci file, the user must write these
functions for F90Charm’s runtime:

``SUBROUTINE <ChareName>_allocate(objPtr, aid, index)``

You can think of this function as a constructor for each array element
with array index [index]. For a 3D array, for example, you can replace
index in the example by a 3D array index [index1, index2, index3]. In
this function the user must allocate memory for the Chare’s user data
and perform any initialization.

For each Chare entry method you have declared, you should write the
corresponding Fortran90 subroutine for it:

``SUBROUTINE <ChareName>_Entry_<EntryName>(charePtr, myIndex, data1, data2 ... )``

Note that the first argument is the Chare pointer as declared
previously, and the second argument is the array index which will be passed
from the Charm runtime. The rest of the parameters should be the same as
declared in the .ci file. For higher dimensional
arrays, replace ``myIndex`` by ``myIndex1, myIndex2`` for example.

On the caller side, the decl/def files generated by Charm++’s translator
also provide the following functions for Chare creation and remote method
invocation. For each Chare declared in .ci files, these subroutines are
generated for use in Fortran90 code:

``<ChareName>_CkNew(integer n, integer*8 aid)``

This subroutine creates a chare array of size ``n``. For higher
dimensional array creation, specify one integer for each dimension. For
example, to create a 3D array:

``<ChareName>_CkNew(integer dim1, integer dim2, integer dim3, integer*8 aid)``

For each entry method, a corresponding ``Invoke`` method is provided (1D
example shown here):

``<ChareName>_Invoke_<EntryName>(charePtr, myIndex, data1, data2 ... )``

This subroutine will send a message to the array element with index
``myIndex``. Similarly, for arrays with higher dimensions, replace ``myIndex``
by a corresponding number of array indices. Broadcasts are supported as
well:

``<ChareName>_Broadcast_<EntryName>(charePtr, data1, data2 ... )``

There are several others things you need to know.

First, as in Charm++, each .ci file will generate two header files:
.decl.h and .def.h. However, in Fortran90 charm, you are not able to
include these C++ files in Fortran90 code. Thus, currently, it is the user’s
task to write a short C++ file including these two headers files. You
should also provide definitions for readonly variables in this C++ file.
It can be as simple as this:

::

   #include "hello.decl.h"
   int chunkSize;  // define readonly variables here
   #include "hello.def.h"

In the future, this file could be generated automatically by the
translator.

Second, you can still use readonly variables as in Charm++. However,
since there are no global variables as in C++ in Fortran90, you have to
access them explicitly via function call. Here are the two helper
functions that the translator generates:

Take the readonly variable chunkSize as an example:

::

   Set_Chunksize(chunkSize);
   Get_Chunksize(chunkSize);

These two functions can be used in user’s Fortran program to set and get
readonly variables.

Third, for the user’s convenience, several Charm++ runtime library functions
have their Fortran interface defined in the F90Charm library. These
currently include:

::

   CkExit()
   CkMyPe(integer mype)
   CkNumPes(integer pes)
   CkPrintf(...)    // note, the format string must terminated with '$$'

Here is a summary of current constraints to write F90 binding Charm++
programs:

#. Only one- to three-dimensional chare arrays are supported.

#. readonly variables must be basic types, i.e. they have to be integers,
   floats, etc. scalar types or array types of these basic scalar types.

#. Instead of ``program main``, your f90 main program starts from
   ``subroutine f90charmmain``.

These details are best illustrated with an example: a hello world program.
When executed, an array of several parallel chares is created, forming a ring.
Each chare prints a string when it receives a message, and then sends a
message to the next chare in the ring. The Fortran ``f90charmmain`` subroutine
begins execution, and the ``SayHi`` subroutine performs each chare's task.

Writing the Charm++ Interface File
==================================

In this step, you need to write a Charm++ interface file (.ci).
In the file you can declare parallel chare arrays and their
entry methods. The syntax is the same as in Charm++.

::

         // ## Just replace Hello throughout with your chare's name. ##
         // ## and add your chare's entry points below where indicated ##
         // ## Everything else remains the same ##
         mainmodule hello {
           // declare readonly variables which once set is available to all
           // Chares across processors.
           readonly int chunkSize;

           array [1D] Hello {
             entry Hello();

             // Note how your Fortran function takes the above defined
             // message instead of a list of parameters.
             entry void SayHi(int a, double b, int n, int arr[n]);

             // Other entry points go here
             entry [reductiontarget] void MyReduction(int result);
           };
         };

Note, you cannot declare a main chare in the interface file, and you
also are not supposed to declare messages. Furthermore, the entry
functions must be declared with explicit parameters instead of using
messages.

Writing the F90 Program
=======================

To start, you need to create a Fortran Module to represent a chare, e.g.
{ChareName}Mod.

.. code-block:: fortran

         ! ## Just replace Hello throughout with your chare's name. ##
         ! ## and add your chare's personal data below where indicated ##
         ! ## Everything else remains the same ##
         MODULE HelloMod

         TYPE Hello
         ! ## your chare's data goes here ##
         integer data
         END TYPE

         TYPE HelloPtr
         TYPE (Hello), POINTER ::  obj
         integer*8 aid
         END TYPE

         END MODULE

In the Fortran file you must write an allocate function for this chare
with the name: Hello_allocate.

.. code-block:: fortran

         ! ## Just replace Hello throughout with your chare's name. ##
         ! ## Everything else remains the same ##
         SUBROUTINE Hello_allocate(objPtr, aid, index)
         USE HelloMod
         TYPE(HelloPtr) objPtr
         integer*8 aid
         integer index

         allocate(objPtr%obj)
         objPtr%aid = aid;
         ! ## you can initialize the Chare user data here
         objPtr%obj%data = index
         END SUBROUTINE

Now that you have the chare and the chare constructor function, you can
start to write entry functions as declared in the .ci files.

.. code-block:: fortran

         ! ## p1, p2, etc represent user parameters
         ! ## the "objPtr, myIndex" stuff is required in every Entry Point.
         ! ## CkExit() must be called by the chare to terminate.
         SUBROUTINE Hello_Entry_SayHi(objPtr, myIndex, data, data2, len, s)
         USE HelloMod
         IMPLICIT NONE

         TYPE(HelloPtr) objPtr
         integer myIndex
         integer data
         double precision data2
         integer len
         integer s(len)

         objPtr%obj%data = 20
         if (myIndex < 4) then
             call Hello_Invoke_SayHi(objPtr%aid, myIndex+1, 1, data2, len, s);
         else
             call CkExit()
         endif

Preliminary support for reductions is available as well. Support is
limited to reducing from a chare array to the first member of the same
array. Only basic built-in reducers are available. For an entry method
named MyReduction, tagged as a reduction target in the interface file, a
contribution can be made as follows:

.. code-block:: fortran

         external Hello_ReductionTarget_MyReduction

         call Hello_contribute(objPtr%aid, myIndex, sizeof(myIndex), myValue, CHARM_SUM_INT, Hello_ReductionTarget_MyReduction)

Now, you can write the main program to create the chare array and start
the program by sending the first message.

.. code-block:: fortran

         SUBROUTINE f90charmmain()
         USE HelloMod
         integer i
         double precision d
         integer*8 aid
         integer  s(8)

         call Hello_CkNew(5, aid)

         call set_ChunkSize(10);

         do i=1,8
             s(i) = i;
         enddo
         d = 2.50
         call Hello_Invoke_SayHi(aid, 0, 1, d, 4, s(3:6));

         END

This main program creates an chare array Hello of size 5 and send a
message with an integer, an double and array of integers to the array
element of index 0.

Compilation and Linking
=======================

Lastly, you need to compile and link the Fortran program with the Charm
runtime system as follows: (Let’s say you have written ``hellof.f90``, ``hello.ci`` and
``hello.C``.)

.. code-block:: bash

     > charmc hello.ci -language f90charm

will create ``hello.decl.h`` and ``hello.def.h``.

.. code-block:: bash

     > charmc -c hello.C

will compile ``hello.C`` with ``hello.decl.h`` and ``hello.def.h``.

.. code-block:: bash

     > charmc -c hellof.f90

charmc will invoke the Fortran compiler:

.. code-block:: bash

     > charmc -o hello hello.o hellof.o -language f90charm

will link ``hellof.o`` and ``hello.o`` against Charm’s Fortran90 library to create
a new executable program, ``hello``.

A 2D array example can be found in ``charm/examples/charm++/f90charm/hello2D``.

Running the Program
===================

To run the program, type:

.. code-block:: bash

    > ./charmrun +p2 hello

which will run ``hello`` on two PEs.