| blob | c73025f3a5077baca5ed919a425544ee3a90771b |
1 Summary of changes from WRF to WRF-fire November 6, 2008
3 1. Subgrid support
4 ------------------
6 The base WRF code is https://svn-wrf-model.cgd.ucar.edu/trunk as
7 of April 2, 2007 Last Changed Rev: 2285 Last Changed Date: 2007-04-02
8 10:10:25 -0600 (Mon, 02 Apr 2007). John Michalakes implemented
9 the subgrid support by changes to the files
11 external/RSL_LITE/gen_comms.c frame/module_domain.F tools/gen_wrf_io.c
12 tools/type.c tools/data.h tools/misc.c share/mediation_feedback_domain.F
13 Registry/registry.io_boilerplate dyn_em/solve_em.F
14 and new file dyn_em/module_initialize_fire.F
16 The subgrid support enables declaration in the registry of refined
17 meshes on the finest WRF grid, essentially by allocating more memory
18 to them than what normally would be. Subgrid variables live on the same
19 domain as the finest WRF grid but they are 2D only and have more points
20 by refinement ratios sr_x sr_y, which are given in namelist.input
22 The baseline code and John's modified code are stored in our git
23 repository in branch svnbranch', which has two commits that look
24 exactly like the file trees he sent us. The git repository can be
25 cloned from http://github.com/jbeezley/wrf-fire/tree/master or
26 http://repo.or.cz/w/wrffire.git
28 To get the changes that John made:
29 git diff wrfsvn wrfsubgrid
30 or a summary all modifications from the bare subgrid version:
31 git diff --stat wrfsubgrid origin/master
34 2. Coupling with the fire code
35 ------------------------------
37 The following files were changed (started by Net Patton, continued
38 by the present developers):
40 Registry/Registry.EM - new section to declare fire variables and halos,
41 new variables sr_x sr_y for the refinement ratios in the domain section
42 phys/module_physics_addtendc.F - code add the output of fire code
43 to heat and moisture tendencies
44 dyn_em/solve_em.F - call to the fire driver
45 phys/Makefile - new entries for the fire model
46 Makefile (top level) - new target em_fire
49 3. The fire model code itself
50 -----------------------------
52 The fire model is in files phys/*_fr_sfire_* in the phys directory.
53 We'll support these files as well as the fire section in the registry,
54 the call to our our driver in dyn_em/solve_em.F, and our entries
55 in phys/Makefile
57 The fire model is written following WRF coding conventions. The
58 fire code itself is independent of the WRF implementation with the
59 following exceptions:
61 phys/module_fr_sfire_driver.F:
62 USE module_domain USE module_configure USE module_driver_constants
63 USE module_machine USE module_tiles USE module_dm
64 Used to extract state variables and grid definitions from the domain
65 structure, extract configuration flags obtained from the registry,
66 interpolate atmospheric variables to fire subgrid resolution, and
67 set up tile level (openmp) parallelism. Any changes in WRF framework
68 could effect this file. The driver in this file takes care of
69 openmp threading for the fire code, so it also contains several
70 halo exchange includes. *Halo exchanging support is necessary in
71 the subgrid implementation.*
73 phys/module_fr_sfire_phys.F:
74 use module_model_constants, only: cp,xlv
75 Encapsulates everything related to physics. All other files contain
76 mathematical algorithms only with no reference to the physics,
77 units, etc. Used to compute heat fluxes to the atmosphere.
79 phys/module_fr_sfire_util.F:
80 use module_wrf_error - For printing debug and error messages
81 use module_dm - For getting MPI process information for debug
82 messages only.
83 Support routines such as interpolation and priting. All calls to
84 WRF routines pass through this module. It contains a number of
85 routines with I/O statements for debugging purposes. These can be
86 removed if necessary.
88 No other WRF standard modules or includes are used in the fire code.
91 4. Features not to be included in the WRF release
92 -------------------------------------------------
94 The following features are present in the master branch of our git
95 repository and will not be contained in the WRF release (which is
96 being prepared in the release branch of our repository):
98 Standalone model: The model is independent of WRF. Using phys/testmakefile
99 builds a completely standalone model. The standalone model replaces
100 module_fr_sfire_phys.F by a main program and uses the file
101 phys/wrf_fakes.F to simulate a subset of WRF infrastructure.
103 The CAWFE model: This is the original Clark-Coen model as ported
104 by Net Patton, implemented as files phys/*_fire_* and phys/*_cawfe_*,
105 and call fire_driver in dyn_em/solve_em.F
108 5. Fire data input and WPS
109 --------------------------
111 The fire code needs to have data inputs at the subgrid resolution.
112 At the moment, it is using fuel category data and interpolating
113 other data sources like slope and latitude/longitude from
114 WRF atmospheric resolution variables. In the future, we may find
115 it necessary to import more subgrid resolution variables from
116 standard data sources.
118 We are importing this data through a modified (for geogrid_wrapper.sh)
119 version of WPS, and WRF imports it from wrfinput_d?? just like
120 every other data source. The fuel category data we are currently
121 using comes from this website:
123 http://www.landfire.gov/viewer/
125 under 13 Anderson Fire Behavior Fuel Models, and
127 http://seamless.usgs.gov/index.php
129 under 1/3" NED, which is the same as the standard topo data, but much
130 higher resolution (10 m). The data from these sources need
131 to be converted into a geogrid compatible format. The script
132 geogrid_wrapper.sh in the WPS directory will attempt to fetch
133 the data and convert it automatically. This process is tricky because
134 it relies on reverse engineering the source's websites. It is
135 picky about checking that all of the form elements don't change
136 so that it doesn't do anything mean to the webservers like requesting
137 1 GB chunks of data continuously. As a result, if anything on
138 this website changes, the script will fail with an obscure message.
139 The conversion scripts are also somewhat prone to failure if the
140 dataset metadata changes in any way. There are also a number of
141 known limitations of the scripts at this time:
143 1. The dataset must be less than 250 Mb total in size. Anything
144 bigger and the website splits into multiple downloads. This
145 is fairly easy to fix if anyone runs into problems with it,
146 but that is unlikely in the near future because the domain
147 would have be many orders of magnitude larger than the test
148 problem.
150 2. The dataset must have less than 100,000 pixels in each axis.
151 This is a limitation of the geogrid format. The only way
152 around this is to split it into multiple different datasets
153 which geogrid will use simultaneously for a single field.
154 The dataset for the test problem is of size 746 x 575, so
155 again much larger domains will occur before this becomes and
156 issue.
158 The geogrid_wrapper.sh script is simple a caller for more a series
159 of more complicated python modules located in
160 other/convert_to_geogrid. Most of these modules can be
161 executed individually for use with custom data. Most importantly,
162 geogrid.py will convert most geotagged datasets into a format
163 importable by geogrid. Run `python geogrid.py -h` for syntax
164 and options. The limitation #2 above also applies to this script.
166 These scripts REQUIRE a number of external utilities:
167 Python 2.5+ (www.python.org)
168 Numeric python/numpy (www.scipy.org)
169 GDAL library and python bindings (www.gdal.org)
170 (See the binary distributions in the downloads section.)
171 twill (http://twill.idyll.org)
172 (Install with python's easy_install or simply extract the source somewhere
173 and add that path to PYTHONPATH.)
175 After the conversion the data and geogrid execution, the new data
176 is imported in the standard way. The repository contains a
177 slightly modified version of WPS that allows it to interact with
178 the wrapper script when data is missing. All other components are
179 part of the standard WPS distribution as of version 3.1.
182 6. Impact on portability, performance, and parallelization
183 ----------------------------------------------------------
185 The subgrid supports halo under RSL-LITE but not under RSL. The
186 fire code can be used only with the EM core at the moment.
188 In our testing so far the fire code takes about 20% of CPU, less
189 for larger number of processes or threads. So, the fire component
190 scales better than WRF itself. The fire code is not optimized yet
191 so the % of time it takes may well further decrease.
193 The fire code has been written to conform to WRF physics coding
194 standards as possible. There are no direct calls to MPI routines,
195 I/O, memory allocations, etc. It is written to Fortran 90 standards.
196 We primarily do development work on ifort v10.1 on Linux; however,
197 we have tested the code on the following compilers/platforms without
198 issues:
200 ifort v10.1/v11/gcc Linux; pgf90 v6.x/pgcc Linux (NCAR's walnut.mmm);
201 g95/gcc MacOS 10.5; xlf90/xlc on NCAR's frost
203 The current support of subgrid data in WPS should be considered
204 preliminary and it runs on a single processor and thread only.
206 The fire code can be deactivated at run-time through the namelist
207 variable ifire, in which case, the entire fire code will be skipped.
208 There is little performance penalty from a non-fire build. Any
209 difference is a result of increased allocated (but unused) memory
210 from the fire state variables and additional file I/O into restart
211 and history files.