doc: update Vala documentation

[automake.git] / HACKING

============================================================================
= This file

* This file attempts to describe the conventions to use when hacking automake.

* After git checkout from Savannah, you can do an initial build with, e.g.:
    ./bootstrap && ./configure --prefix=/tmp/amdev && make

============================================================================
= Administrivia

* The correct response to most actual bugs is to write a new test case
  which demonstrates the bug.  Then fix the bug, rerun the test suite,
  and check everything in.  Run the test suite in parallel or it takes ages.

* If you incorporate a change from somebody on the net:
  - First, if it is not a tiny change, you must make sure they have
    signed the appropriate paperwork.
  - Second, add their name and email address to THANKS.

* If a change fixes or adds a test, mention the test in the commit
  message.  If a change fixes a bug registered in the Automake debbugs
  tracker, mention the bug number in the commit message, using
  the short url form, like https://bugs.gnu.org/1234.  See section below
  about commit messages.

* If a person or people report a new bug, mention their name(s) in the
  commit message that fixes or exposes the bug, and add a line for them
  in THANKS.

* When documenting a non-trivial idiom or example in the manual, be
  sure to add a test case for it, and to reference such test case from
  a proper Texinfo comment.

* Some files in the automake package are not owned by automake (many
  by gnulib, https://gnu.org/s/gnulib); these files are listed in the
  $(FETCHFILES) variable in maintainer/maint.mk.  They should never be
  edited here.  All but two of them can be updated from respective
  upstreams with "make fetch" (this should be done especially before
  releases). The only exceptions are help2man (install the current
  version and copy in by hand) and 'lib/COPYING' (from FSF), which
  should be updated by hand whenever the GPL gets updated (which
  shouldn't happen that often anyway :-).

  Conversely, automake holds the master copy of a number of files that
  are copied into other projects, such as install-sh, mdate-sh,
  Channels.pm and ChannelDefs.pm, and plenty more; grep for bug-automake
  in lib/* for most of the scripts, and see <gnulib>/config/srclist.txt
  and <autoconf>/build-aux/fetch.pl for lists of such files on two
  notable receiving ends. Do your best not to break them.

* All changes from the last release that are not trivial bug fixes should
  be mentioned in NEWS.

* Changes which are potentially controversial, require a non-trivial
  plan, or must be implemented gradually with a roadmap spanning several
  releases (either minor or major) should be discussed on the list,
  and have a proper entry in the PLANS directory.  This entry should be
  always committed in the "maint" branch, even if the change it deals
  with is only for the master branch, or a topic branch.  Usually, in
  addition to this, it is useful to open a "wishlist" report on the
  Automake debbugs tracker, to keep the idea more visible, and have the
  discussions surrounding it easily archived in a central place.

===========================================================================
= Setting the development environment

* In development, ./GNUmakefile is used, not (the generated) ./Makefile.
  Run make V=1 to see the commands that are run.

* The required and optional dependencies used by Automake and its test suite
  can be automatically fetched using the GNU Guix package manager with the
  following command:

    guix environment automake --ad-hoc \
      gettext help2man texinfo libtool flex bison dejagnu zip icedtea \
      python gcc-toolchain gfortran pkg-config vala

  For other environments, you'll need to install the equivalent.

* To run the bin/automake or bin/aclocal scripts in a checkout, it is
  necessary to set PERL5LIB, as in:
     am=/path/to/automake/checkout
     env PERL5LIB=$am/lib $am/bin/automake --help
     env PERL5LIB=$am/lib $am/bin/aclocal --help

* If you need to test different Python versions, pyenv is probably the
  most convenient way at this writing. https://github.com/pyenv

============================================================================
= Naming

* Automake's convention is that internal AC_SUBSTs and make variables
  should be named with a leading 'am__', and internally generated targets
  should be named with a leading 'am--'.  This convention, although in
  place from at least February 2001, isn't yet universally used.
  But all new code should use it.

  We used to use '_am_' as the prefix for an internal AC_SUBSTs.
  However, it turns out that NEWS-OS 4.2R complains if a Makefile
  variable begins with the underscore character.  Yay for them.
  I changed the target naming convention just to be safe.

* If you'd like to read some general background on automake and the
  other GNU autotools, you might try this not-too-long web page (not
  affiliated with GNU):
  https://www.linux.com/news/best-practices-autotools/

============================================================================
= Editing '.am' files

* For variables, always use $(...) and not ${...}.

* Prefer ':' over 'true', mostly for consistency with existing code.

* Use '##' comments liberally.  Comment anything even remotely unusual,
  and even usual things, to help future readers of the code understand.

* Never use basename or dirname.  Instead, use sed.

* Do not use 'cd' within back-quotes, use '$(am__cd)' instead.
  Otherwise the directory name may be printed, depending on CDPATH.
  More generally, do not ever use plain 'cd' together with a relative
  directory that does not start with a dot, or you might end up in one
  computed with CDPATH.

* For install and uninstall rules, if a loop is required, it should be
  silent.  Then the body of the loop itself should print each "important"
  command it runs.  The printed commands should be preceded by a single
  space.

* Ensure install rules do not create any installation directory where
  nothing is to be actually installed.  See automake bug#11030.

============================================================================
= Editing automake.in and aclocal.in

* Indent using GNU style.  For historical reasons, the perl code
  contains portions indented using Larry Wall's style (perl-mode's
  default), and other portions using the GNU style (cperl-mode's
  default).  Write new code using GNU style.

* Don't use & for function calls, unless really required.
  The use of & prevents prototypes from being checked.

============================================================================
= Automake versioning and compatibility scheme

* There are three kinds of automake releases:

    - new major releases (e.g., 2.0, 5.0)
    - new minor releases (e.g., 1.14, 2.1)
    - micro a.k.a. "bug-fixing" releases (e.g., 1.13.2, 2.0.1, 3.5.17).

  A new major release should have the major version number bumped, and
  the minor and micro version numbers reset to zero.  A new minor release
  should have the major version number unchanged, the minor version number
  bumped, and the micro version number reset to zero.  Finally, a new
  micro version should have the major and minor version numbers unchanged,
  and the micro version number bumped by one.

  For example, the first minor version after 1.13.2 will be 1.14; the
  first bug-fixing version after 1.14 that will be 1.14.1; the first
  new major version after all such releases will be 2.0; the first
  bug-fixing version after 2.0 will be 2.0.1; and a further bug-fixing
  version after 2.0.1 will be 2.0.2.

* Micro releases should be just bug-fixing releases; no new features
  should be added, and ideally, only trivial bugs, recent regressions,
  or documentation issues should be addressed by them.  On the other
  hand, it's OK to include testsuite work and even testsuite refactoring
  in a micro version, since a regression there is not going to annoy or
  inconvenience Automake users, but only the Automake developers.

* Minor releases can introduce new "safe" features, do non-trivial but
  mostly safe code clean-ups, and even add new runtime warnings (rigorously
  non-fatal).  But they shouldn't include any backward incompatible change,
  nor contain any potentially destabilizing refactoring or sweeping change,
  nor introduce new features whose implementation might be liable to cause
  bugs or regressions in existing code.  However, it might be acceptable to
  introduce very limited and localized backward-incompatibilities, *only*
  if that is necessary to fix non-trivial bugs, address serious performance
  issues, or greatly enhance usability.  But please, do this sparsely and
  rarely!

* Major releases can introduce backward-incompatibilities (albeit such
  incompatibilities should be announced well in advance, and a smooth
  transition plan prepared for them), and try more risky and daring
  refactorings and code cleanups.  Still, backward incompatibilities are
  extremely undesirable and should be avoided at all costs.

* For more information, refer to the extensive discussion associated
  with automake bug#13578.

============================================================================
= Working with git

* To regenerate dependent files created by aclocal and automake,
  use the 'bootstrap' script.  It uses the code from the source
  tree, so the resulting files (aclocal.m4 and Makefile.in) should
  be the same as you would get if you install this version of
  automake and use it to generate those files. 

* To get a faithful and correct rebuild, run:
    ./bootstrap && ./config.status --recheck && make clean all

* Usually, it is best to run again latest stable versions of Autoconf,
  Libtool, etc., since that is what most users will do. Sometimes it may
  be necessary to use the development versions due to bugs fixed,
  etc. Whatever is first in PATH wins.
  (Some background: https://bugs.gnu.org/11347)
  
* The Automake git tree currently carries three basic branches:
  'master', 'next' and 'maint'.  In practice, for quite a few years now,
  as of this writing in 2023, we have been using only the master branch,
  due to lack of time and desire to deal with anything but fixing bugs.
  The branch information in the items below should thus be taken with a
  large dose of salt.

* The 'master' branch is where the development of the next release
  takes place.  It should be kept in a stable, almost-releasable state,
  to simplify testing and deploying of new minor version.  Note that
  this is not a hard rule, and such "stability" is not expected to be
  absolute (emergency releases are cut from the 'maint' branch anyway).

* When planning a release a dedicated branch should be created and after
  the release is done, the release branch is to be merged both into the
  'master' branch and the 'maint' branch.

* Besides merges from release branches, the 'maint' branch can contain
  fixes for regressions, trivial bugs, or documentation issues, that
  will be part of an emergency regression-fixing or security releases.
  As a consequence it should always be kept in a releasable state and no
  "active" development should be done whatsoever.

* The 'next' branch is reserved for the development of the next major
  release.  Experimenting a little is OK here, but don't let the branch
  grow too unstable; if you need to do exploratory programming or
  over-arching change, you should use a dedicated topic branch, and
  only merge that back once it is reasonably stable.

* The 'master' branch should be kept regularly merged into the 'next'
  branch.  It is advisable to merge only after a set of related
  commits have been applied, to avoid introducing too much noise in
  the history.

* There may be a number of longer-lived feature branches for new
  developments.  They should be based off of a common ancestor of all
  active branches to which the feature should or might be merged later.

* When merging, prefer 'git merge --log' over plain 'git merge', so that
  a later 'git log' gives an indication of which actual patches were
  merged even when they don't appear early in the list.

* The 'master', 'maint' and 'next' branches should not be rewound,
  i.e., should always fast-forward, except maybe for privacy issues.
  For feature branches, the announcement for the branch should
  document the rewinding policy.
  If a topic branch is expected to be rewound, it is good practice to put
  it in the 'experimental/*' namespace; for example, a rewindable branch
  dealing with Vala support could be named like "experimental/vala-work".

* If you need to trivially fix a change after a commit, e.g., a typo in
  the NEWS entry, edit the file and then:
    git commit --amend --no-edit NEWS
    git commit --amend --date="$(date -R)" # update date of commit

* If you want to just completely lose your local changes:
    git fetch && git reset --hard origin && git checkout && git submodule update
  (the submodule stuff is for gnulib).

* The preferred way to create a patch to mail around:
    git format-patch --stdout -1 >/tmp/some-patch.diff
  which can then be applied with:
    git am mboxfile

============================================================================
= Writing a good commit message

* Here is the general format that Automake's commit messages are expected
  to follow.  See the further points below for clarifications and minor
  corrections.

      topic: brief description (this is the "summary line")

      <reference to relevant bugs, if any>

      Here goes a more detailed explanation of why the commit is needed,
      and a general overview of what it does, and how.  This section
      should almost always be provided, possibly only with the exception
      of obvious fixes or very trivial changes.

      And if the detailed explanation is quite long or detailed, you can
      want to break it in more paragraphs.

      Then you can add references to relevant mailing list discussions
      (if any), with proper links.  But don't take this as an excuse for
      writing incomplete commit messages!  The "distilled" conclusions
      reached in such discussions should have been placed in the
      paragraphs above.

      Finally, here you can thank people that motivated or helped the
      change.  So, thanks to John Doe for bringing up the issue, and to
      J. Random Hacker for providing suggestions and testing the patch.

      <detailed list of touched files>

* To choose the topic label, please review the previous commits in the
  git log or the ChangeLog file from a release tarball. There are no
  hard-and-fast rules; the aim is a usable description. So script
  names, Makefile targets, and more are all viable. Some examples:
  aclocal automake build cosmetics doc maint python release test ylwrap

* The <detailed list of touched files> should usually be provided (but
  for short or trivial changes), and should follow the GNU guidelines
  for ChangeLog entries (described explicitly in the GNU Coding
  Standards); it might be something of this sort:

    * some/file (func1): Improved frobnication.
    (func2): Adjusted accordingly.
    * another/file (foo, bar): Likewise.
    * t/foo.sh: New test.
    * t/list-of-tests.mk (handwritten_TESTS): Add it.
    * doc/automake.texi (Some Node): Document it.
    * NEWS: Mention it.

* If your commit fixes an automake bug registered in the tracker (say
  numbered 1234), you should put the following line after the summary
  line:

      Fixes automake bug https://bugs.gnu.org/1234.

* If your commit is just related to the given bug report, but does not
  fix it, you might want to add a line like this instead:

      Related to automake bug https://bugs.gnu.org/1234.

* When referring to older commits, use 'git describe' output as pointer.
  But also try to identify the given commit by date and/or summary line
  if possible.  Examples:

      Since yesterday's commit, v1.11-2019-g4d2bf42, ...

      ... removed in commit 'v1.11-1674-g02e9072' of 2012-01-01,
      "dist: ditch support for lzma"...

* If the commit is a tiny change that is exempt from copyright paperwork, the
  commit message should contain a separate line after the detailed list of
  touched files like the following:

      Copyright-paperwork-exempt: yes

* Generally write the commit message in present tense.

============================================================================
= Test suite

* See file 't/README' for more information.

* Use "make check" and "make maintainer-check" liberally.  It is often
  useful to set VERBOSE=1 for more reporting on what is being done.

* Export the 'keep_testdirs' environment variable to "yes" to keep
  *.dir test directories for successful tests also.

* Use Perl coverage information to ensure your new code is thoroughly
  tested by your new tests.

* To run the tests, you should install expect, shar, language compilers,
  gettext macros.  Anything you don't install won't be tested.  The test
  suite will report on tests skipped due to software not available.

* Run the test suite in parallel (e.g., "make -j12 check"), both so it
  doesn't take forever and because that is what most users will do.  You
  can also parallelize the makes that run inside each test with, e.g.:
    make check AM_TESTSUITE_MAKE="make -j$(( 1*$(nproc) + 1 ))"
  If you like, try different levels of parallelization to see what
  runs the fastest on your machine.  We'll use -j12 in this document.

* To summarize, here is a typical make check invocation:
    make -j12 VERBOSE=1 check keep_testdirs=yes

* Then check t/test-suite.log for the overall results.  The directory
  t/TESTNAME.dir is where the work will be left, if the test fails.

* You can run a single test, with, e.g.,
    make check TESTS='t/aclocal-acdir.sh'
  where t/aclocal-acdir.sh can be any t/*.sh test, including a new one
  you are writing.  You may want to add --no-print-dir to silence GNU
  make about the many cd commands, and/or env VERBOSE=1 to get more
  information about what make is running.

* Sometimes you may want to see when each test starts running, not only
  when they complete.  This can be done by setting TESTS_ENVIRONMENT:
    make -j12 VERBOSE=1 TESTS_ENVIRONMENT='echo RUNNING: "$$f";' check

* Run "make maintainer-check" before commit.  Watch out for trailing spaces.
  Probably useful to run it more verbosely:
    make AM_V_GEN= AM_V_at= VERBOSE=1 maintainer-check

* After "make -j12 check" succeeds. run "make -j12 distcheck" before
  pushing a commit, since that exercises yet more of the code.

* To un-silence (some) of what make does, use make V=1.
  For yet more verbosity, make VERBOSE=1.

* To set up a new test, first write the test file in t/good-name.sh.
  Choose a name that fits with existing ones, as best you can devise.

  - You'll likely want to copy material from an existing test, which is
  fine and good; depending on how much is copied, it may be useful to
  mention the other test(s) you used.

  - Nevertheless, start the copyright year list in the new file fresh,
  with the current year.

  - You can run the new test on its own with
    make check TESTS='t/good-name.sh'

  - During development of a new test, it can be useful to end it with
  "exit 33" (or any random value) to force it to fail even when it would
  otherwise succeed, so you can inspect the test.dir/test.log and other
  test.dir/* files to be sure things worked as expected.

  - At some point before releasing, add the test to the appropriate
  variable in t/list-of-tests.mk, most likely the (alphabetical)
  handwritten_TESTS.

* Some hints for debugging test failures or trying to understand the
  (complex) test infrastructure.
  
  - You may want a simple test just to exercise the setup; t/amassign.sh
  is such a test. For a simple test that runs automake twice (sometimes
  useful in finding concurrency failures), try t/nodef.sh.
  
  - The t/ax/ dir contains most of the test infrastructure files.

  - To trace the (complex) test initialization code, change the set -e
  to set -ex (or whatever you wish) in t/ax/test-init.sh.
  Automake itself enables -x for the *.log and test-suite.log files,
  in am_test_setup in t/ax/test-lib.sh. You may want to change to -vx.
  
  - If you want to run the individual commands of a test one by one in a
  shell, you have to reproduce the test environment. Start a subshell and:
    PATH=$am/bin:$am/t/ax:$PATH # where $am is your automake source checkout
    set +u          # if needed; automake cannot handle this "nounset" option
    . test-init.sh  # beginning of test
  Then each command you type will be executed in the test environment,
  including -x being enabled. Each test prints the PATH value at the beginning.
  
  - If you need to debug a test which fails under installcheck, you need
  to invoke make with am_running_installcheck=yes, as is done by the
  installcheck-testsuite target in t/local.mk.
  
  - If you want to see the (complex shell) commands that make is
  running, despite all of Automake's attempt at silence, run (GNU)
    make --debug=p ... other make args ...

============================================================================
= Bug tracker

* Automake uses the debbugs instance at https://bugs.gnu.org.  Email
  from the tracker is sent to bug-automake@gnu.org, and vice versa.
  Ditto automake-patches@gnu.org. (See
  https://gnu.org/s/automake for all the mailing lists; if you're
  working on Automake, please subscribe.)

* To see all open bugs resp. patches (and recently closed ones):
    https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=automake
    https://debbugs.gnu.org/cgi/pkgreport.cgi?pkg=automake-patches

* For a full search form, initialized to show only bugs tagged confirmed:
    https://debbugs.gnu.org/cgi/pkgreport.cgi?package=automake;include=tags%3Aconfirmed

* We use the "confirmed" tag to mean bugs that have been reviewed, are
  evidently valid, but no one is currently working or plans to work on
  them.  Thus they are good candidates for new volunteers to get involved.

* To download the so-called maintainer mbox for a bug, say 12345:
    https://debbugs.gnu.org/cgi/bugreport.cgi?bug=12345;mbox=yes;mboxmaint=yes
  Only the maintainer mbox consistently has the bug# prefix on Subject:
  lines, so it is the most usable, as far as we know.

* To close a bug, you can mail (or bcc) to 12345-done@debbugs.gnu.org.
  It's best to do this with a note saying that the bug is being closed
  because the fix was pushed, or whatever the reason.

* To add tags "help" and "confirmed" to bug 12345, mail to
  control@debbugs.gnu.org with a one-line body:
    tags 12345 + help confirmed

* To merge bugs 12345 and 56789, mail to
  control@debbugs.gnu.org with a one-line body:
    merge 12345 56789

* In general, all bug operations are done by mail.  Reference information:
    https://debbugs.gnu.org/server-control.html # dev info

============================================================================
= Release procedure

* The steps outlined here are meant to be followed for alpha and stable
  releases as well.  Where differences are expected, they will be
  explicitly described.

* Fetch new versions of the files that are maintained by the FSF by
  running "make fetch".  help2man has to be updated separately.  In case
  any file in the automake repository has been updated, commit and
  rerun the testsuite.

* Ensure that the copyright notices of the distributed files are up to
  date.  The maintainer-only target "update-copyright" can help with this.

* Check NEWS; in particular, ensure that all the relevant differences
  with the last release are actually reported.

* Update the version number in configure.ac.  Leading up to a release,
  say 1.17, pretests should be numbered from 1.16.90. Even numbers
  (1.16.90, 1.16.92, ...) should be the pretest releases; odd numbers
  (1.16.91, 1.16.93, ...) should be only the development versions.
  Leading up to a release 1.17.1, we would have 1.16.0.90, etc.
  Before 1.17 (January 2024), we used suffixed letters for pretests,
  as is traditional, but deficient sorting algorithms did not like that.

* Run these commands, in this order (as mentioned, adjust -j as desired):
    make bootstrap
    make -j12 check keep_testdirs=yes
    make maintainer-check
    make -j12 distcheck                                # regular distcheck
    make -j12 distcheck AM_TESTSUITE_MAKE="make -j12"  # parallelize makes
    make -j12 check-no-trailing-backslash-in-recipes
    make -j12 check-cc-no-c-o

  It is also advised to run "git clean -fdx" before invoking the
  bootstrap, to ensure a really clean rebuild.  However, it must
  be done carefully, because that command will remove *all* the
  files that are not tracked by git!

* Run "make git-tag-release".
  This will run the maintainer checks, verify that the local git
  repository and working tree are clean and up-to-date, and create
  a proper signed git tag for the release (based on the contents
  of $(VERSION)).

* Run "make git-upload-release".
  For pretest releases: "DEVEL_SNAPSHOT=1 make git-upload-release".
  
  This will first verify that you are releasing from a tagged version
  and that the local git repository and working tree are clean and
  up-to-date, and will then run "make dist" to create the tarballs,
  and invoke the 'gnupload' script sign and upload them to the correct
  locations.  In case you need to sign with a non-default key, you can
  use "make GNUPLOADFLAGS='--user KEY' git-upload-release".

* For stable releases you'll have to update the manuals at www.gnu.org.

  - Check links:
       make html
       make checklinkx # various perl prerequisites, see contrib/checklinkx
    or use:
      <https://validator.w3.org/checklink>

  - Generate manuals (with the help of the standard gendocs.sh script):
       make web-manual

    The ready-to-be-uploaded manuals (in several formats) will be left
    in the 'doc/web-manuals' directory.

  - Commit the updated manuals to web CVS:
      make web-manual-update

    If your local username is different from your username at Savannah,
    you'll have to override the 'CVS_USER' make variable accordingly;
    for example:
      make web-manual-update CVS_USER=slattarini

* Create an announcement message with "make announcement".  Edit the
  generated 'announcement' file appropriately, in particularly filling
  in by hand any "TODO" left in there.

* Update version number in configure.ac to next alpha number.
  Rerun ./bootstrap and commit.

* Don't forget to "git push" your changes so they appear in the public
  git tree.

* Send the announcement generated in the earlier steps at least to
  <autotools-announce@gnu.org> and <automake@gnu.org>.  If the release
  is a stable one, the announcement must also go to <info-gnu@gnu.org>;
  if it is an alpha or beta release, announcement should be sent also
  to <platform-testers@gnu.org>, to maximize the possibility of early
  testing on exotic or proprietary systems.  Finally, copy an abridged
  version of the announcement into the NEWS feed at:
  <https://savannah.gnu.org/projects/automake>.
  Be sure to link a  version to the complete announcement (from
  the version you sent to the automake list, as get archived on
  <https://lists.gnu.org/archive/html/automake/>).

---------
Copyright (C) 2003-2024 Free Software Foundation, Inc.

This program is free software; you can redistribute it and/or modify
it under the terms of the GNU General Public License as published by
the Free Software Foundation; either version 2, or (at your option)
any later version.

This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
GNU General Public License for more details.

You should have received a copy of the GNU General Public License
along with this program.  If not, see <https://www.gnu.org/licenses/>.

Local Variables:
mode: text
End: