fix regex error message parsing (GRegex indexes subgroups, not matches)

[geany-mirror.git] / HACKING

=============
Hacking Geany
=============
.. contents::

General
=======

About this file
---------------
This file contains information for anyone wanting to work on the Geany
codebase. You should be aware of the open source licenses used - see
the README file or the documentation. It is reStructuredText; the
source file is HACKING. You can generate hacking.html by running ``make
hacking-doc`` from the doc/ subdirectory.

Writing plugins
---------------
* src/plugindata.h contains the plugin API data types.
* See plugins/demoplugin.c for a very basic example plugin.
* src/plugins.c loads and unloads plugins (you shouldn't need to read
  this really).
* The API documentation contains a few basic guidelines and hints to
  write plugins.

You should generate and read the plugin API documentation, see below.

Plugin API documentation
^^^^^^^^^^^^^^^^^^^^^^^^
You can generate documentation for the plugin API using the doxygen
tool. Run ``make api-doc`` in the doc subdirectory. The documentation
will be output to doc/reference/index.html.
Alternatively you can view the API documentation online at
http://www.geany.org/manual/reference/.

Patches
-------
We are happy to receive patches, but it's best to check with us by email
or mailing list whether a new feature is appropriate, and whether someone
is already working on similar code.

In general it's best to provide git-formatted patches made from the
current Git (see `Committing`_)::

    $ git commit -a
    $ git format-patch HEAD^

We also accept patches against other releases, but it's more work for us.

If you're not using Git, although you're strongly suggested to used it,
you can use the diff command::

    $ diff -u originalpath modifiedpath > new-feature.patch

However, such a patch won't contain the authoring information nor the
patch's description.

.. note::
    Please make sure patches follow the style of existing code - In
    particular, use tabs for indentation. See `Coding`_.

Windows tools
-------------
* Git: http://git-scm.com/ and http://code.google.com/p/msysgit/
* diff, grep, etc: http://mingw.org/ or http://unxutils.sourceforge.net/

See also the 'Building on Windows' document on the website.

File organization
-----------------
callbacks.c is just for Glade callbacks.
Avoid adding code to geany.h if it will fit better elsewhere.
See the top of each ``src/*.c`` file for a brief description of what
it's for.

Plugin API code
---------------
Please be aware that anything with a doc-comment (a comment with an
extra asterix: ``/**``) is something in the plugin API. Things like
enums and structs can usually still be appended to, ensuring that all
the existing elements stay in place - this will keep the ABI stable.

.. warning::

    Some structs like GeanyCallback cannot be appended to without
    breaking the ABI because they are used to declare structs by
    plugins, not just for accessing struct members through a pointer.
    Normally structs should never be allocated by plugins.

Keeping the plugin ABI stable
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Before the 1.0 release series, the ABI can change when necessary, and
even the API can change. An ABI change just means that all plugins will
not load and they must be rebuilt. An API change means that some plugins
might not build correctly.

If you're reordering or changing existing elements of structs that are
used as part of the plugin API, you must increment GEANY_ABI_VERSION
in plugindata.h. This is usually not needed if you're just appending
fields to structs. The GEANY_API_VERSION value should be incremented
for any changes to the plugin API, including appending elements.

If you're in any doubt when making changes to plugin API code, just ask us.

Plugin API/ABI design
^^^^^^^^^^^^^^^^^^^^^
You should not make plugins rely on the size of a struct. This means:

* Don't let plugins allocate any structs (stack or heap).
* Don't let plugins index any arrays of structs.
* Don't add any array fields to structs in case we want to change the
  array size later.

Doc-comments
^^^^^^^^^^^^
* The @file tag can go in the source .c file, but use the .h header name so
  it appears normally in the generated documentation. See ui_utils.c for an
  example.
* Function doc-comments should always go in the source file, not the
  header, so they can be updated if/when the implementation changes.

Glade
-----
Add user-interface widgets to the Glade 3 file ``data/geany.glade``.
Callbacks for the user-interface should go in ``src/callbacks.c``.

GTK versions & API documentation
--------------------------------
Geany requires GTK >= 2.16 and GLib >= 2.20. API symbols from newer
GTK/GLib versions should be avoided or made optional to keep the source
code building on older systems.

The official GTK 2.16 API documentation may not be available online
anymore, so we put it on http://www.geany.org/manual/gtk/. There
is also a tarball with all available files for download and use with
devhelp.

Using the 2.16 API documentation of the GTK libs (including GLib, GDK
and Pango) has the advantages that you don't get confused by any
newer API additions and you don't have to take care about whether
you can use them or not.

Coding
------
* Don't write long functions with a lot of variables and/or scopes - break
  them down into smaller static functions where possible. This makes code
  much easier to read and maintain.
* Use GLib types and functions - gint not int, g_free() not free().
* Your code should build against GLib 2.20 and GTK 2.16. At least for the
  moment, we want to keep the minimum requirement for GTK at 2.16 (of
  course, you can use the GTK_CHECK_VERSION macro to protect code using
  later versions).
* Variables should be declared before statements. You can use
  gcc's -Wdeclaration-after-statement to warn about this.
* Don't let variable names shadow outer variables - use gcc's -Wshadow
  option.
* Do not use G_LIKELY or G_UNLIKELY (except in critical loops). These
  add noise to the code with little real benefit.

Compiler options & warnings
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Use ``CFLAGS='-Wfoo' ./configure`` or ``CFLAGS='-Wfoo' ./autogen.sh``
to set warning options (as well as anything else e.g. -g -O2).

* Enable warnings - for gcc use '-Wall -W' (and optionally
  -Wno-unused-parameter to avoid unused parameter warnings in Glade
  callbacks).
* You should try to write ISO C99 code for portability, so always
  use C ``/* */`` comments and function_name(void) instead of
  function_name(). This is for compatibility with various Unix-like
  compilers. You should use -std=c99 to help check this.

.. tip::
    Remember for gcc you need to enable optimization to get certain
    warnings like uninitialized variables, but for debugging it's
    better to have no optimization on.

Style
^^^^^
* We use a tab width of 4 and indent completely with tabs not spaces.
  Note the documentation files use (4) spaces instead, so you may want
  to use the 'Detect from file' indent pref.
* Do not add whitespace at the end of lines, this adds to commit noise.
  When editing with Geany set preference files->Strip trailing spaces
  and tabs.
* Use the multiline comment ``/* */`` to comment small blocks of code,
  functions descriptions or longer explanations of code, etc. The more
  comments are in your code the better. (See also
  ``scripts/fix-cxx-comments.pl`` in Git).
* Lines should not be longer than about 100 characters and after 100
  characters the lines should be wrapped and indented once more to
  show that the line is continued.
* We don't put spaces between function names and the opening brace for
  argument lists.
* Variable declarations come first after an opening brace, then one
  newline to separate declarations and code.
* 2-operand operators should have a space each side.
* Function bodies should have 2 blank newlines after them.
* Align braces together on separate lines.
* Don't put assignments in 'if/while/etc' expressions except for loops,
  for example ``for (int i = 0; i < some_limit; i++)``.
* if statements without brace bodies should have the code on a separate
  line, then a blank line afterwards.
* Use braces after if/while statements if the body uses another
  if/while statement.
* Try to fit in with the existing code style.

.. note::
    A few of the above can be done with the Git
    ``scripts/fix-alignment.pl``, but it is quite dumb and it's much better
    to write it correctly in the first place.
    ``scripts/rstrip-whitespace.py`` just removes trailing whitespace.


.. below tabs should be used, but spaces are required for reST.

Example::

    struct Bar;

    typedef struct Foo  /* struct names normally are the same as typedef names */
    {
        gint    foo;    /* names are somewhat aligned visually */
        gint    bar;    /* fields don't share the same line */
        SomeLongTypeName baz; /* alignment is not strict */
        gchar   *ptr;   /* pointer symbol must go next to variable name, not type */
        Bar     public; /**< only plugin API fields have a doc-comment */
    }
    Foo;


    gint some_func(void);

    gint some_other_func(void);


    /* optional function comment explains something important */
    gint function_long_name(gchar arg1, <too many args to fit on this line>,
            gchar argN)
    {
        /* variable declarations always go before code in each scope */
        /* variable names should NOT be aligned at all */
        gint foo, bar;  /* variables can go on the same line */
        gint baz;       /* but often don't */
        gchar *ptr;     /* pointer symbol must go next to variable name, not type */
        gchar *another; /* pointers should normally go on separate lines */

        /* Some long comment block
         * taking several different
         * lines to explain */
        if (foo)
        {
            /* variables only used in one scope should normally be declared there */
            gint dir = -1;

            bar = foo;
            if ((bar & (guint)dir) != 7)
                some_code(arg1, <too many args to fit on this line>,
                    argN - 1, argN);

            some_func();
        }
    }


    /** Explains using doc-comments for plugin API functions.
     * First line should be short and use the third person tense - 'explains',
     * not 'explain'.
     *
     * @return Some number.
     * @since 1.22. */
    gint another_function(void)
    {
        ...


Committing
----------

* Commit one thing at a time, do small commits.  Commits should be
  meaningful and not too big when possible; multiple small commits are
  good if there is no good reason to group them.
* Use meaningful name and email in the Author and Committer fields.
  This helps knowing who did what and allows to contact the author if
  there is a good reason to do so (unlikely, but can happen).
* When working on a new feature, create a new branch for it.  When
  merging it, use the --no-ff option to make sure a merge commit will
  be created to better track what happened.  However, if the feature
  only took one commit you might merge it fast-forward since there is
  not history to keep together.

Commit messages
^^^^^^^^^^^^^^^
Follow the standard Git formatting:

* No line should use more than about 80 characters (around 72 is best).
* The first line is the commit's summary and is followed by an empty
  line.  This summary should be one line and one line only, thus less
  than 80 characters.  This summary should not include any punctuation
  unless really needed.  See it as the subject of an email: keep it
  concise and as precise as you can, but not tool long.
* Following lines are optional detailed commit information, with
  paragraphs separated by blank lines.  This part should be as long as
  needed to describe the commit in depth, should use proper
  punctuation and should include any useful information, like the
  motivation for the patch and/or any valuable details the diff itself
  don't provide or don't make clear.  Make it as complete as you think
  it makes sense, but don't include an information that is better
  explained by the commit's diff.

  It is OK to use ASCII formatting like bullet list using "*" or "-",
  etc. if useful, but emphasis (bold, italic, underline) should be
  avoided.

Example::

    Ask the user if spawn fails in utils_open_browser()

    Ask the user to configure a valid browser command if spawning it
    fails rather than falling back to some arbitrary hardcoded defaults.

    This avoid spawning an unexpected browser when the configured one is
    wrong, and gives the user a chance to correctly fix the preference.


Testing
-------
* Run with ``-v`` to print any debug messages.
* You can use a second instance (``geany -i``).
* To check first-run behaviour, use an alternate config directory by
  passing ``-c some_dir`` (but make sure the directory is clean first).
* For debugging tips, see `GDB`_.

Bugs to watch out for
---------------------
* Forgetting to check *doc->is_valid* when looping through
  *documents_array* - instead use *foreach_document()*.
* Inserting fields into structs in the plugin API instead of appending.
* Not breaking the plugin ABI when necessary.
* Using an idle callback that doesn't check main_status.quitting.
* Forgetting to call vStringTerminate in CTags code.
* Forgetting CRLF line endings on Windows.
* Not handling Tabs & Spaces indent mode.

Libraries
---------
We try to use an unmodified version of Scintilla - any new lexers or
other changes should be passed on to the maintainers at
http://scintilla.org. We normally update to a new Scintilla release
shortly after one is made. See also scintilla/README.

Tagmanager was originally taken from Anjuta 1.2.2, and parts of it
(notably c.c) have been merged from later versions of Anjuta and
CTags. The independent Tagmanager library itself ceased development
before Geany was started. It's source code parsing is mostly taken from
Exuberant CTags (see http://ctags.sf.net). If appropriate it's good to
pass language parser changes back to the CTags project.


Notes
=====
Some of these notes below are brief (or maybe incomplete) - please
contact the geany-devel mailing list for more information.

Using pre-defined autotools values
----------------------------------
When you are use macros supplied by the autotools like GEANY_PREFIX,
GEANY_LIBDIR, GEANY_DATADIR and GEANY_LOCALEDIR be aware that these
might not be static strings when Geany is configured with
--enable-binreloc. Then these macros will be replaced by function calls
(in src/prefix.h). So, don't use anything like
printf("Prefix: " GEANY_PREFIX); but instead use
printf("Prefix: %s", GEANY_PREFIX);

Adding a source file foo.[hc] in src/ or plugins/
-------------------------------------------------
* Add foo.c, foo.h to SRCS in path/Makefile.am.
* Add foo.o to OBJS in path/makefile.win32.
* Add path/foo.c to geany_sources in wscript.
* Add path/foo.c to po/POTFILES.in (for string translation).

Adding a filetype
-----------------
You can add a filetype without syntax highlighting or tag parsing, but
check to see if those features have been written in upstream projects
first (scintilla or ctags).

**Custom:**

If you want to reuse an existing lexer and/or tag parser, making a
custom filetype is probably easier - it doesn't require any
changes to the source code. Follow instructions in the manual:
http://geany.org/manual/geany.html#custom-filetypes. Don't forget to
update the ``[Groups]`` section in ``filetype_extensions.conf``.

.. warning::
  You should use the newer `[build-menu]` section for default build
  commands - the older `[build_settings]` may not work correctly for
  custom filetypes.

**Built-in:**

* Add GEANY_FILETYPES_FOO to filetypes.h.
* Initialize GEANY_FILETYPES_FOO in init_builtin_filetypes() of
  filetypes.c.
* Update data/filetype_extensions.conf.

The remaining notes relate mostly to built-in filetypes.

filetypes.* configuration file
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
All languages need a data/filetypes.foo configuration file. See
the "Filetype definition files" section in the manual and/or
data/filetypes.c for an example.

Programming languages should have:

* [keywords] if the lexer supports it.
* [settings] mostly for comment settings.
* [build-menu] (or [build_settings]) for commands to run.

For languages with a Scintilla lexer, there should be a [styling] section,
to correspond to the styles used in highlighting_styles_FOO[] in
highlightingmappings.h - see below.

Syntax highlighting
^^^^^^^^^^^^^^^^^^^
It may be possible to use an existing Scintilla lexer in the scintilla/
subdirectory - if not, you will need to find (or write) one,
LexFoo.cxx. Try the official Scintilla project first.

.. warning::
    We won't accept adding a lexer that conflicts with one in
    Scintilla. All new lexers should be submitted back to the Scintilla
    project to save duplication of work.

When adding a lexer, update:

* scintilla/Makefile.am
* scintilla/makefile.win32
* wscript
* scintilla/src/Catalogue.cxx - add a LINK_LEXER command *manually*

For syntax highlighting, you will need to edit highlighting.c and
highlightingmappings.h and add the following things:

1. In highlightingmappings.h:

   a. define ``highlighting_lexer_FOO`` to the Scintilla lexer ID for
      this filtype, e.g. ``SCLEX_CPP``.
   b. define the ``highlighting_styles_FOO`` array that maps Scintilla
      style states to style names in the configuration file.
   c. define ``highlighting_keywords_FOO`` to ``EMPTY_KEYWORDS`` if the
      filtype has no keywords, or as an ``HLKeyword`` array mapping
      the Scintilla keyword IDs to names in the configuration file.
   d. define ``highlighting_properties_FOO`` to ``EMPTY_PROPERTIES``, or
      as an array of ``HLProperty`` if the filetype requires some lexer
      properties to be set.  However, note that properties should
      normally be set in the ``[lexer_properties]`` section of the
      configuration file instead.

   You may look at other filtype's definitions for some examples
   (Ada, CSS or Diff being good examples).

2. In highlighting.c:

   a. Add ``init_styleset_case(FOO);`` in ``highlighting_init_styles()``.
   b. Add ``styleset_case(FOO);`` in ``highlighting_set_styles()``.

3. Write data/filetypes.foo configuration file [styling] section. See
   the manual and see data/filetypes.d for a named style example.

.. note::
    Please try to make your styles fit in with the other filetypes'
    default colors, and to use named styles where possible (e.g.
    "commentline=comment"). Filetypes that share a lexer should have
    the same colors. If not using named styles, leave the background color
    empty to match the default color.

Error message parsing
^^^^^^^^^^^^^^^^^^^^^
New-style error message parsing is done with an extended GNU-style regex
stored in the filetypes.foo file - see the [build_settings] information
in the manual for details.

Old-style error message parsing is done in
msgwin_parse_compiler_error_line() of msgwindow.c - see the ParseData
typedef for more information.

Other features
^^^^^^^^^^^^^^
If the lexer has comment styles, you should add them in
highlighting_is_comment_style(). You should also update
highlighting_is_string_style() for string/character styles. For now,
this prevents calltips and autocompletion when typing in a comment
(but it can still be forced by the user).

For brace indentation, update lexer_has_braces() in editor.c;
indentation after ':' is done from on_new_line_added().

If the Scintilla lexer supports user type keyword highlighting (e.g.
SCLEX_CPP), update document_update_tags() in document.c.

Adding a TagManager parser
^^^^^^^^^^^^^^^^^^^^^^^^^^
This assumes the filetype for Geany already exists.

First write or find a CTags compatible parser, foo.c. Note that there
are some language patches for CTags at:
http://sf.net/projects/ctags - see the tracker.

(You can also try the Anjuta project's tagmanager codebase.)

.. note::
    From Geany 1.22 GLib's GRegex engine is used instead of POSIX
    regex, unlike CTags. It should be close enough to POSIX to work
    OK.
    We no longer support regex parsers with the "b" regex flag
    option set and Geany will print debug warnings if it's used.
    CTags supports it but doesn't currently (2011) include any
    parsers that use it. It should be easy to convert to extended
    regex syntax anyway.

Method
``````
* Add foo.c to SRCS in Makefile.am.
* Add foo.o to OBJS in makefile.win32.
* Add path/foo.c to geany_sources in wscript.
* Add Foo to parsers.h
* Add TM_PARSER_FOO to tagmanager/src/tm_parser.h.  The list here must follow
  exactly the order in parsers.h.

In foo.c:
Edit FooKinds 3rd column to match a s_tag_type_names string in tm_tag.c.
(You may want to make the symbols.c change before doing this).

In filetypes.c, init_builtin_filetypes():
Set the 2nd argument of the FT_INIT() macro for this filetype to FOO.

In symbols.c:
Unless your parser uses C-like tag type kinds, update
add_top_level_items() for foo, calling tag_list_add_groups(). See
get_tag_type_iter() for which tv_iters fields to use.

Tests
`````
The tag parser tests checks if the proper tags are emitted
for a given source. Tests for tag parsers consist of two files: the
source to parse, and the expected output. Tests are run using ``make
check``.

The source to parse should be in the file ``tests/ctags/mytest.ext``,
where ``mytest`` is the name you choose for your test, and ``ext`` is an
extension recognized by Geany as the language the test file is for.
This file should contain a snippet of the language to test for.
It can be either long or short, depending on what it tests.

The expected output should be in the file ``tests/ctags/mytest.ext.tags``
(which is the same name as the source, but with ``.tags`` appended), and
should be in the format generated by ``geany -g``. This file contains
the tag information expected to be generated from the corresponding
source file.

When you have these two files, you have to list your new test along the
other ones in the ``test_source`` variable in ``tests/ctags/Makefile.am``.
Please keep this list sorted alphabetically.

GDB
---

Stop on warnings
^^^^^^^^^^^^^^^^
When a GLib or GTK warning is printed, often you want to get a
backtrace to find out what code caused them. You can do that with the
``--g-fatal-warnings`` argument, which will abort Geany on the first
warning it receives.

But for ordinary testing, you don't always want your editor to abort
just because of a warning - use::

    (gdb) b handler_log if level <= G_LOG_LEVEL_WARNING


Running with batch commands
^^^^^^^^^^^^^^^^^^^^^^^^^^^
Use::

    $ gdb src/geany -x gdb-commands

Where ``gdb-commands`` is a file with the following lines::

    set pagination off
    b handler_log if level <= G_LOG_LEVEL_WARNING
    r -v


Loading a plugin
^^^^^^^^^^^^^^^^
This is useful so you can load plugins without installing them first.
Alternatively you can use a symlink in ~/.config/geany/plugins or
$prefix/lib/geany (where $prefix is /usr/local by default).

The gdb session below was run from the toplevel Geany source directory.
Start normally with e.g. "gdb src/geany".
Type 'r' to run.
Press Ctrl-C from the gdb window to interrupt program execution.

Example::

    Program received signal SIGINT, Interrupt.
    0x00d16402 in __kernel_vsyscall ()
    (gdb) call plugin_new("./plugins/.libs/demoplugin.so")
    ** INFO: Loaded:   ./plugins/.libs/demoplugin.so (Demo)
    $1 = (Plugin *) 0x905a890
    (gdb) c
    Continuing.

    Program received signal SIGINT, Interrupt.
    0x00d16402 in __kernel_vsyscall ()
    (gdb) call plugin_free(0x905a890)
    ** INFO: Unloaded: ./plugins/.libs/demoplugin.so
    (gdb) c
    Continuing.

Building Plugins
^^^^^^^^^^^^^^^^

The geany-plugins autotools script automatically detects the
installed system Geany and builds the plugins against that.

To use plugins with a development version of Geany built with
a different prefix, the plugins will need to be compiled against
that version if the ABI has changed.

To do this you need to specify both --prefix and --with-geany-libdir
to the plugin configure.  Normally the plugin prefix is the
same as the Geany prefix to keep plugins with the version of Geany
that they are compiled against, and with-geany-libdir is the Geany
prefix/lib.

Whilst it is possible for the plugin prefix to be different to
the prefix of the libdir (which is why there are two settings),
it is probably better to keep the version of Geany and its plugins
together.