vargars: cleanup varargs handling

[gtk-doc.git] / TODO

The TODO list for the gtk-doc project is at Bugzilla,
the bugtracking system of the GNOME project.

Visit
 http://bugzilla.gnome.org/buglist.cgi?product=gtk-doc&bug_status=UNCONFIRMED&bug_status=NEW&bug_status=ASSIGNED&bug_status=REOPENED
to see what is allready requested, or where you can help. :-)

To put an other request on the TODO list, visit
 http://bugzilla.gnome.org/enter_bug.cgi?product=gtk-doc

Also have a look at
 http://live.gnome.org/DocumentationProject/GtkDocFuture
and join discussion about future features.


Developers can also add items here :)

= Cleanups =
== get rid of the -section.txt file ==
* section-title can be in SECTION comment already
* when scanning a header file, everything will be put to one section
  * like we do for autogenerated section.txt file (gtkdoc-scan)
* we need comment markup to override the section
  e.g.: " * InSection: xxxx"
  this needs gtkdoc-mkdb::ScanSourceFile to understand a new tag, which
  otherwise would appear as verbatim in the sources
* if one has "@InSection: xxxx" in a section comment we could patch the
  main.xml file and insert then xi:include line (would possibly rely on a
  special comment-pair there, we might also need to rewrite these
  auto-generated xi:includes everytime as the placements could have been
  changed, or we have a xi:inlcude for each chapter, that we regenerate).
* we need a way to specify subsections (Standart, Private)
  this could be done in the SECTION comment
  e.g.: "@HideSymbols: <list-of-symbols-to-hide>
  * we could allow to have a SUBSECTION:xxx comment block too
    but then we list all the symbols here to override the auto-section placement
* if we want to avoid the "InSection in symbol docs we could also have
  e.g.: "@ExtraSymbols: <list-of-symbols-to-include>
* documented symbols that are in a file without section comment and do not
  appear in "@ExtraSymbols", "@HideSymbols" would go to unused.txt
  * shall we deprecated the unused sub-sections?
* can we have both at the same time (for migration)
  * yes, read section-file first and add/override from inline comments


= More abbreviations =
* expand urls (needds more work, see gtkdoc-mkdb : ExpandAbbreviations)


= Testing =
cd test/gobject
diff -u --exclude="Makefile*" docs docs-tmpl | diffstat


= Running =
gtk-doc is using a makefile with several targets to get from sources to docs.
It might be easier to have a gtk-doc tool that can run the other gtk-doc tools
in the right order (ev. by importing them as modules). This could handle a few
things nicer that the makefiles don't do well.

= Issues =
* gtkdoc-fixxref makefile targets use $HTML_DIR
  * HTML_DIR: The directory where gtk-doc generated documentation is installed
    it comes from gtk-doc.m4 (--with-html-dir) but has no default
  * automake exports $htmldir which is by default:
    ${prefix}/share/doc/${PACKAGE_TARNAME}
  * the Makefile uses $(DESTDIR)$(TARGET_DIR)
    where TARGET_DIR = $(HTML_DIR)/$(DOC_MODULE)
    http://www.gnu.org/software/libtool/manual/automake/DESTDIR.html


= Extensibility =
We'd like to extend gtk-doc to understand conventions/features of gobject libs.
Ideally libs register their extension hooks, so that other libs that use these
libs can benefit from the extensions too.

== custom get_types collector ==
- the default method takes the types from a type file
- gstreamer plugin docs take a list of types from the gst plugin registry

== custom properties ==
- gtk has style and child properties

== extra gobject property flags ==
- gstreamer has 'controllable' properties

= Output =
* http://sagehill.net/docbookxsl/index.html
* multipage-html
  * would be good to be able to have page titles as a concatenation of document
    name and page name (gtk+:GtkWIdget)
* formats
  http://bugzilla.gnome.org/show_bug.cgi?id=531572 : html-single
  http://bugzilla.gnome.org/show_bug.cgi?id=466535 : pdf
  http://bugzilla.gnome.org/show_bug.cgi?id=467488 : man
  we need more configure options in gtk-doc.m4:
  --(enable|disable)-gtk-doc-(html|pdf|man|html-single|rtf)
  - html : enabled by default
  - html-single : is single page html
* validation
    xmllint --noout --xinclude --postvalid tester-docs.xml
    xmllint --noout --postvalid tester-docs.fo --dtdvalid file://$HOME/download/fo.dtd
    - fo.dtd : http://www.renderx.com/Tests/validator/fo.zip
* single page
  xsltproc  --nonet --xinclude -o gtk-docs.html /home/ensonic/projects/gtk-doc/gtk-doc-single.xsl  gtk-docs.sgml
  * need to check if we can pass the style-sheet class as a parameter (--stringparam gtkdoc.stylesheet=(chunk|docbook))
  * we might also need to reflow some things, as gtk-doc.xsl also runs the devhelp/devhelp2 generation
    - but then the urls in the devhelp file, refer to the chunked html anyway
* rtf
 ~/download/fop-0.94/fop -fo tester-docs.fo -rtf tester-docs.rtf
 * unrtf
   unrtf -t ps tester-docs.rtf >tester-docs.ps
   unrtf -t latex tester-docs.rtf >tester-docs.tex
   - bad output


= Warnings =
Bugzilla has some requests for extra warnings. We should support a common
commandline option(s) in all tools to enable/disable the warnings. The makefiles
should pass the flags from an env-var (GTKDOC_OPTIONS). The env-var should be
used after the regular flags, so that the env-var can override hardcoded
settings (in Makefile.am).

Lets take this warning for the example:
  "Symbol name not found at the start of the comment block."

Version 1: (template "warn-xxx!, warn-yyy!")
--warn-missing-symbol-at-comment-start
--no-warn-missing-symbol-at-comment-start

Version 2: (template "warn:s@")
-Wmissing-symbol-at-comment-start
-Wno-missing-symbol-at-comment-start
-warn missing-symbol-at-comment-start
-warn no-missing-symbol-at-comment-start

more warnings:
  - 'deprecated' deprecating 'features'
  - 'dummy-docs' check if symbol docs are very short and repeat mainly words
    from the symbol.
  - possible xrefs (e.g. adding a # or () would make it a successful xref)


= Markup =
* protected scope
  * we can have /* < protected > */ in classes
  * we can have <SUBSECTION Protected> in -section.txt
  * ideally we have Scope: {Public, Protected, Private} supported in doc comments
  * there is a bg for gir, https://bugzilla.gnome.org/show_bug.cgi?id=594125

= GIR =
== scanning ==
* ideas
  * use gir files
    1) replace gtkdoc-scan/gtkdoc-scangobject by gtkdoc-gir and output the classical files or
       patch gtkdoc-scan/gtkdoc-scangobject to output gir files
    2) patch gtkdoc-mkdb to read stuff from gir instead of classical files
  * if gir-files would have the comments too (they are getting this now):
    * we could even drop scanning the sources
    * IDEs could use the gir-files to show doc-tooltips for symbols
  * we might need yet another makefile flavour to use gir files
* perl and xml
  * http://www.xml.com/pub/a/2001/04/18/perlxmlqstart1.html

== binding docs ==
* simmillar workflow to gettext
* add gtkdoc-mk??? to generate binding doc templates
  * have c-comments there as comments
  * when updating templates, mark sections that have changed as fuzzy
* add options to gtkdoc-mkdb to build docbook from those templates
* questions
  * could we use the tmpl file mechanism?
  * directory structure?
    * we need to list the languages like ALL_LINGUAS for translations
    * we need to create subdirs for binding docs, ideally we use one for 'C' as well
* devhelp
  * devhelp files need a language attribute in the book-tag
    language={C,C++,JavaScript,Perl,Python,...}
  * devhelp could show a selector for the language
  * need to get existing python/~mm docs to use it, gtk-doc could output
    language=C for own docs
== installation ==
* need to install each book with a prefix
* would be good to have a language attribute in book tag to allow filter by language
* look at /usr/share/gtk-doc/html/pygobject/index.html


= docbook xml =
Its tedious to write large amounts of docbook.

== more macros ==
We need parametric, user definable macros.
|[ ... ]| - programlistings
|macro(arg1,arg2,...)[ ... ]| - call macro
  - pass args as parameters (on the commandline)
  - pass some gtk-doc vars in environment
    (gtk-doc version, module, srcdir, buildir)
  - content of [] on stdin or as a file
  - get output on stdout or file
  - and replace the macro with it
The changes could be made in gtkdoc-mkdb::ExpandAbbreviations()
=== example macros ===
|highlight(c)[...]| - highlight source code for a specific language (c)
  - what will this output? preformatted html to be xincluded?
  - we could have macros for each format, the docbook xml macro would leave
    enough traces in the html so that a html macro can continue
|dot(svg)[...]| - format dot graph and include result as mediaobject (in svg format)
|ditta(svg)[...]| - parse ascii art and include result as mediaobject (in svg format)
  - we need to generate a filename for the image or use anoter parameter

=== where to define macros ===
* system wide and with the package, <prefix>/share/gtk-doc/macros, $(srcdir)
* prefix for custom macros?
* we could require stdin for input and stdout for output, the wrapper for the
  actual tool can ensure the convention

== ascii doc as a frontend ==
Can we offer integration with asciidoc (http://www.methods.co.nz/asciidoc/)?
This way the master document could be written much easier. It would be cool if
we could use the asciidoc markup in source-comments also.

== extract other bits and pieces ==
=== library api ==
gtkdoc-scan could be obsoleted and gtkdoc-mkdb would build docbook fragemnts for
api docs and their indexes
=== DBUs Interfaces ===
http://hal.freedesktop.org/docs/DeviceKit/DeviceKit.html
http://cgit.freedesktop.org/DeviceKit/DeviceKit/tree/doc/dbus
=== GConf schemas ===

=== Wiki imports ===



= styling =
== source code examples==
http://bugzilla.gnome.org/show_bug.cgi?id=536928
We could also run a postprocessing script in gtkdoc-mkhtml/gtkdoc-fixxref

tools:
  source-highlight (/usr/bin/source-highlight)
  source-highlight -itests/gobject/examples/gobject.c -o$HOME/temp/gobject.html -n -t4 -sc
  source-highlight -itests/gobject/examples/gobject.c -o$HOME/temp/gobject.html -n -t4 -sc -cstyle.css --no-doc
  source-highlight -itests/gobject/examples/gobject.c -o$HOME/temp/gobject.xml -n -t4 -sc -f docbook

  highlight -itests/gobject/examples/gobject.c -o$HOME/temp/gobject.xml -l -X -f -j2

some tips about styling code listings in html
http://www.tjkdesign.com/articles/how_to_style_a_code_listing.asp

=== process docbook ===
if we highlight to docbook, we just get emphasis (bold)
=== process html ===
if we highlight to html we get colors, we need to check what tags we should process though:
<pre class="programlisting"> is used for all code boxes.
<div class="informalexample"><pre class="programlisting"> is used for examples.
problems:
* in html we don't know the language anymore
  * add another div
* with source-highlight, constants and types are not markedup.
  for types we might need to build an own lang file dynamically and include
  /usr/share/source-highlight/c.lang
=== |[ ... ]| does not allow setting the language ===
* check for vi/emacs/jedit modelines
  jedit: http://www.jedit.org/users-guide/buffer-local.html
  vim: http://vim.wikia.com/wiki/Modeline_magic
  emacs: http://www.delorie.com/gnu/docs/emacs/emacs_486.html
* allow <!-- language="C" --> comments after |[
* we need to catch those when processing the docbook and expanding the |[
* require new macro syntax


= syntax =
== wildcards in symbol names ==
Somtimes one defines a set of function and macros with very simillar purpose, e.g.
READ_INT8, READ_INT16, READ_INT32. It would be great to allow documenting a symbol
READ_INT* instead of 3 docs which are copy'n'pasted anyway. In the output we will have
all matching declarations in one source listing. Multiple wildcards are okay.


= documentation best practises #518427 =
* we'd like offer a more complete skelleton
  * structure
  * docbook markup (part/chapter structure)
* structure
  Sugested structure for api-docs.
  Idea is to have more content that api reference. It would be good to have a
  gnome-platform document in devhelp, so that we could xref that instead of
  explaining 100 times how to use pkg-config.

  * examples
    * gobject in devhelp
      * concepts / api / tools / tutorial / related tools
    * gtk in devhelp
      * overview / api / migation / tools
    * qt reference docs in qt assistant
      * classes / overview / tutorial&examples
  * recommendation
    * front-page
      * table with details (http://www.docbook.org/tdg/en/html/bookinfo.html)
        (problem: what enclosing tag)
        Logo, Module Version
        Copyright and Legalnotice
        Links
        * homepage, mailing lists, irc channel
        * repository, source releases, bugtracker
      * TOC
    * introduction - what is is about
    * concepts - explain basic ideas and terminology
    * development - how to build and run, env-vars, different platforms
    * api - classic api docs
    * tutorial & examples - integrated to keep it up-to-date and cross referenced
    * migration - how to for api changes, deprecations
    * (releated) tools - tools part of the package or recommended for development
    * indexes - api-index, depretations, new api since xxx

proposed structure in docbook xml:
<book>
  <bookinfo>
  </bookinfo>
  <preface><title>Introduction</title>
    ...
  </preface>
  <part label="I"><title>xxx Overview</title>
    <xi:include href="building.xml" />
    ...
  </part>
  <reference label="II"><title>xxx Core Reference</title>
    <xi:include href="xml/gtkmain.xml" />
    ...
  </part>
  <reference label="III"><title>xxx Object Reference</title>
    <chapter><title>Object Hierarchy</title>
      <xi:include href="xml/tree_index.sgml" />
    </chapter>
    <chapter>...
  </part>
  <index>...</index>
</book>

some things to check:
* gtk,glib: can we make a <part> for the glosary and index's (according to docbook, yes)
  should we use <appendix>? its like a chapter.
* gobject: uses a <preface> for introductions
* gobject: uses <reference> as a parent for the xi:includeed <refentry> docs


= extra link for symbols =
need options for configure:
--enable-gtk-doc-codesearch-links
--enable-gtk-doc-liveedit-links
== viewvc,cgit,... ==
- link to some online service for the code
- problem: most don't have local anchors for the symbols
- where to set the uri (in the document, like for online url)?
- what about a template URL containing a %s for the "path/file" or a special macro
  http://svn.gnome.org/viewvc/gtk-doc/trunk/tests/gobject/src/gobject.c?view=markup
  http://buzztard.svn.sourceforge.net/viewvc/buzztard/trunk/buzztard/src/lib/core/core.c?view=markup
  - unfortunately we can't link to symbols (only lines)
  - linking to files is difficult as in gtkdoc we have modules

== codesearch ==
- google (code) link : http://www.google.com/codesearch?q=g_object_unref
== live editing ==
The idea is to have an 'edit' link in an online version of the docs (build from
head development version) per doc-entry (symbols and section).
The link goes to a cgi and that gets following parameters: docmodule,symbol.
E.g. http://library.gnome.org/devel/references/edit?docmodule=glib&symbol=g_new
The cgi would need a hashmap to get from docmodule to the way to check it out
(ideally it has a recent checkout and only updates it).
problems:
- signal that this has been edited already?
- support for xi:included examples
- updating the checkout could be slow


= fix missing since docs =
cd gstreamer/gstreamer/docs/gst
gtkdoc-mkdb --module=gstreamer --source-dir=../../gst --outputsymbolswithoutsince
cd gstreamer/gstreamer/src
git bisect start
git bisect good
git bisect bad RELEASE-0_10_0
git bisect run script.sh

script:
#!/bin/sh
make ctags
grep "gst_caps_is_always_compatible" tags


= performance =
- timestamp each step
  make check >make.log
- try CFLAGS=-O0 for compiling the scanner, no need to optimize it
  CFLAGS="-O0" make check >make.log
  safes max 0.5 sec.
- xslt
  http://docbook2x.sourceforge.net/latest/doc/performance.html
  - play with xsltproc --profile --verbose --timing
    cd tests/gobject/docs/html
    time /usr/bin/xsltproc 2>xslt.log --profile --verbose --timing --path /home/ensonic/projects/gnome/gtk-doc/gtk-doc/tests/gobject/docs --nonet --xinclude --stringparam gtkdoc.bookname tester --stringparam gtkdoc.version 1.14 /home/ensonic/projects/gnome/gtk-doc/gtk-doc/gtk-doc.xsl ../tester-docs.xml
    - l10n.language is slow
      bug: https://sourceforge.net/tracker/index.php?func=detail&aid=2918673&group_id=21935&atid=373750
      see: http://www.mail-archive.com/docbook-apps@lists.oasis-open.org/msg05412.html
      - overide l10n.language
        glib/gobject
        real        user        sys
        2m15.221s   1m58.740s   0m1.456s
        >
        1m55.480s   1m44.296s   0m2.125s
      - override many template related to gentext
        real        user        sys
        0m43.327s   0m38.594s   0m4.724s
        >
        real        user        sys
        0m33.282s   0m29.266s   0m4.012s
      - removing the gentext calls for nav-bar alt tags does not help


  - try plain docbook xslt to see if maybe we have bad xslt templates in the
    customisation layer (gtk-doc.xsl)

  - we could do the xinlcude processing once and use it for both html and pdf
    time /usr/bin/xsltproc 2>../xslt4.log --path /home/ensonic/projects/gnome/gtk-doc/gtk-doc/tests/gobject/docs --nonet --xinclude --stringparam gtkdoc.bookname tester --stringparam gtkdoc.version 1.14 /home/ensonic/projects/gnome/gtk-doc/gtk-doc/gtk-doc.xsl ../tester-docs.xml
    real        user       sys
    0m4.846s    0m4.434s   0m0.147s
    0m4.842s    0m4.386s   0m0.145s


    time xmllint --nonet --xinclude ../tester-docs.xml >./tester-docs-all.xml
    real        user       sys
    0m0.596s    0m0.546s   0m0.023s

    time /usr/bin/xsltproc 2>../xslt5.log --path /home/ensonic/projects/gnome/gtk-doc/gtk-doc/tests/gobject/docs --nonet --stringparam gtkdoc.bookname tester --stringparam gtkdoc.version 1.14 /home/ensonic/projects/gnome/gtk-doc/gtk-doc/gtk-doc.xsl ./tester-docs-all.xml
    real        user       sys
    0m4.167s    0m3.834s   0m0.106s
    0m4.248s    0m3.851s   0m0.114s


    time xmllint --nonet --c14n --xinclude ../tester-docs.xml >./tester-docs-all2.xml

    real        user       sys
    0m0.700s    0m0.636s   0m0.034s

    time /usr/bin/xsltproc 2>../xslt6.log --path /home/ensonic/projects/gnome/gtk-doc/gtk-doc/tests/gobject/docs --nonet --stringparam gtkdoc.bookname tester --stringparam gtkdoc.version 1.14 /home/ensonic/projects/gnome/gtk-doc/gtk-doc/gtk-doc.xsl ./tester-docs-all2.xml

    real        user       sys
    0m3.344s    0m3.026s   0m0.109s
    0m3.372s    0m3.037s   0m0.115s


    l ../tester-docs.xml ./tester-docs-all*.xml

  - we could also try to compact the installed xslt
    xmllint --nonet --c14n --xinclude gtk-doc.xsl | sed -ne '/<!--/ { :c; /-->/! { N; b c; }; /-->/s/<!--.*-->//g }; /^  *$/!p;' | sed '/^$/d' >gtk-doc.pre.xsl
    - unfortunately there is no way to ask xsltproc to pre-transform an xslt, that could
      - strip comments
      - process xsl:import and xsl:include
  - compile xslt
    http://sourceforge.net/projects/xsltc/
    http://www.xmlhack.com/read.php?item=618
  - extra xsltproc options:
    --novalid: saves ~ 0.12 sec

- perl regexps
  - not really an issue, but we can improve by compiling the regexps
    http://stackoverflow.com/questions/550258/does-the-o-modifier-for-perl-regular-expressions-still-provide-any-benefit
  - we use $&, $', and $` in several places (comple match, pre-match and post-match).
    Those are slow and once they are used a single time perl prepares them for every match operation.
    Since perl 5.10 one can use the /p flag for matches where this is needed and then use
    ${^PREMATCH} , ${^MATCH}  and ${^POSTMATCH}