Finish doctest documentation update

[pylit.git] / rstdocs / tutorial / index.txt

.. -*- rst-mode -*-
.. 
  restindex
      crumb: Tutorial
      page-title: PyLit - Tutorial
  /restindex

How to write program documents with PyLit
=========================================

.. sectnum::
.. contents::

Hello world
-----------

We start with a classical example in Python_

.. include:: hello.py
     :literal:

save it as ``hello.py`` and convert to a `reStructured Text`_ document
with ``pylit.py``::

  #> python pylit.py hello.py
  extract written to hello.py.txt

The output file ``hello.py.txt`` looks like

.. include:: hello.py.txt
     :literal:

We can see the difference between "commented code" and "code living in a
text document".

Points to mention:

* One can start literate programming with an existing code file (and without
  knowledge of reStructured Text syntax).
  
* *Documentation* is uncommented (if it is separated from code by a
  blank line and has a recognised comment string at the start of each line).

* A double colon is added at the end of the text block. It is the
  `reStructured Text`_ marker for the following `literal block`_.
  (No marker is added, if the text block already ends with a double colon.)

* *Code* is indented to form a literal block. It will be printed using a
  monospaced font and without reStructured Text substitutions.

* PyLit adds ".txt" to the filename for the text version.

Now we can add some more documentation and a link (of course, knowledge of
`reStructured Text syntax`_ helps in this stage):

.. include:: hello_2.py.txt
     :literal:

Pretty-printed with docutils, it looks like

.. admonition:: hello_2

   .. include:: hello_2.py.txt

If we re-convert the result to code, ::

  #> python pylit.py hello_2.py.txt
  extract written to hello_2.py

we get

.. include:: hello_2.py
     :literal:

Points to mention:

* The double colon that was added in the first conversion is not stripped in
  the re-conversion. 
  
  (Generally, a round-trip should not introduce changes after the first
  cycle. This way it is ensured that the line-numbers are the same in text
  and code source.)
  
* The code block ends at the first non indented line (Precisely, at the
  first line that is not more indented than the preceding text block.)


Textblocks and Comments
-----------------------

Comment lines are only transformed to a text block, if they

* start with a matching `comment string` (whitespace counts!, the Python
  default is ``'# '``), and
* are separated from non-text lines by at least one blank [#]_ line

Otherwise, they are kept as commented code.

An example will illustrate this. The code::

  # 99bottles.py -- print the famous "99 bottles of beer" song lyrics
  
  # Introductory example to literate programming
  #
  # count down from 99 to 1
  for bottles in range(99,0,-1):
      ....

is mapped to text as::

  99bottles.py -- print the famous "99 bottles of beer" song lyrics
  
  ::

    # Introductory example to literate programming
    #
    # count down from 99 to 1
    for bottles in range(99,0,-1):
        ....

The comment in the 5th line marks the "secondary documentation" as part of
the code block.

However, ::

  # 99bottles.py -- print the famous "99 bottles of beer" song lyrics
  #
  # Introductory example to literate programming
  
  # count down from 99 to 1
  for bottles in range(99,0,-1):
      ....

is mapped to text as::

  99bottles.py -- print the famous "99 bottles of beer" song lyrics

  Introductory example to literate programming
  
  ::
  
    # count down from 99 to 1
    for bottles in range(99,0,-1):
        ....

The comment in the 2nd line is removed, as it is inside a documentation block.

.. [#] a line is considered blank, if it contains only whitespace


Non-code literal block
----------------------

How can I include a literal block that should not be in the executable code
(e.g. an example, an earlier version or variant)?

Workarounds:
  
- Use a `parsed-literal block`_ directive and mark lines containing
  "accidential" markup as `inline literals`_. E.g. ::
  
    This will be printed as literal block but not become part of the source
    code:
    
    .. parsed-literal::
    
      print "code example that should not run"
      ``result = 5 *not_bold* 2``
    
  will be typeset as

  
    This will be printed as literal block but not become part of the source
    code:
    
    .. parsed-literal::
    
      print "code example that should not run"
      ``result = 5 *not_bold* 2``


- Python session examples and doctests can use `doctest block`_ syntax:

  No double colon! Start first line of block with ``>>>``.

- Use a "code-block" directive and set the `code_block_marker` option
  or use a distinct directive for ordinary literal blocks.
  
  Drawback: such directives are not part of the core rst syntax (yet) but
  must be defined in an add-on (see `syntax highlight`_ for examples).
    



File Headers
------------

Sometimes code needs to remain on the first line(s) of the document to be
valid. The most common example is the shebang_ line that tells a POSIX shell
how to process an executable file. In Python, the magic comment specifying
the `source code encoding`_ must occure on line one or two:

.. include:: hello_with_header.py
     :literal:

Headers are converted to a comment in the text source: 

.. include:: hello_with_header.py.txt
     :literal:

Pretty-printed with docutils, it looks like

.. admonition:: hello_with_header

   .. include:: hello_with_header.py.txt

Everything before the first text block (i.e. before the first paragraph
using the matching comment string) will be hidden in HTML or PDF output.

It may come as surprise that a part of the file is not "printed".
(In the case that there is no matching comment at all, the complete code
source will become a comment, however, in this case it is not likely
the source is a literate program anyway). But there are advantages also:

* line numbers are kept during the text <--> code conversion (which would be
  impossible with a literal block marker as this needs to be at the end of
  the preceding paragraph)
* you can hide a repeating (or boring) header in a project consisting of
  many source files.

If needed for the documentation, it is possible to repeat the header in the
the first text block, e.g. using a `parsed literal block`_:

.. parsed-literal::

  #!/usr/bin/env python
  # -*- coding: iso-8859-1 -*-


Doctests
--------

Pylit supports `Python doctests`_ in a literate script. 

We add a `doctest block`_ [#]_ to our example:

.. include:: hello_with_doctest.py
   :literal:

Now try it with ::

  #>  python -c "import doctest; doctest.testfile('hello_with_doctest.py')"
  
There is no output. So everything is OK? Unfortunatly not:
``doctest.testfile`` does not find the test, as it is "hidden" in a comment.
[#]_ 

Pylit converts the source to the text version (stripping the comments),
feeds it to the doctest_ module's `Advanced API`_ (introduced in Python
2.4), and we get ::

  #> pylit --doctest hello_with_doctest.py  
  **********************************************************************
  File "hello_with_doctest.py", line 3, in 
  Failed example:
      execfile("hello_with_doctest.py")
  Expected:
      Hello world
  Got:
      Hello world.

Ah yes, we forgot the full-stop in our test. Adding it and testing again::

  #> pylit --doctest hello_with_doctest_2.py  
  0 failures in 1 tests

The printed summary will ensure us that the test actually passed.

Read more about doctests in the `literate doctests example`_.


.. [#] There is no double colon before the doctest; a doctest block is
       recognised by starting with the Python interpreter prompt ``>>>``
       instead.

.. [#] The tests will be found, if ``doctest.testfile`` is run on the text
       source, i.e.  
       ``python -c "import doctest; doctest.testfile('hello_with_doctest.py.txt')"``


Including files
---------------

PyLit does not allow the specification of a separate output file for
individual code blocks like e.g. noweb_. The "dual source" concept limits
the choice to one output file per input file. However, this can be
compensated by the use of the `include directive`_.

Let us assume that for some reason, the friendly greeting should be defined
in a separate file ``greeting.py``:

.. include:: greeting.py
     :literal:

The documentation of the calling file can include the executed file

.. include:: hello_multifile.py
     :literal:

Saved to ``hello_multifile.py.txt`` and pretty-printed with docutils, this
looks like

.. admonition:: hello_multifile

   .. include:: hello_multifile.py.txt


* you have to convert both, ``greeting.py`` and ``hello_multifile.py``.
  (Currently, pylit cannot do 'batch processing' of multiple input files.)
  

.. References:

.. _reStructured Text: 
     http://docutils.sourceforge.net/docs/user/rst/quickref.html

.. _Installation: ../download/index.html#installation

.. _shebang: http://en.wikipedia.org/wiki/Shebang_(Unix)

.. _reStructured Text syntax: 
     http://docutils.sourceforge.net/docs/ref/rst/restructuredtext.html

.. _literal block: 
     http://docutils.sourceforge.net/docs/user/rst/quickref.html#literal-blocks

.. _parsed literal block:
     http://docutils.sourceforge.net/docs/ref/rst/directives.html#parsed-literal-block
.. _noweb: http://www.eecs.harvard.edu/~nr/noweb/

.. _include directive:
     http://docutils.sourceforge.net/docs/ref/rst/directives.html#including-an-external-document-fragment

.. _source code encoding:
    http://docs.python.org/tut/node4.html#SECTION004230000000000000000

.. _doctest:
.. _Python doctests: http://docs.python.org/lib/module-doctest.html
.. _Advanced API: http://docs.python.org/lib/doctest-advanced-api.html
.. _literate doctests example: ../examples/literate-doctests/index.html

.. _parsed-literal block: 
    http://docutils.sf.net/docs/ref/rst/directives.html#parsed-literal-block
.. _doctest block: 
    http://docutils.sf.net/docs/ref/rst/restructuredtext.html#doctest-blocks
.. _line block:
.. _line blocks:
    http://docutils.sf.net/docs/ref/rst/restructuredtext.html#line-blocks
.. _inline literal:
.. _inline literals:
    http://docutils.sf.net/docs/ref/rst/restructuredtext.html#inline-literals
.. _syntax highlight: ../features/syntax-highlight.html