1 ================================
2 Docutils_ Internationalization
3 ================================
6 :Contact: docutils-develop@lists.sourceforge.net
9 :Copyright: This document has been placed in the public domain.
15 This document describes the internationalization facilities of the
16 Docutils_ project. `Introduction to i18n`_ by Tomohiro KUBOTA is a
17 good general reference. "Internationalization" is often abbreviated
18 as "i18n": "i" + 18 letters + "n".
22 The i18n facilities of Docutils should be considered a "first
23 draft". They work so far, but improvements are welcome.
24 Specifically, standard i18n facilities like "gettext" have yet to
27 Docutils is designed to work flexibly with text in multiple languages
28 (one language at a time). Language-specific features are (or should
29 be [#]_) fully parameterized. To enable a new language, two modules
30 have to be added to the project: one for Docutils itself (the
31 `Docutils Language Module`_) and one for the reStructuredText parser
32 (the `reStructuredText Language Module`_).
34 .. [#] If anything in Docutils is insufficiently parameterized, it
35 should be considered a bug. Please report bugs to the Docutils
36 project bug tracker on SourceForge at
37 http://sourceforge.net/tracker/?group_id=38414&atid=422030.
39 .. _Docutils: http://docutils.sourceforge.net/
40 .. _Introduction to i18n:
41 http://www.debian.org/doc/manuals/intro-i18n/
47 Language modules are named using `language tags`_ as defined in
48 `BCP 47`_. [#]_ in lowercase, converting hyphens to underscores [#]_.
50 A typical language identifier consists of a 2-letter language code
51 from `ISO 639`_ (3-letter codes can be used if no 2-letter code
52 exists). The language identifier can have an optional subtag,
53 typically for variations based on country (from `ISO 3166`_ 2-letter
54 country codes). If no language identifier is specified, the default
55 is "en" for English. Examples of module names include ``en.py``,
56 ``fr.py``, ``ja.py``, and ``pt_br.py``.
58 .. [#] BCP stands for 'Best Current Practice', and is a persistent
59 name for a series of RFCs whose numbers change as they are updated.
60 The latest RFC describing language tag syntax is RFC 5646, Tags for
61 the Identification of Languages, and it obsoletes the older RFCs
64 .. [#] Subtags are separated from primary tags by underscores instead
65 of hyphens, to conform to Python naming rules.
67 .. _language tags: http://www.w3.org/International/articles/language-tags/
68 .. _BCP 47: http://www.rfc-editor.org/rfc/bcp/bcp47.txt
69 .. _ISO 639: http://www.loc.gov/standards/iso639-2/php/English_list.php
70 .. _ISO 3166: http://www.iso.ch/iso/en/prods-services/iso3166ma/
71 02iso-3166-code-lists/index.html
77 Generally Python code in Docutils is ASCII-only. In language modules,
78 Unicode-escapes can be used for non-ASCII characters.
80 `PEP 263`_ introduces source code encodings to Python modules,
81 implemented beginning in Python 2.3. Especially for languages with
82 non-Latin scripts, using UTF-8 encoded literal Unicode strings increases the
83 readability. Start the source code file with the magic comment::
85 # -*- coding: utf-8 -*-
87 As mentioned in the note above, developers are invited to explore
88 "gettext" and other i18n technologies.
90 .. _PEP 263: http://www.python.org/peps/pep-0263.html
93 Docutils Language Module
94 ========================
96 Modules in ``docutils/languages`` contain language mappings for
97 markup-independent language-specific features of Docutils. To make a
98 new language module, just copy the ``en.py`` file, rename it with the
99 code for your language (see `Language Module Names`_ above), and
100 translate the terms as described below.
102 Each Docutils language module contains three module attributes:
105 This is a mapping of node class names to language-dependent
106 boilerplate label text. The label text is used by Writer
107 components when they encounter document tree elements whose class
108 names are the mapping keys.
110 The entry values (*not* the keys) should be translated to the
113 ``bibliographic_fields``
114 This is a mapping of language-dependent field names (converted to
115 lower case) to canonical field names (keys of
116 ``DocInfo.biblio_notes`` in ``docutils.transforms.frontmatter``).
117 It is used when transforming bibliographic fields.
119 The keys should be translated to the target language.
121 ``author_separators``
122 This is a list of strings used to parse the 'Authors'
123 bibliographic field. They separate individual authors' names, and
124 are tried in order (i.e., earlier items take priority, and the
125 first item that matches wins). The English-language module
126 defines them as ``[';', ',']``; semi-colons can be used to
127 separate names like "Arthur Pewtie, Esq.".
129 Most languages won't have to "translate" this list.
132 reStructuredText Language Module
133 ================================
135 Modules in ``docutils/parsers/rst/languages`` contain language
136 mappings for language-specific features of the reStructuredText
137 parser. To make a new language module, just copy the ``en.py`` file,
138 rename it with the code for your language (see `Language Module
139 Names`_ above), and translate the terms as described below.
141 Each reStructuredText language module contains two module attributes:
144 This is a mapping from language-dependent directive names to
145 canonical directive names. The canonical directive names are
146 registered in ``docutils/parsers/rst/directives/__init__.py``, in
147 ``_directive_registry``.
149 The keys should be translated to the target language. Synonyms
150 (multiple keys with the same values) are allowed; this is useful
154 This is a mapping language-dependent role names to canonical role
155 names for interpreted text. The canonical directive names are
156 registered in ``docutils/parsers/rst/states.py``, in
157 ``Inliner._interpreted_roles`` (this may change).
159 The keys should be translated to the target language. Synonyms
160 (multiple keys with the same values) are allowed; this is useful
164 Testing the Language Modules
165 ============================
167 Whenever a new language module is added or an existing one modified,
168 the unit tests should be run. The test modules can be found in the
169 docutils/test directory from CVS_ or from the `latest CVS snapshot`_.
171 The ``test_language.py`` module can be run as a script. With no
172 arguments, it will test all language modules. With one or more
173 language codes, it will test just those languages. For example::
175 $ python test_language.py en
177 ----------------------------------------
178 Ran 2 tests in 0.095s
182 Use the "alltests.py" script to run all test modules, exhaustively
183 testing the parser and other parts of the Docutils system.
185 .. _CVS: http://sourceforge.net/cvs/?group_id=38414
186 .. _latest CVS snapshot: http://docutils.sf.net/docutils-snapshot.tgz
189 Submitting the Language Modules
190 ===============================
192 If you do not have repository write access and want to contribute your
193 language modules, feel free to submit them via the `SourceForge patch
196 __ http://sourceforge.net/tracker/?group_id=38414&atid=422032