pipadoc

   1 #!/bin/sh
   2 #
   3 #TIT pipadoc - Documentation extractor
   4 #TIT =================================
   5 #TIT Christian Thaeter <ct@pipapo.org>
   6 #TIT
   7 #LIC Copyright (C)              Pipapo Project
   8 #LIC  2009,                     Christian Thaeter <ct@pipapo.org>
   9 #LIC
  10 #LIC This program is free software; you can redistribute it and/or modify
  11 #LIC it under the terms of the GNU General Public License as published by
  12 #LIC the Free Software Foundation; either version 2, or (at your option)
  13 #LIC any later version.
  14 #LIC
  15 #LIC This program is distributed in the hope that it will be useful,
  16 #LIC but WITHOUT ANY WARRANTY; without even the implied warranty of
  17 #LIC MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  18 #LIC GNU General Public License for more details.
  19 #LIC
  20 #LIC You should have received a copy of the GNU General Public License
  21 #LIC along with this program; if not, contact Christian Thaeter <ct@pipapo.org>.
  22 #LIC
  23
  24 #INT Embedding documentation in program source files often yields the problem that the
  25 #INT structure of a program is most often not the optimal structure for the associated documentation.
  26 #INT Still there are good reasons to maintain documentation together with the source right at the code
  27 #INT which defines the documented functionality. Pipadoc addresses this problem by extracting
  28 #INT special comments out of a source file and let one define rules how to bring the
  29 #INT documentation into proper order.
  30 #INT
  31 #INT Pipadoc only extracts and reorders the text from it special comments, it never ever looks at the
  32 #INT sourcecode or the text it extracts.
  33 #INT
  34 #INT This is somewhat similar to ``Literate Programming'' but it puts the emphasis back to the code.
  35 #INT There is no need to extract the source from a literate source and in contrast to ``Literate Programming''
  36 #INT the order of source and text is defined by the programmer and programming language constraints.
  37 #INT
  38 #INT Pipadoc is programming language and documentation system agnostic, all it requires is that
  39 #INT the programming language has some line comments or block comments where one places doc statements
  40 #INT on each block line (see xref:c-example[Example for C]).
  41 #INT
  42 #BAS [[BAS]]
  43 #BAS Basic concepts
  44 #BAS --------------
  45 #BAS
  46 #BAS NOTE: The following description uses the xref:ENV[default] settings for all examples.
  47 #BAS
  48 #BAS Pipadoc is controlled by special line comments.
  49 #BAS
  50 #DIR [[DIR]]
  51 #DIR .Direct comments
  52 #DIR Line comments immediately followed by a special documentation character (the underscore `_` by default)
  53 #DIR are treated as direct comments. They will appear in order of appearance in the generated output.
  54 #DIR These can be used to do some boilerplate stuff. Usually one wants to define a controlling document and
  55 #DIR use this direct comments only there, since using them in different files might yield unexpected results
  56 #DIR since the order then depends on the load order of the files.
  57 #DIR ----
  58 #DIR //_ This is a direct comment,
  59 #DIR //_ it will appear just verbatim in the generated output
  60 #DIR ----
  61 #DIR
  62 #KEY [[KEY]]
  63 #KEY .Keys
  64 #KEY A line comment immediately followed by a alphanumeric keyword (including the `_` underscore character)
  65 #KEY is treated as key, all such keyed comments can later be placed in intended order with
  66 #KEY a xref:SUB[substitution] comment.
  67 #KEY ----
  68 #KEY //example This text will later be inserted where one uses the `//=example` substitution.
  69 #KEY //example All example lines are appended there in order even if they are defined at different
  70 #KEY //example places or in different files
  71 #KEY ----
  72 #KEY
  73 #KEY .Sorted Keys
  74 #KEY A key can be appended with a dot `.` and a non-space string. This string will then be used to sort
  75 #KEY these lines alphabetically. This can be used to create sorted indices and glossars, as well as reordering
  76 #KEY paragraphs in stored under one primary key.
  77 #KEY ----
  78 #KEY //example.omega This is sorted after the next line
  79 #KEY //example.alpha comes before the omega line above
  80 #KEY ----
  81 #KEY
  82 #SUB [[SUB]]
  83 #SUB .Substitutions
  84 #SUB A line comment immediately followed by a special `substitution` character (the equal `=` sign by default)
  85 #SUB followed by a xref:KEY[key] will be replaced by the text defined under that key. The rest of the line
  86 #SUB is ignored and can be used as comment.
  87 #SUB ----
  88 #SUB //=example this will insert anything defined under `//example` here
  89 #SUB ----
  90 #SUB
  91
  92 #USE [[USE]]
  93 #USE Documenting Files
  94 #USE -----------------
  95 #USE
  96 #USE Usually one wants to write documentation in more or less smaller blocks which later shall be
  97 #USE brought into proper order. The xref:SUB[substitutions] feature is the key for this. One writes
  98 #USE his documentation blocks with comments which later get replaced by the right sorting key and
  99 #USE finally brought into (alphabetical) stable-sort order. You might take a look at the pipadoc
 100 #USE source itself to see it in action.
 101 #USE
 102
 103 #ENV [[ENV]]
 104 #ENV Environment Variables
 105 #ENV ---------------------
 106 #ENV
 107 #ENV `COM`::
 108 #ENV   Defines the line-comment character used in the source file.
 109 #ENV   Defaults to `//` (C++/C99) if not set. Set this to `#` for shell
 110 #ENV   like languages.
 111 test "$COM" || COM='//'
 112 #ENV `DOC`::
 113 #ENV   The Documentation character which must follow a line comment to be recognized
 114 #ENV   by pipadoc as documentation. Either one for local definitions or two for global
 115 #ENV   definitions are used. Defaults to `_` and needs rarely to be changed.
 116 test "$DOC" || DOC='_'
 117 #ENV `SUB`::
 118 #ENV   Substitution character. Defaults to `=` and rarely needs to be changed.
 119 #ENV   See xref:SUB[substitutions] for details.
 120 #ENV
 121 test "$SUB" || SUB='='
 122 #ENV [[ENVSXT]]
 123 #ENV `SXT`::
 124 #ENV   Section eXTention. Defaults to `.txt`.
 125 #ENV   See xref:SXT[plaintext files] for details.
 126 #ENV
 127 test "$SXT" || SXT='.txt'
 128
 129 #SXT [[SXT]]
 130 #SXT Documentation Only Files
 131 #SXT ------------------------
 132 #SXT
 133 #SXT One can write documentation without the need of pipadoc special comments in files
 134 #SXT which have a configured extension (see the xref:ENVSXT[SXT] environment variable).
 135 #SXT Each line in such a file which does not have a pipadoc special comment is then implicitly
 136 #SXT prepended with the line comment sequence and the basename of that file, for example
 137 #SXT lines in a file 'foo.txt' will be treated as if they where written with `//foo ` in front of
 138 #SXT them. Later xref:SUB[substitutions] can be used to organize the document.
 139 #SXT When such a file has an ordinary pipadoc special comment line then this takes precedence over
 140 #SXT the implicit commenting.
 141 #SXT
 142
 143 #INV [[INV]]
 144 #INV Invocation
 145 #INV ----------
 146 #INV
 147 #INV Pipadoc is called with a list of files from which the documentation shall be extracted.
 148 #INV The extracted documentation is piped to stdout. No other command line options are available.
 149 #INV
 150 #INV There are few xref:ENV[environment variables] to do basic configuration.
 151 #INV
 152
 153 # here we go
 154 awk -f /dev/fd/3 ${COM:+-v com="$COM"} ${DOC:+-v doc="$DOC"} ${SUB:+-v subs="$SUB"} ${SXT:+-v sxt="$SXT"} "$@" 3<<"EOF"
 155
 156 # Plaintext file handling
 157 FILENAME ~ sxt "$" && $0 !~ "^" com {
 158   match(FILENAME, "/?([^/]*)" sxt "$", p)
 159   $0 = com p[1] " " $0
 160 }
 161
 162 # Substitution
 163 match($0, com subs "([[:alpha:]][[:alnum:]_]*)", s) {
 164   ++n
 165   subst[n] = s[1]
 166   next
 167 }
 168
 169 # doc comment
 170 match($0, com doc "([[:space:]](.*))?$", s) {
 171   ++n
 172   output[n] = s[2]
 173   next
 174 }
 175
 176 # record all other comments which may be candidate comments
 177 match($0, "(.*)" com "([[:alpha:]][[:alnum:]_]*)(([.]([^[:space:]]*)))?([[:space:]](.*))?", m) && m[1] !~ com {
 178   if (m[2] in maybe)
 179     maybe[m[2]] = maybe[m[2]] "\n.." m[5] " " m[7]
 180   else
 181     maybe[m[2]] = "." m[5] " " m[7]
 182   next
 183 }
 184
 185 # finally output
 186 END {
 187   for (i=1; i<=n; ++i)
 188     {
 189       if (i in output)
 190         print output[i]
 191       else
 192         {
 193           nelements = split(maybe[subst[i]], s, "\n[.]")
 194           split("", tosort)
 195
 196           for (j=1; j <=nelements; ++j)
 197             {
 198               match(s[j], ".([^[:space:]]*) (.*)", entries)
 199
 200               if ("." entries[1] in tosort)
 201                 tosort["." entries[1]] = tosort["." entries[1]] "\n" entries[2]
 202               else
 203                 tosort["." entries[1]] = entries[2]
 204             }
 205
 206           elements = asorti(tosort, sorted)
 207           for (k = 1; k <= elements; ++k)
 208             {
 209               print tosort[sorted[k]]
 210             }
 211         }
 212     }
 213 }
 214 EOF
 215
 216 #Document structure:
 217 #=TIT Titles and stuff
 218 #=INT Introduction
 219 #=BAS Basics
 220 #=DIR Direct comments
 221 #=KEY Key comments
 222 #=SUB Substitutions
 223 #=INV Invocation
 224 #=USE Real usage
 225 #=SXT Plain Documentation
 226 #=ENV Environment variables
 227 #_ Appendix
 228 #_ --------
 229 #=APP Appendix
 230 #_ License
 231 #_ -------
 232 #_ [[LIC]]
 233 #_ ....
 234 #=LIC
 235 #_ ....
 236
 237 # Examples and appendixes:
 238 #USE .A small C99 Program
 239 #USE [source,c]
 240 #USE ----
 241 #USE //intro This is the well known ``Hello World'' example
 242 #USE //glos.helloworld A common program to show examples for programming language and tools
 243 #USE
 244 #USE //depends Only the Standard C library is needed
 245 #USE #include <stdio.h>
 246 #USE
 247 #USE int main(int argc, char* argv[])
 248 #USE {
 249 #USE   //hello print the first commandline argument
 250 #USE   //glos.argument the text you pass to the programm when calling it from the shell
 251 #USE   //hello if no argument is given, then exit silently
 252 #USE   if (argc > 1)
 253 #USE     printf("Hello %s\n", argv[1]);
 254 #USE   return 0;
 255 #USE }
 256 #USE
 257 #USE // Now the document structure with substitutions:
 258 #USE //_ Yet another 'Hello' program
 259 #USE //_
 260 #USE //=intro introduction first right after the title
 261 #USE //intro
 262 #USE //=hello The main documentation
 263 #USE //_
 264 #USE //_ Appendix
 265 #USE //_ Dependencies:
 266 #USE //=depends
 267 #USE //_ Glossary
 268 #USE //=glos glossary will be sorted
 269 #USE ----
 270 #USE
 271 #USE Runnning this through pipadoc gives following output:
 272 #USE ----
 273 #USE Yet another 'Hello' program
 274 #USE
 275 #USE This is the well known ``Hello World'' example
 276 #USE
 277 #USE print the first commandline argument
 278 #USE if no argument is given, then exit silently
 279 #USE
 280 #USE Appendix
 281 #USE Dependencies:
 282 #USE Only the Standard C library is needed
 283 #USE Glossary
 284 #USE the text you pass to the programm when calling it from the shell
 285 #USE A common program to show examples for programming language and tools
 286 #USE ----
 287 #USE
 288
 289 #APP [[c-example]]
 290 #APP
 291 #APP .Using C block comments with pipadoc: +example.c+
 292 #APP ----
 293 #APP /*
 294 #APP //_ this is a documentation line
 295 #APP */
 296 #APP ----
 297 #APP use `pipadoc example.c` to process the documentation.
 298 #APP