pipadoc

   1 #!/bin/sh
   2 #
   3 #TIT pipadoc - Documentation extractor
   4 #TIT =================================
   5 #TIT Christian Thaeter <ct@pipapo.org>
   6 #TIT
   7 #LIC Copyright (C)              Pipapo Project
   8 #LIC  2009,                     Christian Thaeter <ct@pipapo.org>
   9 #LIC
  10 #LIC This program is free software; you can redistribute it and/or modify
  11 #LIC it under the terms of the GNU General Public License as published by
  12 #LIC the Free Software Foundation; either version 2, or (at your option)
  13 #LIC any later version.
  14 #LIC
  15 #LIC This program is distributed in the hope that it will be useful,
  16 #LIC but WITHOUT ANY WARRANTY; without even the implied warranty of
  17 #LIC MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
  18 #LIC GNU General Public License for more details.
  19 #LIC
  20 #LIC You should have received a copy of the GNU General Public License
  21 #LIC along with this program; if not, contact Christian Thaeter <ct@pipapo.org>.
  22 #LIC
  23
  24 #INT Embedding documentation in program source files often yields the problem that the
  25 #INT structure of a program is most often not the optimal structure for the associated documentation.
  26 #INT Still there are good reasons to maintain documentation together with the source right at the code
  27 #INT which defines the documented functionality. Pipadoc addresses this problem by extracting
  28 #INT special comments out of a source file and let one define rules how to bring the
  29 #INT documentation into proper order.
  30 #INT
  31 #INT Pipadoc only extracts and reorders the text from it special comments, it never ever looks at the
  32 #INT sourcecode or the text it extracts.
  33 #INT
  34 #INT This is somewhat similar to ``Literate Programming'' but it puts the emphasis back to the code.
  35 #INT There is no need to extract the source from a literate source and in contrast to ``Literate Programming''
  36 #INT the order of source and text is defined by the programmer and programming language constraints.
  37 #INT
  38 #INT Pipadoc is programming language and documentation system agnostic, all it requires is that
  39 #INT the programming language has some line comments or block comments where one places doc statements
  40 #INT on each block line (see xref:c-example[Example for C]).
  41 #INT
  42 #BAS [[BAS]]
  43 #BAS Basic concepts
  44 #BAS --------------
  45 #BAS
  46 #BAS NOTE: The following description uses the xref:ENV[default] settings for all examples.
  47 #BAS
  48 #BAS Pipadoc is controlled by special line comments.
  49 #BAS
  50 #DIR [[DIR]]
  51 #DIR .Direct comments
  52 #DIR Line comments immediately followed by a special documentation character (the underscore `_` by default)
  53 #DIR are treated as direct comments. They will appear in order of appearance in the generated output.
  54 #DIR These can be used to do some boilerplate stuff. Usually one wants to define a controlling document and
  55 #DIR use this direct comments only there, since using them in different files might yield unexpected results
  56 #DIR since the order then depends on the load order of the files.
  57 #DIR ----
  58 #DIR //_ This is a direct comment,
  59 #DIR //_ it will appear just verbatim in the generated output
  60 #DIR ----
  61 #DIR
  62 #KEY [[KEY]]
  63 #KEY .Keys
  64 #KEY A line comment immediately followed by a alphanumeric keyword (including the `_` underscore character)
  65 #KEY is treated as key, all such keyed comments can later be placed in intended order with
  66 #KEY a xref:SUB[substitution] comment.
  67 #KEY ----
  68 #KEY //example This text will later be inserted where one uses the `//=example` substitution.
  69 #KEY //example All example lines are appended there in order even if they are defined at different
  70 #KEY //example places or in different files
  71 #KEY ----
  72 #KEY
  73 #KEY .Sorted Keys
  74 #KEY A key can be appended with a dot `.` and a non-space string. This string will then be used to sort
  75 #KEY these lines alphabetically. This can be used to create sorted indices and glossars, as well as reordering
  76 #KEY paragraphs in stored under one primary key.
  77 #KEY ----
  78 #KEY //example.omega This is sorted after the next line
  79 #KEY //example.alpha comes before the omega line above
  80 #KEY ----
  81 #KEY
  82 #SUB [[SUB]]
  83 #SUB .Substitutions
  84 #SUB A line comment immediately followed by a special `substitution` character (the equal `=` sign by default)
  85 #SUB followed by a xref:KEY[key] will be replaced by the text defined under that key. The rest of the line
  86 #SUB is ignored and can be used as comment.
  87 #SUB ----
  88 #SUB //=example this will insert anything defined under `//example` here
  89 #SUB ----
  90 #SUB
  91
  92 #USE [[USE]]
  93 #USE Documenting Files
  94 #USE -----------------
  95 #USE
  96 #USE Usually one wants to write documentation in more or less smaller blocks which later shall be
  97 #USE brought into proper order. The xref:SUB[substitutions] feature is the key for this. One writes
  98 #USE his documentation blocks with comments which later get replaced by the right sorting key and
  99 #USE finally brought into (alphabetical) stable-sort order. You might take a look at the pipadoc
 100 #USE source itself to see it in action.
 101 #USE
 102
 103 #ENV [[ENV]]
 104 #ENV Environment Variables
 105 #ENV ---------------------
 106 #ENV
 107 #ENV `COM`::
 108 #ENV   Defines the line-comment character used in the source file.
 109 #ENV   Defaults to `//` (C++/C99) if not set. Set this to `#` for shell
 110 #ENV   like languages.
 111 [[ "$COM" ]] || COM='//'
 112 # [[ "$COM" ]] || COM='#'
 113 #ENV `DOC`::
 114 #ENV   The Documentation character which must follow a line comment to be recognized
 115 #ENV   by pipadoc as documentation. Either one for local definitions or two for global
 116 #ENV   definitions are used. Defaults to `_` and needs rarely to be changed.
 117 [[ "$DOC" ]] || DOC='_'
 118 #ENV `SUB`::
 119 #ENV   Substitution character. Defaults to `=` and rarely needs to be changed.
 120 #ENV   See xref:SUB[substitutions] for details.
 121 #ENV
 122 [[ "$SUB" ]] || SUB='='
 123 #ENV [[ENVSXT]]
 124 #ENV `SXT`::
 125 #ENV   Section eXTention. Defaults to `.txt`.
 126 #ENV   See xref:SXT[plaintext files] for details.
 127 #ENV
 128 [[ "$SXT" ]] || SXT='.txt'
 129
 130 #SXT [[SXT]]
 131 #SXT Documentation Only Files
 132 #SXT ------------------------
 133 #SXT
 134 #SXT One can write documentation without the need of pipadoc special comments in files
 135 #SXT which have a configured extension (see the xref:ENVSXT[SXT] environment variable).
 136 #SXT Each line in such a file which does not have a pipadoc special comment is then implicitly
 137 #SXT prepended with the line comment sequence and the basename of that file, for example
 138 #SXT lines in a file 'foo.txt' will be treated as if they where written with `//foo ` in front of
 139 #SXT them. Later xref:SUB[substitutions] can be used to organize the document.
 140 #SXT When such a file has an ordinary pipadoc special comment line then this takes precedence over
 141 #SXT the implicit commenting.
 142 #SXT
 143
 144 #INV [[INV]]
 145 #INV Invocation
 146 #INV ----------
 147 #INV
 148 #INV Pipadoc is called with a list of files from which the documentation shall be extracted.
 149 #INV The extracted documentation is piped to stdout. No other command line options are available.
 150 #INV
 151 #INV There are few xref:ENV[environment variables] to do basic configuration.
 152 #INV
 153
 154 # here we go
 155 awk -f /dev/fd/3 ${COM:+-v com="$COM"} ${DOC:+-v doc="$DOC"} ${SUB:+-v subs="$SUB"} ${SXT:+-v sxt="$SXT"} "$@" 3<<"EOF"
 156
 157 # Plaintext file handling
 158 FILENAME ~ sxt "$" && $0 !~ "^" com {
 159   match(FILENAME, "/?([^/]*)" sxt "$", p)
 160   $0 = com p[1] " " $0
 161 }
 162
 163 # Substitution
 164 match($0, com subs "([[:alpha:]][[:alnum:]_]*)", s) {
 165   ++n
 166   subst[n] = s[1]
 167   next
 168 }
 169
 170 # doc comment
 171 match($0, com doc "([[:space:]](.*))?$", s) {
 172   ++n
 173   output[n] = s[2]
 174   next
 175 }
 176
 177 # record all other comments which may be candidate comments
 178 match($0, "(.*)" com "([[:alpha:]][[:alnum:]_]*)(([.]([^[:space:]]*)))?([[:space:]](.*))?", m) && m[1] !~ com {
 179   if (m[2] in maybe)
 180     maybe[m[2]] = maybe[m[2]] "\n.." m[5] " " m[7]
 181   else
 182     maybe[m[2]] = "." m[5] " " m[7]
 183   next
 184 }
 185
 186 # finally output
 187 END {
 188   for (i=1; i<=n; ++i)
 189     {
 190       if (i in output)
 191         print output[i]
 192       else
 193         {
 194           nelements = split(maybe[subst[i]], s, "\n[.]")
 195           split("", tosort)
 196
 197           for (j=1; j <=nelements; ++j)
 198             {
 199               match(s[j], ".([^[:space:]]*) (.*)", entries)
 200
 201               if ("." entries[1] in tosort)
 202                 tosort["." entries[1]] = tosort["." entries[1]] "\n" entries[2]
 203               else
 204                 tosort["." entries[1]] = entries[2]
 205             }
 206
 207           elements = asorti(tosort, sorted)
 208           for (k = 1; k <= elements; ++k)
 209             {
 210               print tosort[sorted[k]]
 211             }
 212         }
 213     }
 214 }
 215 EOF
 216
 217 #Document structure:
 218 #=TIT Titles and stuff
 219 #=INT Introduction
 220 #=BAS Basics
 221 #=DIR Direct comments
 222 #=KEY Key comments
 223 #=SUB Substitutions
 224 #=INV Invocation
 225 #=USE Real usage
 226 #=SXT Plain Documentation
 227 #=ENV Environment variables
 228 #_ Appendix
 229 #_ --------
 230 #=APP Appendix
 231 #_ License
 232 #_ -------
 233 #_ [[LIC]]
 234 #_ ....
 235 #=LIC
 236 #_ ....
 237
 238 # Examples and appendixes:
 239 #USE .A small C99 Program
 240 #USE [source,c]
 241 #USE ----
 242 #USE //intro This is the well known ``Hello World'' example
 243 #USE //glos.helloworld A common program to show examples for programming language and tools
 244 #USE
 245 #USE //depends Only the Standard C library is needed
 246 #USE #include <stdio.h>
 247 #USE
 248 #USE int main(int argc, char* argv[])
 249 #USE {
 250 #USE   //hello print the first commandline argument
 251 #USE   //glos.argument the text you pass to the programm when calling it from the shell
 252 #USE   //hello if no argument is given, then exit silently
 253 #USE   if (argc > 1)
 254 #USE     printf("Hello %s\n", argv[1]);
 255 #USE   return 0;
 256 #USE }
 257 #USE
 258 #USE // Now the document structure with substitutions:
 259 #USE //_ Yet another 'Hello' program
 260 #USE //_
 261 #USE //=intro introduction first right after the title
 262 #USE //intro
 263 #USE //=hello The main documentation
 264 #USE //_
 265 #USE //_ Appendix
 266 #USE //_ Dependencies:
 267 #USE //=depends
 268 #USE //_ Glossary
 269 #USE //=glos glossary will be sorted
 270 #USE ----
 271 #USE
 272 #USE Runnning this through pipadoc gives following output:
 273 #USE ----
 274 #USE Yet another 'Hello' program
 275 #USE
 276 #USE This is the well known ``Hello World'' example
 277 #USE
 278 #USE print the first commandline argument
 279 #USE if no argument is given, then exit silently
 280 #USE
 281 #USE Appendix
 282 #USE Dependencies:
 283 #USE Only the Standard C library is needed
 284 #USE Glossary
 285 #USE the text you pass to the programm when calling it from the shell
 286 #USE A common program to show examples for programming language and tools
 287 #USE ----
 288 #USE
 289
 290 #APP [[c-example]]
 291 #APP
 292 #APP .Using C block comments with pipadoc: +example.c+
 293 #APP ----
 294 #APP /*
 295 #APP //_ this is a documentation line
 296 #APP */
 297 #APP ----
 298 #APP use `pipadoc example.c` to process the documentation.
 299 #APP