manual/luatex-modifications.tex

   1 % language=uk
   2
   3 \environment luatex-style
   4 \environment luatex-logos
   5
   6 \startcomponent luatex-modifications
   7
   8 \startchapter[reference=modifications,title={Modifications}]
   9
  10 \startsection[title=The merged engines]
  11
  12 \startsubsection[title=The need for change]
  13
  14 The first version of \LUATEX\ only had a few extra primitives and it was largely
  15 the same as \PDFTEX. Then we merged substantial parts of \ALEPH\ into the code
  16 and got more primitives. When we got more stable the decision was made to clean
  17 up the rather hybrid nature of the program. This means that some primitives have
  18 been promoted to core primitives, often with a different name, and that others
  19 were removed. This made it possible to start cleaning up the code base. We will
  20 describe most in following paragraphs.
  21
  22 Besides the expected changes caused by new functionality, there are a number of
  23 not|-|so|-|expected changes. These are sometimes a side|-|effect of a new
  24 (conflicting) feature, or, more often than not, a change neccessary to clean up
  25 the internal interfaces. These will also be mentioned.
  26
  27 \stopsubsection
  28
  29 \startsubsection[title=Changes from \TEX\ 3.1415926]
  30
  31 Of course it all starts with traditional \TEX. Even if we started with \PDFTEX,
  32 most still comes from the original. But we divert a bit.
  33
  34 \startitemize
  35
  36 \startitem
  37     The current code base is written in \CCODE, not \PASCAL. We use \CWEB\
  38     when possible.
  39 \stopitem
  40
  41 \startitem
  42     See \in {chapter} [languages] for many small changes related to paragraph
  43     building, language handling and hyphenation. The most important change is
  44     that adding a brace group in the middle of a word (like in \type {of{}fice})
  45     does not prevent ligature creation.
  46 \stopitem
  47
  48 \startitem
  49     There is no pool file, all strings are embedded during compilation.
  50 \stopitem
  51
  52 \startitem
  53     The specifier \type {plus 1 fillll} does not generate an error. The extra
  54     \quote{l} is simply typeset.
  55 \stopitem
  56
  57 \startitem
  58     The upper limit to \type {\endlinechar} and \type {\newlinechar} is 127.
  59 \stopitem
  60
  61 \startitem
  62     The hz optimization code has been partially redone so that we no longer need
  63     to create extra font instances. The front- and backend have been decoupled and
  64     more efficient (\PDF) code is generated.
  65 \stopitem
  66
  67 \stopitemize
  68
  69 \stopsubsection
  70
  71 \startsubsection[title=Changes from \ETEX\ 2.2]
  72
  73 Being the de factor standard extension of course we provide the \ETEX\
  74 functionality, but with a few small adaptations.
  75
  76 \startitemize
  77
  78 \startitem
  79     The \ETEX\ functionality is always present and enabled so the prepended
  80     asterisk or \type {-etex} switch for \INITEX\ is not needed.
  81 \stopitem
  82
  83 \startitem
  84     The \TEXXET\ extension is not present, so the primitives \type
  85     {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type
  86     {\endL} are missing.
  87 \stopitem
  88
  89 \startitem
  90     Some of the tracing information that is output by \ETEX's \type
  91     {\tracingassigns} and \type {\tracingrestores} is not there.
  92 \stopitem
  93
  94 \startitem
  95     Register management in \LUATEX\ uses the \ALEPH\ model, so the maximum value
  96     is 65535 and the implementation uses a flat array instead of the mixed
  97     flat|\&|sparse model from \ETEX.
  98 \stopitem
  99
 100 \startitem
 101     When kpathsea is used to find files, \LUATEX\ uses the \type {ofm} file
 102     format to search for font metrics. In turn, this means that \LUATEX\ looks at
 103     the \type {OFMFONTS} configuration variable (like \OMEGA\ and \ALEPH) instead
 104     of \type {TFMFONTS} (like \TEX\ and \PDFTEX). Likewise for virtual fonts
 105     (\LUATEX\ uses the variable \type {OVFFONTS} instead of \type {VFFONTS}).
 106 \stopitem
 107
 108 \stopitemize
 109
 110 \stopsubsection
 111
 112 \startsubsection[title=Changes from \PDFTEX\ 1.40]
 113
 114 Because we want to produce \PDF\ the most natural starting point was the popular
 115 \PDFTEX\ program. We inherit the stable features, dropped most of the
 116 experimental code and promoted some functionality to core \LUATEX\ functionality
 117 which in turn triggered renaming primitives.
 118
 119 \startitemize
 120
 121 \startitem
 122     The (experimental) support for snap nodes has been removed, because it is
 123     much more natural to build this functionality on top of node processing and
 124     attributes. The associated primitives that are now gone are: \type
 125     {\pdfsnaprefpoint}, \type {\pdfsnapy}, and \type {\pdfsnapycomp}.
 126 \stopitem
 127
 128 \startitem
 129     The (experimental) support for specialized spacing around nodes has also been
 130     removed. The associated primitives that are now gone are: \type
 131     {\pdfadjustinterwordglue}, \type {\pdfprependkern}, and \type {\pdfappendkern}, as
 132     well as the five supporting primitives \type {\knbscode}, \type {\stbscode}, \type
 133     {\shbscode}, \type {\knbccode}, and \type {\knaccode}.
 134 \stopitem
 135
 136 \startitem
 137     A number of \quote {\PDFTEX\ primitives} have been removed as they can be
 138     implemented using \LUA:
 139
 140     \start \raggedright
 141     \type {\pdfelapsedtime}, \type {\pdfescapehex}, \type {\pdfescapename}, \type
 142     {\pdfescapestring}, \type {\pdffiledump}, \type {\pdffilemoddate}, \type
 143     {\pdffilesize}, \type {\pdfforcepagebox}, \type {\pdflastmatch}, \type
 144     {\pdfmatch}, \type {\pdfmdfivesum}, \type {\pdfmovechars}, \type
 145     {\pdfoptionalwaysusepdfpagebox}, \type {\pdfoptionpdfinclusionerrorlevel},
 146     \type {\pdfresettimer}, \type {\pdfshellescape}, \type {\pdfstrcmp} and \type
 147     {\pdfunescapehex}
 148     \par \stop
 149 \stopitem
 150
 151 \startitem
 152     The version related primitives \type {\pdftexbanner}, \type {\pdftexversion}
 153     and \type {\pdftexrevision} are no longer present as there is no longer a
 154     strict relationship with \PDFTEX\ development.
 155 \stopitem
 156
 157 \startitem
 158     The experimental snapper mechanism has been removed and therefore also the
 159     primitives:
 160
 161     \start \raggedright
 162     \type {\pdfignoreddimen}, \type {\pdffirstlineheight}, \type
 163     {\pdfeachlineheight}, \type {\pdfeachlinedepth} and \type
 164     {\pdflastlinedepth}
 165     \par \stop
 166 \stopitem
 167
 168 \startitem
 169     The experimental primitives \type {\primitive}, \type {\ifprimitive}, \type
 170     {\ifabsnum} and \type {\ifabsdim} are promoted to core primitives. The \type
 171     {\pdf*} prefixed originals are not available.
 172 \stopitem
 173
 174 \startitem
 175     The \PNG\ transparency fix from 1.40.6 is not applied as high|-|level
 176     support is pending.
 177 \stopitem
 178
 179 \startitem
 180     Two extra token lists are provides, \type {\pdfxformresources} and \type
 181     {\pdfxformattr}, as an alternative to \type {\pdfxform} keywords.
 182 \stopitem
 183
 184 \startitem
 185     The current version of \LUATEX\ no longer replaces and|/|or merges fonts in
 186     embedded \PDF\ files with fonts of the enveloping \PDF\ document. This
 187     regression may be temporary, depending on how the rewritten font backend will
 188     look like.
 189 \stopitem
 190
 191 \startitem
 192     The primitives \type {\pdfpagewidth} and \type {\pdfpageheight} have been removed
 193     because \type {\pagewidth} and \type {\pageheight} have that purpose.
 194 \stopitem
 195
 196 \startitem
 197     The primitives \type {\pdfnormaldeviate}, \type {\pdfuniformdeviate}, \type
 198     {\pdfsetrandomseed} and \type {\pdfrandomseed} have been promoted to core
 199     primitives without \type {pdf} prefix so the original commands are no longer
 200     recognized.
 201 \stopitem
 202
 203 \startitem
 204     The primitives \type {\ifincsname}, \type {\expanded} and \type {\quitvmode} are now
 205     core primitives.
 206 \stopitem
 207
 208 \startitem
 209     As the hz and protrusion mechanism are part of the core the related
 210     primitives \type {\lpcode}, \type {\rpcode}, \type {\efcode}, \type
 211     {\leftmarginkern}, \type {\rightmarginkern} are promoted to core primitives. The
 212     two commands \type {\protrudechars} and \type {\adjustspacing} replace their
 213     prefixed with \type {\pdf} originals.
 214 \stopitem
 215
 216 \startitem
 217     When \type {\adjustspacing} has value~2, hz optimization will be applied to
 218     glyphs and kerns. When the value is~3, only glyphs will be treated. A value
 219     smaller than~2 disables this feature.
 220 \stopitem
 221
 222 \startitem
 223     The \type {\tagcode} primitive is promoted to core primitive.
 224 \stopitem
 225
 226 \startitem
 227     The \type {\letterspacefont} feature is now part of the core but will not be
 228     changed (improved). We just provide it for legacy use.
 229 \stopitem
 230
 231 \startitem
 232     The \type {\pdfnoligatures} primitive is now \type {\ignoreligaturesinfont}.
 233 \stopitem
 234
 235 \startitem
 236     The \type {\pdfcopyfont} primitive is now \type {\copyfont}.
 237 \stopitem
 238
 239 \startitem
 240     The \type {\pdffontexpand} primitive is now \type {\expandglyphsinfont}.
 241 \stopitem
 242
 243 \startitem
 244     Because position tracking is also available in \DVI\ mode the
 245     \type {\savepos}, \type {\lastxpos} and \type {\lastypos} commands now
 246     replace their \type {pdf} prefixed originals.
 247 \stopitem
 248
 249 \startitem
 250     The introspective primitives \type {\pdflastximagecolordepth} and \type
 251     {\pdfximagebbox} have been removed. One can use external applications to
 252     determine these properties or use the built|-|in \type {img} library.
 253 \stopitem
 254
 255 \startitem
 256     The initializers \type {\pdfoutput} has been replaced by \type {\outputmode} and
 257     \type {\pdfdraftmode} is now \type {\draftmode}.
 258 \stopitem
 259
 260 \startitem
 261     The pixel multiplier dimension \type {\pdfpxdimen} lots its prefix and is now calles
 262     \type {\pxdimen}.
 263 \stopitem
 264
 265 \startitem
 266     An extra \type {\pdfimageaddfilename} option has been added that can be used to block
 267     writing the filename to the \PDF\ file.
 268 \stopitem
 269
 270 \stopitemize
 271
 272 One change involves the so called xforms and ximages. In \PDFTEX\ these are
 273 implemented as so called whatsits. But contrary to other whatsits they have
 274 dimensions that need to be taken into account when for instance calculating
 275 optimal line breaks. In \LUATEX\ these are now promoted to normal nodes, which
 276 simplifies code that needs those dimensions.
 277
 278 Another reason for promotion is that these are useful concepts. Backends can
 279 provide the ability to use content that has been rendered in several places,
 280 and images are also common. For that reason we also changed the names:
 281
 282 \starttabulate[|l|l|]
 283 \NC \bf new name                         \NC \bf old name \NC \NR
 284 \NC \type {\saveboxresource}             \NC \type {\pdfxform}           \NC \NR
 285 \NC \type {\saveimageresource}           \NC \type {\pdfximage}          \NC \NR
 286 \NC \type {\useboxresource}              \NC \type {\pdfrefxform}        \NC \NR
 287 \NC \type {\useimageresource}            \NC \type {\pdfrefximage}       \NC \NR
 288 \NC \type {\lastsavedboxresourceindex}   \NC \type {\pdflastxform}       \NC \NR
 289 \NC \type {\lastsavedimageresourceindex} \NC \type {\pdflastximage}      \NC \NR
 290 \NC \type {\lastsavedimageresourcepages} \NC \type {\pdflastximagepages} \NC \NR
 291 \stoptabulate
 292
 293 There are a few \type {\pdf...} primitives that relate to this but these are
 294 typical backend specific ones. The index that gets returned is to be considered
 295 as \quote {just a number} and although it still has the same meaning (object
 296 related) as before, you should not depend on that.
 297
 298 The protrusion detection mechanism is enhanced a bit to enable a bit more complex
 299 situations. When protrusion characters are identified some nodes are skipped:
 300
 301 \startitemize[packed]
 302 \startitem zero glue \stopitem
 303 \startitem penalties \stopitem
 304 \startitem empty discretionaries \stopitem
 305 \startitem normal zero kerns \stopitem
 306 \startitem rules with zero dimensions \stopitem
 307 \startitem math nodes with a surround of zero \stopitem
 308 \startitem dir nodes \stopitem
 309 \startitem empty horizontal lists \stopitem
 310 \startitem local par nodes \stopitem
 311 \startitem inserts, marks and adjusts \stopitem
 312 \startitem boundaries \stopitem
 313 \startitem whatsits \stopitem
 314 \stopitemize
 315
 316 Because this can not be enough, you can also use a boundary node to make the next
 317 node being ignored. When the boundary value is~1 or~3, the next node will be
 318 ignored in the test when locating a left boundary condition. When the value is~2
 319 or~3, the previous node will be ignored when locating a right boundary condition
 320 (the search goes from right to left). This permits protrusion combined with for
 321 instance content moved into the margin:
 322
 323 \starttyping
 324 \boundary1\llap{!\quad}«Who needs protrusion?»
 325 \stoptyping
 326
 327 \stopsubsection
 328
 329 \startsubsection[title=Changes from \ALEPH\ RC4]
 330
 331 Because we wanted proper directional typesetting the \ALEPH\ mechanisms looked
 332 most attractive. These are rather close to the ones provided by \OMEGA, so what
 333 we say next applies to both these programs.
 334
 335 \startitemize
 336
 337 \startitem
 338     The extended 16-bit math primitives (\type {\omathcode} etc.) have been
 339     removed.
 340 \stopitem
 341
 342 \startitem
 343     The \OCP\ processing is no longer supported at all. As a consequence, the
 344     following primitives have been removed:
 345
 346     \start \raggedright
 347     \type {\ocp}, \type {\externalocp}, \type {\ocplist}, \type {\pushocplist},
 348     \type {\popocplist}, \type {\clearocplists}, \type {\addbeforeocplist}, \type
 349     {\addafterocplist}, \type {\removebeforeocplist}, \type {\removeafterocplist}
 350     and \type {\ocptracelevel}
 351     \par \stop
 352 \stopitem
 353
 354 \startitem
 355     \LUATEX\ only understands 4~of the 16~direction specifiers of \ALEPH: \type
 356     {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL}
 357     (mongolian). All other direction specifiers generate an error.
 358 \stopitem
 359
 360 \startitem
 361     The input translations from \ALEPH\ are not implemented, the related
 362     primitives are not available:
 363
 364     \start \raggedright
 365     \type {\DefaultInputMode}, \type {\noDefaultInputMode}, \type {\noInputMode},
 366     \type {\InputMode}, \type {\DefaultOutputMode}, \type {\noDefaultOutputMode},
 367     \type {\noOutputMode}, \type {\OutputMode}, \type {\DefaultInputTranslation},
 368     \type {\noDefaultInputTranslation}, \type {\noInputTranslation}, \type
 369     {\InputTranslation}, \type {\DefaultOutputTranslation}, \type
 370     {\noDefaultOutputTranslation}, \type {\noOutputTranslation} and \type
 371     {\OutputTranslation}
 372     \par \stop
 373 \stopitem
 374
 375 \startitem
 376     Several bugs have been fixed. The \type {\hoffset} bug when \type {\pagedir TRT}
 377     is gone, removing the need for an explicit fix to \type {\hoffset}. Also bug
 378     causing \type {\fam} to fail for family numbers above 15 is fixed. A fair amount
 379     of other minor bugs are fixed as well, most of these related to \type
 380     {\tracingcommands} output.
 381 \stopitem
 382
 383 \startitem
 384     The scanner for direction specifications now allows an optional space after
 385     the direction is completely parsed.
 386 \stopitem
 387
 388 \startitem
 389     The \type {^^} notation has been extended: after \type {^^^^} four hexadecimal
 390     characters are expected and after \type {^^^^^^} six hexadecimal characters
 391     have to be given. The original \TEX\ interpretation is still valid for the
 392     \type {^^} case but the four and six variants do no backtracking, i.e.\ when
 393     they are not followed by the right number of hexadecimal digits they issue an
 394     error message. Because \type{^^^} is a normal \TEX\ case, we don't support the
 395     odd number of \type {^^^^^} either.
 396 \stopitem
 397
 398 \startitem
 399     Glues {\it immediately after} direction change commands are not legal
 400     breakpoints.
 401 \stopitem
 402
 403 \startitem
 404     Several mechanisms that need to be right|-|to|-|left aware have been
 405     improved. For instance placement of formula numbers.
 406 \stopitem
 407
 408 \startitem
 409     The page dimension related primitives \type {\pagewidth} and \type
 410     {\pageheight} have been promoted to core primitives.
 411 \stopitem
 412
 413 \startitem
 414     The primitives \type {\charwd}, \type {\charht}, \type {\chardp} and \type
 415     {\charit} have been removes as we have the \ETEX\ variants \type
 416     {\fontchar*}.
 417 \stopitem
 418
 419 \startitem
 420     The two dimension registers \type {\pagerightoffset} and \type
 421     {\pagebottomoffset} are now core primitives.
 422 \stopitem
 423
 424 \startitem
 425     The direction related primitives \type {\pagedir}, \type {\bodydir}, \type
 426     {\pardir}, \type {\textdir}, \type {\mathdir} and \type {\boxdir} are now
 427     core primitives.
 428 \stopitem
 429
 430 \startitem
 431     The primitive \type {\pdftracingfonts} is now \type {\tracingfonts} as it
 432     doesn't relate to the backend.
 433 \stopitem
 434
 435 \startitem
 436     The experimental primitive \type {\pdfinsertht} is kept as \type {\insertht}.
 437 \stopitem
 438
 439 \startitem
 440     The promotion of primitives to core primitives as well as the removed of all
 441     others mean that the initialization namespace \type {aleph} is gone.
 442 \stopitem
 443
 444 \stopitemize
 445
 446 \stopsubsection
 447
 448 \startsubsection[title=Changes from standard \WEBC]
 449
 450 The compilation framework is \WEBC\ and we keep using that but without the
 451 \PASCAL\ to \CCODE\ step. This framework also provides some common features that
 452 deal with reading bytes from files and locating files in \TDS. This is what we do
 453 different:
 454
 455 \startitemize
 456
 457 \startitem
 458     There is no mltex support.
 459 \stopitem
 460
 461 \startitem
 462     There is no enctex support.
 463 \stopitem
 464
 465 \startitem
 466     The following commandline switches are silently ignored, even in non|-|\LUA\
 467     mode: \type {-8bit}, \type {-translate-file}, \type {-mltex}, \type {-enc}
 468     and \type {-etex}.
 469 \stopitem
 470
 471 \startitem
 472     The \type {\openout} whatsits are not written to the log file.
 473 \stopitem
 474
 475 \startitem
 476     Some of the so|-|called web2c extensions are hard to set up in non|-|\KPSE\
 477     mode because \type {texmf.cnf} is not read: \type {shell-escape} is off (but
 478     that is not a problem because of \LUA's \type {os.execute}), and the paranoia
 479     checks on \type {openin} and \type {openout} do not happen (however, it is
 480     easy for a \LUA\ script to do this itself by overloading \type {io.open}).
 481 \stopitem
 482
 483 \startitem
 484     The \quote{E} option does not do anything useful.
 485 \stopitem
 486
 487 \stopitemize
 488
 489 \stopsubsection
 490
 491 \startsubsection[title=The backend primitives \type {\pdf*}]
 492
 493 In a previous section we mentioned that some \PDFTEX\ primitives were removed and
 494 others promoted to core \LUATEX\ primitives. That is only part of the story. In
 495 order to separate the backend specific primitives in de code these commands are
 496 now replaced by only a few. In traditional \TEX\ we only had the \DVI\ backend
 497 but now we have two: \DVI\ and \PDF. Additional functionality is implemented as
 498 \quote {extensions} in \TEX speak. By separating more strickly we are able to
 499 keep the core (fontend) clean and stable. If for some reason an extra backend
 500 option is needed, it can be implemented without touching the core. The three
 501 \PDF\ backend related primitives are
 502
 503 \starttyping
 504 \pdfextension command [specification]
 505 \pdfvariable  name
 506 \pdffeedback  name
 507 \stoptyping
 508
 509 An extension triggers further parsing, depending on the command given. A variable is
 510 a (kind of) register and can be read and written, while a feedback is reporting
 511 something (as it comes from the backend it's normally a sequence of tokens).
 512
 513 In order for \LUATEX\ to be more than just \TEX\ you need to enable primitives. That
 514 has already be the case right from the start. If you want the traditional \PDFTEX\
 515 primitives (for as far their functionality is still around) you now can do this:
 516
 517 \starttyping
 518 \protected\def\pdfliteral             {\pdfextension literal}
 519 \protected\def\pdfcolorstack          {\pdfextension colorstack}
 520 \protected\def\pdfsetmatrix           {\pdfextension setmatrix}
 521 \protected\def\pdfsave                {\pdfextension save\relax}
 522 \protected\def\pdfrestore             {\pdfextension restore\relax}
 523 \protected\def\pdfobj                 {\pdfextension obj }
 524 \protected\def\pdfrefobj              {\pdfextension refobj }
 525 \protected\def\pdfannot               {\pdfextension annot }
 526 \protected\def\pdfstartlink           {\pdfextension startlink }
 527 \protected\def\pdfendlink             {\pdfextension endlink\relax}
 528 \protected\def\pdfoutline             {\pdfextension outline }
 529 \protected\def\pdfdest                {\pdfextension dest }
 530 \protected\def\pdfthread              {\pdfextension thread }
 531 \protected\def\pdfstartthread         {\pdfextension startthread }
 532 \protected\def\pdfendthread           {\pdfextension endthread\relax}
 533 \protected\def\pdfinfo                {\pdfextension info }
 534 \protected\def\pdfcatalog             {\pdfextension catalog }
 535 \protected\def\pdfnames               {\pdfextension names }
 536 \protected\def\pdfincludechars        {\pdfextension includechars }
 537 \protected\def\pdffontattr            {\pdfextension fontattr }
 538 \protected\def\pdfmapfile             {\pdfextension mapfile }
 539 \protected\def\pdfmapline             {\pdfextension mapline }
 540 \protected\def\pdftrailer             {\pdfextension trailer }
 541 \protected\def\pdfglyphtounicode      {\pdfextension glyphtounicode }
 542 \stoptyping
 543
 544 The introspective primitives can be defined as:
 545
 546 \starttyping
 547 \def\pdftexversion    {\numexpr\pdffeedback version\relax}
 548 \def\pdftexrevision           {\pdffeedback revision}
 549 \def\pdflastlink      {\numexpr\pdffeedback lastlink\relax}
 550 \def\pdfretval        {\numexpr\pdffeedback retval\relax}
 551 \def\pdflastobj       {\numexpr\pdffeedback lastobj\relax}
 552 \def\pdflastannot     {\numexpr\pdffeedback lastannot\relax}
 553 \def\pdfxformname     {\numexpr\pdffeedback xformname\relax}
 554 \def\pdfcreationdate          {\pdffeedback creationdate}
 555 \def\pdffontname      {\numexpr\pdffeedback fontname\relax}
 556 \def\pdffontobjnum    {\numexpr\pdffeedback fontobjnum\relax}
 557 \def\pdffontsize      {\dimexpr\pdffeedback fontsize\relax}
 558 \def\pdfpageref       {\numexpr\pdffeedback pageref\relax}
 559 \def\pdfcolorstackinit        {\pdffeedback colorstackinit}
 560 \stoptyping
 561
 562 The configuration related registers have become:
 563
 564 \starttyping
 565 \edef\pdfcompresslevel            {\pdfvariable compresslevel}
 566 \edef\pdfobjcompresslevel         {\pdfvariable objcompresslevel}
 567 \edef\pdfdecimaldigits            {\pdfvariable decimaldigits}
 568 \edef\pdfgamma                    {\pdfvariable gamma}
 569 \edef\pdfimageresolution          {\pdfvariable imageresolution}
 570 \edef\pdfimageapplygamma          {\pdfvariable imageapplygamma}
 571 \edef\pdfimagegamma               {\pdfvariable imagegamma}
 572 \edef\pdfimagehicolor             {\pdfvariable imagehicolor}
 573 \edef\pdfimageaddfilename         {\pdfvariable imageaddfilename}
 574 \edef\pdfpkresolution             {\pdfvariable pkresolution}
 575 \edef\pdfpkfixeddpi               {\pdfvariable pkfixeddpi}
 576 \edef\pdfinclusioncopyfonts       {\pdfvariable inclusioncopyfonts}
 577 \edef\pdfinclusionerrorlevel      {\pdfvariable inclusionerrorlevel}
 578 \edef\pdfignoreunknownimages      {\pdfvariable ignoreunknownimages}
 579 \edef\pdfgentounicode             {\pdfvariable gentounicode}
 580 \edef\pdfpagebox                  {\pdfvariable pagebox}
 581 \edef\pdfminorversion             {\pdfvariable minorversion}
 582 \edef\pdfuniqueresname            {\pdfvariable uniqueresname}
 583
 584 \edef\pdfhorigin                  {\pdfvariable horigin}
 585 \edef\pdfvorigin                  {\pdfvariable vorigin}
 586 \edef\pdflinkmargin               {\pdfvariable linkmargin}
 587 \edef\pdfdestmargin               {\pdfvariable destmargin}
 588 \edef\pdfthreadmargin             {\pdfvariable threadmargin}
 589 \edef\pdfxformmargin              {\pdfvariable xformmargin}
 590
 591 \edef\pdfpagesattr                {\pdfvariable pagesattr}
 592 \edef\pdfpageattr                 {\pdfvariable pageattr}
 593 \edef\pdfpageresources            {\pdfvariable pageresources}
 594 \edef\pdfxformattr                {\pdfvariable xformattr}
 595 \edef\pdfxformresources           {\pdfvariable xformresources}
 596 \edef\pdfpkmode                   {\pdfvariable pkmode}
 597
 598 \edef\pdfsuppressoptionalinfo     {\pdfvariable suppressoptionalinfo }
 599 \edef\pdftrailerid                {\pdfvariable trailerid }
 600 \stoptyping
 601
 602 The variables are internal ones, so they are anonymous. When you ask for the
 603 meaning of a few previously defined ones:
 604
 605 \starttyping
 606 \meaning\pdfhorigin
 607 \meaning\pdfcompresslevel
 608 \meaning\pdfpageattr
 609 \stoptyping
 610
 611 you will get:
 612
 613 \starttyping
 614 macro:->[internal backend dimension]
 615 macro:->[internal backend integer]
 616 macro:->[internal backend tokenlist]
 617 \stoptyping
 618
 619 The \type {\edef} can also be an \type {\def} but it's a bit more efficient
 620 to expand the lookup related register beforehand. After that you can adapt
 621 the defaults; these are:
 622
 623 \starttyping
 624 \pdfcompresslevel         9
 625 \pdfobjcompresslevel      1 % used: (0,9)
 626 \pdfdecimaldigits         4 % used: (3,6)
 627 \pdfgamma              1000
 628 \pdfimageresolution      71
 629 \pdfimageapplygamma       0
 630 \pdfimagegamma         2200
 631 \pdfimagehicolor          1
 632 \pdfimageaddfilename      1
 633 \pdfpkresolution         72
 634 \pdfpkfixeddpi            0
 635 \pdfinclusioncopyfonts    0
 636 \pdfinclusionerrorlevel   0
 637 \pdfignoreunknownimages   0
 638 \pdfgentounicode          0
 639 \pdfpagebox               0
 640 \pdfminorversion          4
 641 \pdfuniqueresname         0
 642
 643 \pdfhorigin             1in
 644 \pdfvorigin             1in
 645 \pdflinkmargin          0pt
 646 \pdfdestmargin          0pt
 647 \pdfthreadmargin        0pt
 648 \pdfxformmargin         0pt
 649 \stoptyping
 650
 651 If you also want some backward compatibility, you can add:
 652
 653 \starttyping
 654 \let\pdfpagewidth      \pagewidth
 655 \let\pdfpageheight     \pageheight
 656
 657 \let\pdfadjustspacing  \adjustspacing
 658 \let\pdfprotrudechars  \protrudechars
 659 \let\pdfnoligatures    \ignoreligaturesinfont
 660 \let\pdffontexpand     \expandglyphsinfont
 661 \let\pdfcopyfont       \copyfont
 662
 663 \let\pdfxform          \saveboxresource
 664 \let\pdflastxform      \lastsavedboxresourceindex
 665 \let\pdfrefxform       \useboxresource
 666
 667 \let\pdfximage         \saveimageresource
 668 \let\pdflastximage     \lastsavedimageresourceindex
 669 \let\pdflastximagepages\lastsavedimageresourcepages
 670 \let\pdfrefximage      \useimageresource
 671
 672 \let\pdfsavepos        \savepos
 673 \let\pdflastxpos       \lastxpos
 674 \let\pdflastypos       \lastypos
 675
 676 \let\pdfoutput         \outputmode
 677 \let\pdfdraftmode      \draftmode
 678
 679 \let\pdfpxdimen        \pxdimen
 680
 681 \let\pdfinsertht       \insertht
 682
 683 \let\pdfnormaldeviate  \normaldeviate
 684 \let\pdfuniformdeviate \uniformdeviate
 685 \let\pdfsetrandomseed  \setrandomseed
 686 \let\pdfrandomseed     \randomseed
 687
 688 \let\pdfprimitive      \primitive
 689 \let\ifpdfprimitive    \ifprimitive
 690
 691 \let\ifpdfabsnum       \ifabsnum
 692 \let\ifpdfabsdim       \ifabsdim
 693 \stoptyping
 694
 695 And even:
 696
 697 \starttyping
 698 \newdimen\pdfeachlineheight
 699 \newdimen\pdfeachlinedepth
 700 \newdimen\pdflastlinedepth
 701 \newdimen\pdffirstlineheight
 702 \newdimen\pdfignoreddimen
 703 \stoptyping
 704
 705 The backend is derived from \PDFTEX\ so the same syntax applies. However, the
 706 \type {outline} command accepts a \type {objnum} followed by a number. No
 707 checking takes place so when this is used it had better be a valid (flushed)
 708 object.
 709
 710 In order to be (more or less) compatible with \PDFTEX\ we also support the
 711 option to suppress some info:
 712
 713 \starttyping
 714 \pdfvariable suppressoptionalinfo \numexpr
 715         0
 716     +   1   % PTEX.FullBanner
 717     +   2   % PTEX.FileName
 718     +   4   % PTEX.PageNumber
 719     +   8   % PTEX.InfoDict
 720     +  16   % Creator
 721     +  32   % CreationDate
 722     +  64   % ModDate
 723     + 128   % Producer
 724     + 256   % Trapped
 725     + 512   % ID
 726 \relax
 727 \stoptyping
 728
 729 In addition you can overload the trailer id, but we don't do any checking on
 730 validity, so you have to pass a valid array. The following is like the ones
 731 normally generated by the engine:
 732
 733 \starttyping
 734 \pdfvariable trailerid {[
 735     <FA052949448907805BA83C1E78896398>
 736     <FA052949448907805BA83C1E78896398>
 737 ]}
 738 \stoptyping
 739
 740 So, you even need to include the brackets!
 741
 742 \stopsubsection
 743
 744 \stopsection
 745
 746 \startsection[title=Implementation notes]
 747
 748 \startsubsection[title=Memory allocation]
 749
 750 The single internal memory heap that traditional \TEX\ used for tokens and nodes
 751 is split into two separate arrays. Each of these will grow dynamically when
 752 needed.
 753
 754 The \type {texmf.cnf} settings related to main memory are no longer used (these
 755 are: \type {main_memory}, \type {mem_bot}, \type {extra_mem_top} and \type
 756 {extra_mem_bot}). \quote {Out of main memory} errors can still occur, but the
 757 limiting factor is now the amount of RAM in your system, not a predefined limit.
 758
 759 Also, the memory (de)allocation routines for nodes are completely rewritten. The
 760 relevant code now lives in the C file \type {texnode.c}, and basically uses a
 761 dozen or so \quote {avail} lists instead of a doubly|-|linked model. An extra
 762 function layer is added so that the code can ask for nodes by type instead of
 763 directly requisitioning a certain amount of memory words.
 764
 765 Because of the split into two arrays and the resulting differences in the data
 766 structures, some of the macros have been duplicated. For instance, there are now
 767 \type {vlink} and \type {vinfo} as well as \type {token_link} and \type
 768 {token_info}. All access to the variable memory array is now hidden behind a
 769 macro called \type {vmem}.
 770
 771 The implementation of the growth of two arrays (via reallocation) introduces a
 772 potential pitfall: the memory arrays should never be used as the left hand side
 773 of a statement that can modify the array in question.
 774
 775 The input line buffer and pool size are now also reallocated when needed, and the
 776 \type {texmf.cnf} settings \type {buf_size} and \type {pool_size} are silently
 777 ignored.
 778
 779 \stopsubsection
 780
 781 \startsubsection[title=Sparse arrays]
 782
 783 The \type {\mathcode}, \type {\delcode}, \type {\catcode}, \type {\sfcode}, \type {\lccode}
 784 and \type {\uccode} tables are now sparse arrays that are implemented in~\CCODE.
 785 They are no longer part of the \TEX\ \quote {equivalence table} and because each
 786 had 1.1 million entries with a few memory words each, this makes a major
 787 difference in memory usage.
 788
 789 The \type {\catcode}, \type {\sfcode}, \type {\lccode} and \type {\uccode} assignments do
 790 not yet show up when using the etex tracing routines \type {\tracingassigns} and
 791 \type {\tracingrestores} (code simply not written yet).
 792
 793 A side|-|effect of the current implementation is that \type {\global} is now more
 794 expensive in terms of processing than non|-|global assignments.
 795
 796 See \type {mathcodes.c} and \type {textcodes.c} if you are interested in the
 797 details.
 798
 799 Also, the glyph ids within a font are now managed by means of a sparse array and
 800 glyph ids can go up to index $2^{21}-1$.
 801
 802 \stopsubsection
 803
 804 \startsubsection[title=Simple single-character csnames]
 805
 806 Single|-|character commands are no longer treated specially in the internals,
 807 they are stored in the hash just like the multiletter csnames.
 808
 809 The code that displays control sequences explicitly checks if the length is one
 810 when it has to decide whether or not to add a trailing space.
 811
 812 Active characters are internally implemented as a special type of multi|-|letter
 813 control sequences that uses a prefix that is otherwise impossible to obtain.
 814
 815 \stopsubsection
 816
 817 \startsubsection[title=Compressed format]
 818
 819 The format is passed through zlib, allowing it to shrink to roughly half of the
 820 size it would have had in uncompressed form. This takes a bit more \CPU\ cycles
 821 but much less disk \IO, so it should still be faster.
 822
 823 \stopsubsection
 824
 825 \startsubsection[title=Binary file reading]
 826
 827 All of the internal code is changed in such a way that if one of the \type
 828 {read_xxx_file} callbacks is not set, then the file is read by a C function using
 829 basically the same convention as the callback: a single read into a buffer big
 830 enough to hold the entire file contents. While this uses more memory than the
 831 previous code (that mostly used \type {getc} calls), it can be quite a bit faster
 832 (depending on your \IO\ subsystem).
 833
 834 \stopsubsection
 835
 836 \stopsection
 837
 838 \stopchapter
 839
 840 \stopcomponent