manual/luatex-modifications.tex

   1 % language=uk
   2
   3 \environment luatex-style
   4 \environment luatex-logos
   5
   6 \startcomponent luatex-modifications
   7
   8 \startchapter[reference=modifications,title={Modifications}]
   9
  10 \startsection[title=The merged engines]
  11
  12 \startsubsection[title=The need for change]
  13
  14 The first version of \LUATEX\ only had a few extra primitives and it was largely
  15 the same as \PDFTEX. Then we merged substantial parts of \ALEPH\ into the code
  16 and got more primitives. When we got more stable the decision was made to clean
  17 up the rather hybrid nature of the program. This means that some primitives have
  18 been promoted to core primitives, often with a different name, and that others
  19 were removed. This made it possible to start cleaning up the code base. In \in
  20 {chapter} [enhancements] we discussed some new primitives, here we will cover
  21 most of the adapted ones.
  22
  23 Besides the expected changes caused by new functionality, there are a number of
  24 not|-|so|-|expected changes. These are sometimes a side|-|effect of a new
  25 (conflicting) feature, or, more often than not, a change neccessary to clean up
  26 the internal interfaces. These will also be mentioned.
  27
  28 \stopsubsection
  29
  30 \startsubsection[title=Changes from \TEX\ 3.1415926]
  31
  32 Of course it all starts with traditional \TEX. Even if we started with \PDFTEX,
  33 most still comes from the original. But we divert a bit.
  34
  35 \startitemize
  36
  37 \startitem
  38     The current code base is written in \CCODE, not \PASCAL. We use \CWEB\ when
  39     possible. As a consequence instead of one large file plus change files, we
  40     now have multiple files organized in categories like \type {tex}, \type
  41     {pdf}, \type {lang}, \type {font}, \type {lua}, etc. There are some artefacts
  42     of the conversion to \CCODE, but in due time we will clean up the source code
  43     and make sure that the documentation is done right. Many files are in the
  44     \CWEB\ format, but others, like those interfacing to \LUA, are \CCODE\ files.
  45     Of course we want to stay as close as possible to the original so that the
  46     documentation of the fundamentals behind \TEX\ by Don Knuth still applies.
  47 \stopitem
  48
  49 \startitem
  50     See \in {chapter} [languages] for many small changes related to paragraph
  51     building, language handling and hyphenation. The most important change is
  52     that adding a brace group in the middle of a word (like in \type {of{}fice})
  53     does not prevent ligature creation.
  54 \stopitem
  55
  56 \startitem
  57     There is no pool file, all strings are embedded during compilation.
  58 \stopitem
  59
  60 \startitem
  61     The specifier \type {plus 1 fillll} does not generate an error. The extra
  62     \quote{l} is simply typeset.
  63 \stopitem
  64
  65 \startitem
  66     The upper limit to \type {\endlinechar} and \type {\newlinechar} is 127.
  67 \stopitem
  68
  69 \stopitemize
  70
  71 \stopsubsection
  72
  73 \startsubsection[title=Changes from \ETEX\ 2.2]
  74
  75 Being the de factor standard extension of course we provide the \ETEX\
  76 functionality, but with a few small adaptations.
  77
  78 \startitemize
  79
  80 \startitem
  81     The \ETEX\ functionality is always present and enabled so the prepended
  82     asterisk or \type {-etex} switch for \INITEX\ is not needed.
  83 \stopitem
  84
  85 \startitem
  86     The \TEXXET\ extension is not present, so the primitives \type
  87     {\TeXXeTstate}, \type {\beginR}, \type {\beginL}, \type {\endR} and \type
  88     {\endL} are missing. Instead we use the \OMEGA\ approach to directionality.
  89 \stopitem
  90
  91 \startitem
  92     Some of the tracing information that is output by \ETEX's \type
  93     {\tracingassigns} and \type {\tracingrestores} is not there.
  94 \stopitem
  95
  96 \startitem
  97     Register management in \LUATEX\ uses the \ALEPH\ model, so the maximum value
  98     is 65535 and the implementation uses a flat array instead of the mixed
  99     flat|\&|sparse model from \ETEX.
 100 \stopitem
 101
 102 \startitem
 103     When kpathsea is used to find files, \LUATEX\ uses the \type {ofm} file
 104     format to search for font metrics. In turn, this means that \LUATEX\ looks at
 105     the \type {OFMFONTS} configuration variable (like \OMEGA\ and \ALEPH) instead
 106     of \type {TFMFONTS} (like \TEX\ and \PDFTEX). Likewise for virtual fonts
 107     (\LUATEX\ uses the variable \type {OVFFONTS} instead of \type {VFFONTS}).
 108 \stopitem
 109
 110 \stopitemize
 111
 112 \stopsubsection
 113
 114 \startsubsection[title=Changes from \PDFTEX\ 1.40]
 115
 116 Because we want to produce \PDF\ the most natural starting point was the popular
 117 \PDFTEX\ program. We inherit the stable features, dropped most of the
 118 experimental code and promoted some functionality to core \LUATEX\ functionality
 119 which in turn triggered renaming primitives.
 120
 121 For compatibility reasons we still refer to \type {\pdf...} commands but \LUATEX\
 122 has a different backend interface. Instead of these primitives there are three
 123 interfacing primitives: \type {\pdfextension}, \type {\pdfvariable} and
 124 \type {\pdffeedback} that take keywords and optional further arguments. This way
 125 we can extend the features when needed but don't need to adapt the core engine.
 126 The front- and backend are decoupled as much as possible.
 127
 128 \startitemize
 129
 130 \startitem
 131     The (experimental) support for snap nodes has been removed, because it is
 132     much more natural to build this functionality on top of node processing and
 133     attributes. The associated primitives that are now gone are: \type
 134     {\pdfsnaprefpoint}, \type {\pdfsnapy}, and \type {\pdfsnapycomp}.
 135 \stopitem
 136
 137 \startitem
 138     The (experimental) support for specialized spacing around nodes has also been
 139     removed. The associated primitives that are now gone are: \type
 140     {\pdfadjustinterwordglue}, \type {\pdfprependkern}, and \type
 141     {\pdfappendkern}, as well as the five supporting primitives \type
 142     {\knbscode}, \type {\stbscode}, \type {\shbscode}, \type {\knbccode}, and
 143     \type {\knaccode}.
 144 \stopitem
 145
 146 \startitem
 147     A number of \quote {\PDFTEX\ primitives} have been removed as they can be
 148     implemented using \LUA:
 149
 150     \start \raggedright
 151     \type {\pdfelapsedtime}, \type {\pdfescapehex}, \type {\pdfescapename}, \type
 152     {\pdfescapestring}, \type {\pdffiledump}, \type {\pdffilemoddate}, \type
 153     {\pdffilesize}, \type {\pdfforcepagebox}, \type {\pdflastmatch}, \type
 154     {\pdfmatch}, \type {\pdfmdfivesum}, \type {\pdfmovechars}, \type
 155     {\pdfoptionalwaysusepdfpagebox}, \type {\pdfoptionpdfinclusionerrorlevel},
 156     \type {\pdfresettimer}, \type {\pdfshellescape}, \type {\pdfstrcmp} and \type
 157     {\pdfunescapehex}
 158     \par \stop
 159 \stopitem
 160
 161 \startitem
 162     The version related primitives \type {\pdftexbanner}, \type {\pdftexversion}
 163     and \type {\pdftexrevision} are no longer present as there is no longer a
 164     relationship with \PDFTEX\ development.
 165 \stopitem
 166
 167 \startitem
 168     The experimental snapper mechanism has been removed and therefore also the
 169     primitives:
 170
 171     \start \raggedright
 172     \type {\pdfignoreddimen}, \type {\pdffirstlineheight}, \type
 173     {\pdfeachlineheight}, \type {\pdfeachlinedepth} and \type
 174     {\pdflastlinedepth}
 175     \par \stop
 176 \stopitem
 177
 178 \startitem
 179     The experimental primitives \type {\primitive}, \type {\ifprimitive}, \type
 180     {\ifabsnum} and \type {\ifabsdim} are promoted to core primitives. The \type
 181     {\pdf*} prefixed originals are not available.
 182 \stopitem
 183
 184 \startitem
 185     The \PNG\ transparency fix from 1.40.6 is not applied as high|-|level support
 186     is pending. Because \LUATEX\ has a different subsystem for managing images,
 187     more diversion from its ancestor happened in the meantime.
 188 \stopitem
 189
 190 \startitem
 191     Two extra token lists are provides, \type {\pdfxformresources} and \type
 192     {\pdfxformattr}, as an alternative to \type {\pdfxform} keywords.
 193 \stopitem
 194
 195 \startitem
 196     The current version of \LUATEX\ no longer replaces and|/|or merges fonts in
 197     embedded \PDF\ files with fonts of the enveloping \PDF\ document. This
 198     regression may be temporary, depending on how the rewritten font backend will
 199     look like.
 200 \stopitem
 201
 202 \startitem
 203     The primitives \type {\pdfpagewidth} and \type {\pdfpageheight} have been removed
 204     because \type {\pagewidth} and \type {\pageheight} have that purpose.
 205 \stopitem
 206
 207 \startitem
 208     The primitives \type {\pdfnormaldeviate}, \type {\pdfuniformdeviate}, \type
 209     {\pdfsetrandomseed} and \type {\pdfrandomseed} have been promoted to core
 210     primitives without \type {pdf} prefix so the original commands are no longer
 211     recognized.
 212 \stopitem
 213
 214 \startitem
 215     The primitives \type {\ifincsname}, \type {\expanded} and \type {\quitvmode}
 216     are now core primitives.
 217 \stopitem
 218
 219 \startitem
 220     As the hz and protrusion mechanism are part of the core the related
 221     primitives \type {\lpcode}, \type {\rpcode}, \type {\efcode}, \type
 222     {\leftmarginkern}, \type {\rightmarginkern} are promoted to core primitives. The
 223     two commands \type {\protrudechars} and \type {\adjustspacing} replace their
 224     prefixed with \type {\pdf} originals.
 225 \stopitem
 226
 227 \startitem
 228     The hz optimization code has been partially redone so that we no longer need
 229     to create extra font instances. The front- and backend have been decoupled
 230     and more efficient (\PDF) code is generated.
 231 \stopitem
 232
 233 \startitem
 234     When \type {\adjustspacing} has value~2, hz optimization will be applied to
 235     glyphs and kerns. When the value is~3, only glyphs will be treated. A value
 236     smaller than~2 disables this feature.
 237 \stopitem
 238
 239 \startitem
 240     The \type {\tagcode} primitive is promoted to core primitive.
 241 \stopitem
 242
 243 \startitem
 244     The \type {\letterspacefont} feature is now part of the core but will not be
 245     changed (improved). We just provide it for legacy use.
 246 \stopitem
 247
 248 \startitem
 249     The \type {\pdfnoligatures} primitive is now \type {\ignoreligaturesinfont}.
 250 \stopitem
 251
 252 \startitem
 253     The \type {\pdfcopyfont} primitive is now \type {\copyfont}.
 254 \stopitem
 255
 256 \startitem
 257     The \type {\pdffontexpand} primitive is now \type {\expandglyphsinfont}.
 258 \stopitem
 259
 260 \startitem
 261     Because position tracking is also available in \DVI\ mode the \type
 262     {\savepos}, \type {\lastxpos} and \type {\lastypos} commands now replace
 263     their \type {pdf} prefixed originals.
 264 \stopitem
 265
 266 \startitem
 267     The introspective primitives \type {\pdflastximagecolordepth} and \type
 268     {\pdfximagebbox} have been removed. One can use external applications to
 269     determine these properties or use the built|-|in \type {img} library.
 270 \stopitem
 271
 272 \startitem
 273     The initializers \type {\pdfoutput} has been replaced by \type {\outputmode} and
 274     \type {\pdfdraftmode} is now \type {\draftmode}.
 275 \stopitem
 276
 277 \startitem
 278     The pixel multiplier dimension \type {\pdfpxdimen} lots its prefix and is now calles
 279     \type {\pxdimen}.
 280 \stopitem
 281
 282 \startitem
 283     An extra \type {\pdfimageaddfilename} option has been added that can be used to block
 284     writing the filename to the \PDF\ file.
 285 \stopitem
 286
 287 \startitem
 288     The primitive \type {\pdftracingfonts} is now \type {\tracingfonts} as it
 289     doesn't relate to the backend.
 290 \stopitem
 291
 292 \startitem
 293     The experimental primitive \type {\pdfinsertht} is kept as \type {\insertht}.
 294 \stopitem
 295
 296 \startitem
 297     The promotion of primitives to core primitives as well as the separation of
 298     font- and backend means that the initialization namespace \type {pdftex} is
 299     gone.
 300 \stopitem
 301
 302 \stopitemize
 303
 304 One change involves the so called xforms and ximages. In \PDFTEX\ these are
 305 implemented as so called whatsits. But contrary to other whatsits they have
 306 dimensions that need to be taken into account when for instance calculating
 307 optimal line breaks. In \LUATEX\ these are now promoted to normal nodes, which
 308 simplifies code that needs those dimensions.
 309
 310 Another reason for promotion is that these are useful concepts. Backends can
 311 provide the ability to use content that has been rendered in several places,
 312 and images are also common. For that reason we also changed the names:
 313
 314 \starttabulate[|l|l|]
 315 \NC \bf new name                         \NC \bf old name \NC \NR
 316 \NC \type {\saveboxresource}             \NC \type {\pdfxform}           \NC \NR
 317 \NC \type {\saveimageresource}           \NC \type {\pdfximage}          \NC \NR
 318 \NC \type {\useboxresource}              \NC \type {\pdfrefxform}        \NC \NR
 319 \NC \type {\useimageresource}            \NC \type {\pdfrefximage}       \NC \NR
 320 \NC \type {\lastsavedboxresourceindex}   \NC \type {\pdflastxform}       \NC \NR
 321 \NC \type {\lastsavedimageresourceindex} \NC \type {\pdflastximage}      \NC \NR
 322 \NC \type {\lastsavedimageresourcepages} \NC \type {\pdflastximagepages} \NC \NR
 323 \stoptabulate
 324
 325 There are a few \type {\pdffeedback} features that relate to this but these are
 326 typical backend specific ones. The index that gets returned is to be considered
 327 as \quote {just a number} and although it still has the same meaning (object
 328 related) as before, you should not depend on that.
 329
 330 The protrusion detection mechanism is enhanced a bit to enable a bit more complex
 331 situations. When protrusion characters are identified some nodes are skipped:
 332
 333 \startitemize[packed]
 334 \startitem zero glue \stopitem
 335 \startitem penalties \stopitem
 336 \startitem empty discretionaries \stopitem
 337 \startitem normal zero kerns \stopitem
 338 \startitem rules with zero dimensions \stopitem
 339 \startitem math nodes with a surround of zero \stopitem
 340 \startitem dir nodes \stopitem
 341 \startitem empty horizontal lists \stopitem
 342 \startitem local par nodes \stopitem
 343 \startitem inserts, marks and adjusts \stopitem
 344 \startitem boundaries \stopitem
 345 \startitem whatsits \stopitem
 346 \stopitemize
 347
 348 Because this can not be enough, you can also use a protrusion boundary node to
 349 make the next node being ignored. When the value is~1 or~3, the next node will be
 350 ignored in the test when locating a left boundary condition. When the value is~2
 351 or~3, the previous node will be ignored when locating a right boundary condition
 352 (the search goes from right to left). This permits protrusion combined with for
 353 instance content moved into the margin:
 354
 355 \starttyping
 356 \protrusionboundary1\llap{!\quad}«Who needs protrusion?»
 357 \stoptyping
 358
 359 \stopsubsection
 360
 361 \startsubsection[title=Changes from \ALEPH\ RC4]
 362
 363 Because we wanted proper directional typesetting the \ALEPH\ mechanisms looked
 364 most attractive. These are rather close to the ones provided by \OMEGA, so what
 365 we say next applies to both these programs.
 366
 367 \startitemize
 368
 369 \startitem
 370     The extended 16-bit math primitives (\type {\omathcode} etc.) have been
 371     removed.
 372 \stopitem
 373
 374 \startitem
 375     The \OCP\ processing has been removed completely and as a consequence, the
 376     following primitives have been removed:
 377
 378     \start \raggedright
 379     \type {\ocp}, \type {\externalocp}, \type {\ocplist}, \type {\pushocplist},
 380     \type {\popocplist}, \type {\clearocplists}, \type {\addbeforeocplist}, \type
 381     {\addafterocplist}, \type {\removebeforeocplist}, \type {\removeafterocplist}
 382     and \type {\ocptracelevel}
 383     \par \stop
 384 \stopitem
 385
 386 \startitem
 387     \LUATEX\ only understands 4~of the 16~direction specifiers of \ALEPH: \type
 388     {TLT} (latin), \type {TRT} (arabic), \type {RTT} (cjk), \type {LTL}
 389     (mongolian). All other direction specifiers generate an error.
 390 \stopitem
 391
 392 \startitem
 393     The input translations from \ALEPH\ are not implemented, the related
 394     primitives are not available:
 395
 396     \start \raggedright
 397     \type {\DefaultInputMode}, \type {\noDefaultInputMode}, \type {\noInputMode},
 398     \type {\InputMode}, \type {\DefaultOutputMode}, \type {\noDefaultOutputMode},
 399     \type {\noOutputMode}, \type {\OutputMode}, \type {\DefaultInputTranslation},
 400     \type {\noDefaultInputTranslation}, \type {\noInputTranslation}, \type
 401     {\InputTranslation}, \type {\DefaultOutputTranslation}, \type
 402     {\noDefaultOutputTranslation}, \type {\noOutputTranslation} and \type
 403     {\OutputTranslation}
 404     \par \stop
 405 \stopitem
 406
 407 \startitem
 408     Several bugs have been fixed an confusing implementation details have been sorted
 409     out.
 410 \stopitem
 411
 412 \startitem
 413     The scanner for direction specifications now allows an optional space after
 414     the direction is completely parsed.
 415 \stopitem
 416
 417 \startitem
 418     The \type {^^} notation has been extended: after \type {^^^^} four hexadecimal
 419     characters are expected and after \type {^^^^^^} six hexadecimal characters
 420     have to be given. The original \TEX\ interpretation is still valid for the
 421     \type {^^} case but the four and six variants do no backtracking, i.e.\ when
 422     they are not followed by the right number of hexadecimal digits they issue an
 423     error message. Because \type{^^^} is a normal \TEX\ case, we don't support the
 424     odd number of \type {^^^^^} either.
 425 \stopitem
 426
 427 \startitem
 428     Glues {\it immediately after} direction change commands are not legal
 429     breakpoints.
 430 \stopitem
 431
 432 \startitem
 433     Several mechanisms that need to be right|-|to|-|left aware have been
 434     improved. For instance placement of formula numbers.
 435 \stopitem
 436
 437 \startitem
 438     The page dimension related primitives \type {\pagewidth} and \type
 439     {\pageheight} have been promoted to core primitives. The \type {\hoffset} and
 440     \type {\voffset} primitives have been fixed.
 441 \stopitem
 442
 443 \startitem
 444     The primitives \type {\charwd}, \type {\charht}, \type {\chardp} and \type
 445     {\charit} have been removed as we have the \ETEX\ variants \type
 446     {\fontchar*}.
 447 \stopitem
 448
 449 \startitem
 450     The two dimension registers \type {\pagerightoffset} and \type
 451     {\pagebottomoffset} are now core primitives.
 452 \stopitem
 453
 454 \startitem
 455     The direction related primitives \type {\pagedir}, \type {\bodydir}, \type
 456     {\pardir}, \type {\textdir}, \type {\mathdir} and \type {\boxdir} are now
 457     core primitives.
 458 \stopitem
 459
 460 \startitem
 461     The promotion of primitives to core primitives as well as the removed of all
 462     others means that the initialization namespace \type {aleph} is gone.
 463 \stopitem
 464
 465 \stopitemize
 466
 467 The above let's itself summarize as: we took the 32 bit aspects and much of the
 468 directional mechanisms.
 469
 470 \stopsubsection
 471
 472 \startsubsection[title=Changes from standard \WEBC]
 473
 474 The compilation framework is \WEBC\ and we keep using that but without the
 475 \PASCAL\ to \CCODE\ step. This framework also provides some common features that
 476 deal with reading bytes from files and locating files in \TDS. This is what we do
 477 different:
 478
 479 \startitemize
 480
 481 \startitem
 482     There is no mltex support.
 483 \stopitem
 484
 485 \startitem
 486     There is no enctex support.
 487 \stopitem
 488
 489 \startitem
 490     The following encoding related command line switches are silently ignored,
 491     even in non|-|\LUA\ mode: \type {-8bit}, \type {-translate-file}, \type
 492     {-mltex}, \type {-enc} and \type {-etex}.
 493 \stopitem
 494
 495 \startitem
 496     The \type {\openout} whatsits are not written to the log file.
 497 \stopitem
 498
 499 \startitem
 500     Some of the so|-|called \WEBC\ extensions are hard to set up in non|-|\KPSE\
 501     mode because \type {texmf.cnf} is not read: \type {shell-escape} is off (but
 502     that is not a problem because of \LUA's \type {os.execute}), and the paranoia
 503     checks on \type {openin} and \type {openout} do not happen. However, it is
 504     easy for a \LUA\ script to do this itself by overloading \type {io.open}.
 505 \stopitem
 506
 507 \startitem
 508     The \quote{E} option does not do anything useful.
 509 \stopitem
 510
 511 \stopitemize
 512
 513 \stopsubsection
 514
 515 \startsubsection[reference=backendprimitives,title=The backend primitives \type {\pdf*}]
 516
 517 In a previous section we mentioned that some \PDFTEX\ primitives were removed and
 518 others promoted to core \LUATEX\ primitives. That is only part of the story. In
 519 order to separate the backend specific primitives in de code these commands are
 520 now replaced by only a few. In traditional \TEX\ we only had the \DVI\ backend
 521 but now we have two: \DVI\ and \PDF. Additional functionality is implemented as
 522 \quote {extensions} in \TEX speak. By separating more strickly we are able to
 523 keep the core (fontend) clean and stable. If for some reason an extra backend
 524 option is needed, it can be implemented without touching the core. The three
 525 \PDF\ backend related primitives are
 526
 527 \starttyping
 528 \pdfextension command [specification]
 529 \pdfvariable  name
 530 \pdffeedback  name
 531 \stoptyping
 532
 533 An extension triggers further parsing, depending on the command given. A variable is
 534 a (kind of) register and can be read and written, while a feedback is reporting
 535 something (as it comes from the backend it's normally a sequence of tokens).
 536
 537 In order for \LUATEX\ to be more than just \TEX\ you need to enable primitives. That
 538 has already be the case right from the start. If you want the traditional \PDFTEX\
 539 primitives (for as far their functionality is still around) you now can do this:
 540
 541 \starttyping
 542 \protected\def\pdfliteral             {\pdfextension literal}
 543 \protected\def\pdfcolorstack          {\pdfextension colorstack}
 544 \protected\def\pdfsetmatrix           {\pdfextension setmatrix}
 545 \protected\def\pdfsave                {\pdfextension save\relax}
 546 \protected\def\pdfrestore             {\pdfextension restore\relax}
 547 \protected\def\pdfobj                 {\pdfextension obj }
 548 \protected\def\pdfrefobj              {\pdfextension refobj }
 549 \protected\def\pdfannot               {\pdfextension annot }
 550 \protected\def\pdfstartlink           {\pdfextension startlink }
 551 \protected\def\pdfendlink             {\pdfextension endlink\relax}
 552 \protected\def\pdfoutline             {\pdfextension outline }
 553 \protected\def\pdfdest                {\pdfextension dest }
 554 \protected\def\pdfthread              {\pdfextension thread }
 555 \protected\def\pdfstartthread         {\pdfextension startthread }
 556 \protected\def\pdfendthread           {\pdfextension endthread\relax}
 557 \protected\def\pdfinfo                {\pdfextension info }
 558 \protected\def\pdfcatalog             {\pdfextension catalog }
 559 \protected\def\pdfnames               {\pdfextension names }
 560 \protected\def\pdfincludechars        {\pdfextension includechars }
 561 \protected\def\pdffontattr            {\pdfextension fontattr }
 562 \protected\def\pdfmapfile             {\pdfextension mapfile }
 563 \protected\def\pdfmapline             {\pdfextension mapline }
 564 \protected\def\pdftrailer             {\pdfextension trailer }
 565 \protected\def\pdfglyphtounicode      {\pdfextension glyphtounicode }
 566 \stoptyping
 567
 568 The introspective primitives can be defined as:
 569
 570 \starttyping
 571 \def\pdftexversion    {\numexpr\pdffeedback version\relax}
 572 \def\pdftexrevision           {\pdffeedback revision}
 573 \def\pdflastlink      {\numexpr\pdffeedback lastlink\relax}
 574 \def\pdfretval        {\numexpr\pdffeedback retval\relax}
 575 \def\pdflastobj       {\numexpr\pdffeedback lastobj\relax}
 576 \def\pdflastannot     {\numexpr\pdffeedback lastannot\relax}
 577 \def\pdfxformname     {\numexpr\pdffeedback xformname\relax}
 578 \def\pdfcreationdate          {\pdffeedback creationdate}
 579 \def\pdffontname      {\numexpr\pdffeedback fontname\relax}
 580 \def\pdffontobjnum    {\numexpr\pdffeedback fontobjnum\relax}
 581 \def\pdffontsize      {\dimexpr\pdffeedback fontsize\relax}
 582 \def\pdfpageref       {\numexpr\pdffeedback pageref\relax}
 583 \def\pdfcolorstackinit        {\pdffeedback colorstackinit}
 584 \stoptyping
 585
 586 The configuration related registers have become:
 587
 588 \starttyping
 589 \edef\pdfcompresslevel            {\pdfvariable compresslevel}
 590 \edef\pdfobjcompresslevel         {\pdfvariable objcompresslevel}
 591 \edef\pdfdecimaldigits            {\pdfvariable decimaldigits}
 592 \edef\pdfgamma                    {\pdfvariable gamma}
 593 \edef\pdfimageresolution          {\pdfvariable imageresolution}
 594 \edef\pdfimageapplygamma          {\pdfvariable imageapplygamma}
 595 \edef\pdfimagegamma               {\pdfvariable imagegamma}
 596 \edef\pdfimagehicolor             {\pdfvariable imagehicolor}
 597 \edef\pdfimageaddfilename         {\pdfvariable imageaddfilename}
 598 \edef\pdfpkresolution             {\pdfvariable pkresolution}
 599 \edef\pdfpkfixeddpi               {\pdfvariable pkfixeddpi}
 600 \edef\pdfinclusioncopyfonts       {\pdfvariable inclusioncopyfonts}
 601 \edef\pdfinclusionerrorlevel      {\pdfvariable inclusionerrorlevel}
 602 \edef\pdfignoreunknownimages      {\pdfvariable ignoreunknownimages}
 603 \edef\pdfgentounicode             {\pdfvariable gentounicode}
 604 \edef\pdfpagebox                  {\pdfvariable pagebox}
 605 \edef\pdfminorversion             {\pdfvariable minorversion}
 606 \edef\pdfuniqueresname            {\pdfvariable uniqueresname}
 607
 608 \edef\pdfhorigin                  {\pdfvariable horigin}
 609 \edef\pdfvorigin                  {\pdfvariable vorigin}
 610 \edef\pdflinkmargin               {\pdfvariable linkmargin}
 611 \edef\pdfdestmargin               {\pdfvariable destmargin}
 612 \edef\pdfthreadmargin             {\pdfvariable threadmargin}
 613 \edef\pdfxformmargin              {\pdfvariable xformmargin}
 614
 615 \edef\pdfpagesattr                {\pdfvariable pagesattr}
 616 \edef\pdfpageattr                 {\pdfvariable pageattr}
 617 \edef\pdfpageresources            {\pdfvariable pageresources}
 618 \edef\pdfxformattr                {\pdfvariable xformattr}
 619 \edef\pdfxformresources           {\pdfvariable xformresources}
 620 \edef\pdfpkmode                   {\pdfvariable pkmode}
 621
 622 \edef\pdfsuppressoptionalinfo     {\pdfvariable suppressoptionalinfo }
 623 \edef\pdftrailerid                {\pdfvariable trailerid }
 624 \stoptyping
 625
 626 The variables are internal ones, so they are anonymous. When you ask for the
 627 meaning of a few previously defined ones:
 628
 629 \starttyping
 630 \meaning\pdfhorigin
 631 \meaning\pdfcompresslevel
 632 \meaning\pdfpageattr
 633 \stoptyping
 634
 635 you will get:
 636
 637 \starttyping
 638 macro:->[internal backend dimension]
 639 macro:->[internal backend integer]
 640 macro:->[internal backend tokenlist]
 641 \stoptyping
 642
 643 The \type {\edef} can also be an \type {\def} but it's a bit more efficient
 644 to expand the lookup related register beforehand. After that you can adapt
 645 the defaults; these are:
 646
 647 \starttyping
 648 \pdfcompresslevel         9
 649 \pdfobjcompresslevel      1 % used: (0,9)
 650 \pdfdecimaldigits         4 % used: (3,6)
 651 \pdfgamma              1000
 652 \pdfimageresolution      71
 653 \pdfimageapplygamma       0
 654 \pdfimagegamma         2200
 655 \pdfimagehicolor          1
 656 \pdfimageaddfilename      1
 657 \pdfpkresolution         72
 658 \pdfpkfixeddpi            0
 659 \pdfinclusioncopyfonts    0
 660 \pdfinclusionerrorlevel   0
 661 \pdfignoreunknownimages   0
 662 \pdfgentounicode          0
 663 \pdfpagebox               0
 664 \pdfminorversion          4
 665 \pdfuniqueresname         0
 666
 667 \pdfhorigin             1in
 668 \pdfvorigin             1in
 669 \pdflinkmargin          0pt
 670 \pdfdestmargin          0pt
 671 \pdfthreadmargin        0pt
 672 \pdfxformmargin         0pt
 673 \stoptyping
 674
 675 If you also want some backward compatibility, you can add:
 676
 677 \starttyping
 678 \let\pdfpagewidth      \pagewidth
 679 \let\pdfpageheight     \pageheight
 680
 681 \let\pdfadjustspacing  \adjustspacing
 682 \let\pdfprotrudechars  \protrudechars
 683 \let\pdfnoligatures    \ignoreligaturesinfont
 684 \let\pdffontexpand     \expandglyphsinfont
 685 \let\pdfcopyfont       \copyfont
 686
 687 \let\pdfxform          \saveboxresource
 688 \let\pdflastxform      \lastsavedboxresourceindex
 689 \let\pdfrefxform       \useboxresource
 690
 691 \let\pdfximage         \saveimageresource
 692 \let\pdflastximage     \lastsavedimageresourceindex
 693 \let\pdflastximagepages\lastsavedimageresourcepages
 694 \let\pdfrefximage      \useimageresource
 695
 696 \let\pdfsavepos        \savepos
 697 \let\pdflastxpos       \lastxpos
 698 \let\pdflastypos       \lastypos
 699
 700 \let\pdfoutput         \outputmode
 701 \let\pdfdraftmode      \draftmode
 702
 703 \let\pdfpxdimen        \pxdimen
 704
 705 \let\pdfinsertht       \insertht
 706
 707 \let\pdfnormaldeviate  \normaldeviate
 708 \let\pdfuniformdeviate \uniformdeviate
 709 \let\pdfsetrandomseed  \setrandomseed
 710 \let\pdfrandomseed     \randomseed
 711
 712 \let\pdfprimitive      \primitive
 713 \let\ifpdfprimitive    \ifprimitive
 714
 715 \let\ifpdfabsnum       \ifabsnum
 716 \let\ifpdfabsdim       \ifabsdim
 717 \stoptyping
 718
 719 And even:
 720
 721 \starttyping
 722 \newdimen\pdfeachlineheight
 723 \newdimen\pdfeachlinedepth
 724 \newdimen\pdflastlinedepth
 725 \newdimen\pdffirstlineheight
 726 \newdimen\pdfignoreddimen
 727 \stoptyping
 728
 729 The backend is derived from \PDFTEX\ so the same syntax applies. However, the
 730 \type {outline} command accepts a \type {objnum} followed by a number. No
 731 checking takes place so when this is used it had better be a valid (flushed)
 732 object.
 733
 734 In order to be (more or less) compatible with \PDFTEX\ we also support the
 735 option to suppress some info:
 736
 737 \starttyping
 738 \pdfvariable suppressoptionalinfo \numexpr
 739         0
 740     +   1   % PTEX.FullBanner
 741     +   2   % PTEX.FileName
 742     +   4   % PTEX.PageNumber
 743     +   8   % PTEX.InfoDict
 744     +  16   % Creator
 745     +  32   % CreationDate
 746     +  64   % ModDate
 747     + 128   % Producer
 748     + 256   % Trapped
 749     + 512   % ID
 750 \relax
 751 \stoptyping
 752
 753 In addition you can overload the trailer id, but we don't do any checking on
 754 validity, so you have to pass a valid array. The following is like the ones
 755 normally generated by the engine:
 756
 757 \starttyping
 758 \pdfvariable trailerid {[
 759     <FA052949448907805BA83C1E78896398>
 760     <FA052949448907805BA83C1E78896398>
 761 ]}
 762 \stoptyping
 763
 764 So, you even need to include the brackets!
 765
 766 \stopsubsection
 767
 768 \stopsection
 769
 770 \startsection[title=Implementation notes]
 771
 772 \startsubsection[title=Memory allocation]
 773
 774 The single internal memory heap that traditional \TEX\ used for tokens and nodes
 775 is split into two separate arrays. Each of these will grow dynamically when
 776 needed.
 777
 778 The \type {texmf.cnf} settings related to main memory are no longer used (these
 779 are: \type {main_memory}, \type {mem_bot}, \type {extra_mem_top} and \type
 780 {extra_mem_bot}). \quote {Out of main memory} errors can still occur, but the
 781 limiting factor is now the amount of RAM in your system, not a predefined limit.
 782
 783 Also, the memory (de)allocation routines for nodes are completely rewritten. The
 784 relevant code now lives in the C file \type {texnode.c}, and basically uses a
 785 dozen or so \quote {avail} lists instead of a doubly|-|linked model. An extra
 786 function layer is added so that the code can ask for nodes by type instead of
 787 directly requisitioning a certain amount of memory words.
 788
 789 Because of the split into two arrays and the resulting differences in the data
 790 structures, some of the macros have been duplicated. For instance, there are now
 791 \type {vlink} and \type {vinfo} as well as \type {token_link} and \type
 792 {token_info}. All access to the variable memory array is now hidden behind a
 793 macro called \type {vmem}. We mention this because using the \TEX book as
 794 reference is still quite valid but not for memory related details. Another
 795 significate detail is that we have double linked node lists and that some nodes
 796 carry more data.
 797
 798 The implementation of the growth of two arrays (via reallocation) introduces a
 799 potential pitfall: the memory arrays should never be used as the left hand side
 800 of a statement that can modify the array in question. Details like this are
 801 of no concern to users.
 802
 803 The input line buffer and pool size are now also reallocated when needed, and the
 804 \type {texmf.cnf} settings \type {buf_size} and \type {pool_size} are silently
 805 ignored.
 806
 807 \stopsubsection
 808
 809 \startsubsection[title=Sparse arrays]
 810
 811 The \type {\mathcode}, \type {\delcode}, \type {\catcode}, \type {\sfcode}, \type
 812 {\lccode} and \type {\uccode} (and the new \type {\hjcode}) tables are now sparse
 813 arrays that are implemented in~\CCODE. They are no longer part of the \TEX\
 814 \quote {equivalence table} and because each had 1.1 million entries with a few
 815 memory words each, this makes a major difference in memory usage.
 816
 817 The \type {\catcode}, \type {\sfcode}, \type {\lccode}, \type {\uccode} and \type
 818 {\hjcode} assignments do not yet show up when using the \ETEX\ tracing routines
 819 \type {\tracingassigns} and \type {\tracingrestores}.
 820
 821 A side|-|effect of the current implementation is that \type {\global} is now more
 822 expensive in terms of processing than non|-|global assignments.
 823
 824 The glyph ids within a font are also managed by means of a sparse array as glyph
 825 ids can go up to index $2^{21}-1$.
 826
 827 \stopsubsection
 828
 829 \startsubsection[title=Simple single-character csnames]
 830
 831 Single|-|character commands are no longer treated specially in the internals,
 832 they are stored in the hash just like the multiletter csnames.
 833
 834 The code that displays control sequences explicitly checks if the length is one
 835 when it has to decide whether or not to add a trailing space.
 836
 837 Active characters are internally implemented as a special type of multi|-|letter
 838 control sequences that uses a prefix that is otherwise impossible to obtain.
 839
 840 \stopsubsection
 841
 842 \startsubsection[title=Compressed format]
 843
 844 The format is passed through \type {zlib}, allowing it to shrink to roughly half
 845 of the size it would have had in uncompressed form. This takes a bit more \CPU\
 846 cycles but much less disk \IO, so it should still be faster.
 847
 848 \stopsubsection
 849
 850 \startsubsection[title=Binary file reading]
 851
 852 All of the internal code is changed in such a way that if one of the \type
 853 {read_xxx_file} callbacks is not set, then the file is read by a \CCODE\ function
 854 using basically the same convention as the callback: a single read into a buffer
 855 big enough to hold the entire file contents. While this uses more memory than the
 856 previous code (that mostly used \type {getc} calls), it can be quite a bit faster
 857 (depending on your \IO\ subsystem).
 858
 859 \stopsubsection
 860
 861 \stopsection
 862
 863 \stopchapter
 864
 865 \stopcomponent