better condition for *bsd* for ffi (trunk & tags)

[luatex.git] / manual / luatex-tex.tex

% language=uk

\environment luatex-style
\environment luatex-logos

\startcomponent luatex-tex

\startchapter[reference=tex,title={The \TEX\ related libraries}]

\section{The \type {lua} library}

\subsection{\LUA\ version}

This library contains one read|-|only item:

\starttyping
<string> s = lua.version
\stoptyping

This returns the \LUA\ version identifier string. The value is currently
\directlua {tex.print(lua.version)}.

\subsection{\LUA\ bytecode registers}

\LUA\ registers can be used to communicate \LUA\ functions across \LUA\ chunks.
The accepted values for assignments are functions and \type {nil}. Likewise, the
retrieved value is either a function or \type {nil}.

\starttyping
lua.bytecode[<number> n] = <function> f
lua.bytecode[<number> n]()
\stoptyping

The contents of the \type {lua.bytecode} array is stored inside the format file
as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code.

Note: The function must not contain any upvalues. Currently, functions containing
upvalues can be stored (and their upvalues are set to \type {nil}), but this is
an artifact of the current \LUA\ implementation and thus subject to change.

The associated function calls are

\startfunctioncall
<function> f = lua.getbytecode(<number> n)
lua.setbytecode(<number> n, <function> f)
\stopfunctioncall

Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially
an anonymous function, a complete file can be stored in a bytecode register like
this:

\startfunctioncall
lua.bytecode[n] = loadfile(filename)
\stopfunctioncall

Now all definitions (functions, variables) contained in the file can be
created by executing this bytecode register:

\startfunctioncall
lua.bytecode[n]()
\stopfunctioncall

Note that the path of the file is stored in the \LUA\ bytecode to be used in
stack backtraces and therefore dumped into the format file if the above code is
used in \INITEX. If it contains private information, i.e. the user name, this
information is then contained in the format file as well. This should be kept in
mind when preloading files into a bytecode register in \INITEX.

\subsection{\LUA\ chunk name registers}

There is an array of 65536 (0--65535) potential chunk names for use with the
\type {\directlua} and \type {\latelua} primitives.

\startfunctioncall
lua.name[<number> n] = <string> s
<string> s = lua.name[<number> n]
\stopfunctioncall

If you want to unset a \LUA\ name, you can assign \type {nil} to it.

\section{The \type {status} library}

This contains a number of run|-|time configuration items that you may find useful
in message reporting, as well as an iterator function that gets all of the names
and values as a table.

\startfunctioncall
<table> info = status.list()
\stopfunctioncall

The keys in the table are the known items, the value is the current value. Almost
all of the values in \type {status} are fetched through a metatable at run|-|time
whenever they are accessed, so you cannot use \type {pairs} on \type {status},
but you {\it can\/} use \type {pairs} on \type {info}, of course. If you do not
need the full list, you can also ask for a single item by using its name as an
index into \type {status}.

The current list is:

\starttabulate[|lT|p|]
\NC \rmbf key          \NC \bf explanation \NC \NR
\NC pdf_gone           \NC written \PDF\ bytes \NC \NR
\NC pdf_ptr            \NC not yet written \PDF\ bytes \NC \NR
\NC dvi_gone           \NC written \DVI\ bytes \NC \NR
\NC dvi_ptr            \NC not yet written \DVI\ bytes \NC \NR
\NC total_pages        \NC number of written pages \NC \NR
\NC output_file_name   \NC name of the \PDF\ or \DVI\ file \NC \NR
\NC log_name           \NC name of the log file \NC \NR
\NC banner             \NC terminal display banner \NC \NR
\NC var_used           \NC variable (one|-|word) memory in use \NC \NR
\NC dyn_used           \NC token (multi|-|word) memory in use  \NC \NR
\NC str_ptr            \NC number of strings \NC \NR
\NC init_str_ptr       \NC number of \INITEX\ strings \NC \NR
\NC max_strings        \NC maximum allowed strings \NC \NR
\NC pool_ptr           \NC string pool index \NC \NR
\NC init_pool_ptr      \NC \INITEX\ string pool index \NC \NR
\NC pool_size          \NC current size allocated for string characters \NC \NR
\NC node_mem_usage     \NC a string giving insight into currently used nodes \NC \NR
\NC var_mem_max        \NC number of allocated words for nodes \NC \NR
\NC fix_mem_max        \NC number of allocated words for tokens \NC \NR
\NC fix_mem_end        \NC maximum number of used tokens \NC \NR
\NC cs_count           \NC number of control sequences \NC \NR
\NC hash_size          \NC size of hash \NC \NR
\NC hash_extra         \NC extra allowed hash \NC \NR
\NC font_ptr           \NC number of active fonts \NC \NR
\NC input_ptr          \NC th elevel of input we're at \NC \NR
\NC max_in_stack       \NC max used input stack entries \NC \NR
\NC max_nest_stack     \NC max used nesting stack entries \NC \NR
\NC max_param_stack    \NC max used parameter stack entries \NC \NR
\NC max_buf_stack      \NC max used buffer position \NC \NR
\NC max_save_stack     \NC max used save stack entries \NC \NR
\NC stack_size         \NC input stack size \NC \NR
\NC nest_size          \NC nesting stack size \NC \NR
\NC param_size         \NC parameter stack size \NC \NR
\NC buf_size           \NC current allocated size of the line buffer \NC \NR
\NC save_size          \NC save stack size \NC \NR
\NC obj_ptr            \NC max \PDF\ object pointer \NC \NR
\NC obj_tab_size       \NC \PDF\ object table size \NC \NR
\NC pdf_os_cntr        \NC max \PDF\ object stream pointer \NC \NR
\NC pdf_os_objidx      \NC \PDF\ object stream index \NC \NR
\NC pdf_dest_names_ptr \NC max \PDF\ destination pointer \NC \NR
\NC dest_names_size    \NC \PDF\ destination table size \NC \NR
\NC pdf_mem_ptr        \NC max \PDF\ memory used \NC \NR
\NC pdf_mem_size       \NC \PDF\ memory size \NC \NR
\NC largest_used_mark  \NC max referenced marks class \NC \NR
\NC filename           \NC name of the current input file \NC \NR
\NC inputid            \NC numeric id of the current input \NC \NR
\NC linenumber         \NC location in the current input file \NC \NR
\NC lasterrorstring    \NC last \TEX\ error string \NC \NR
\NC lastluaerrorstring \NC last \LUA\ error string \NC \NR
\NC lastwarningtag     \NC last warning string\NC \NR
\NC lastwarningstring  \NC last warning tag, normally an indication of in what part\NC \NR
\NC lasterrorcontext   \NC last error context string (with newlines) \NC \NR
\NC luabytecodes       \NC number of active \LUA\ bytecode registers \NC \NR
\NC luabytecode_bytes  \NC number of bytes in \LUA\ bytecode registers \NC \NR
\NC luastate_bytes     \NC number of bytes in use by \LUA\ interpreters \NC \NR
\NC output_active      \NC \type {true} if the \type {\output} routine is active \NC \NR
\NC callbacks          \NC total number of executed callbacks so far \NC \NR
\NC indirect_callbacks \NC number of those that were themselves
                           a result of other callbacks (e.g. file readers) \NC \NR
\NC luatex_version     \NC the \LUATEX\ version number \NC \NR
\NC luatex_revision    \NC the \LUATEX\ revision string \NC \NR
\NC ini_version        \NC \type {true} if this is an \INITEX\ run \NC \NR
\NC shell_escape       \NC \type {0} means disabled, \type {1} is restricted and
                           \type {2} means anything is permitted \NC \NR
\stoptabulate

The error and warning messages can be wiped with the \type {resetmessages}
function.

\section{The \type {tex} library}

The \type {tex} table contains a large list of virtual internal \TEX\
parameters that are partially writable.

The designation \quote {virtual} means that these items are not properly defined
in \LUA, but are only front\-ends that are handled by a metatable that operates
on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like
\type {pairs} and \type {#}) do not work on such items.

At the moment, it is possible to access almost every parameter that has these
characteristics:

\startitemize[packed]
\item You can use it after \type {\the}
\item It is a single token.
\item Some special others, see the list below
\stopitemize

This excludes parameters that need extra arguments, like \type {\the\scriptfont}.

The subset comprising simple integer and dimension registers are
writable as well as readable (stuff like \type {\tracingcommands} and
\type {\parindent}).

\subsection{Internal parameter values}

For all the parameters in this section, it is possible to access them directly
using their names as index in the \type {tex} table, or by using one of the
functions \type {tex.get} and \type {tex.set}.

The exact parameters and return values differ depending on the actual parameter,
and so does whether \type {tex.set} has any effect. For the parameters that {\it
can\/} be set, it is possible to use \type {global} as the first argument to
\type {tex.set}; this makes the assignment global instead of local.

\startfunctioncall
tex.set (["global",] <string> n, ...)
... = tex.get (<string> n)
\stopfunctioncall

Glue is kind of special because there are five values involved. The return value
is a \type {glue_spec} node btu when you pass \type {false} as last argument to
\type {tex.get} you get the width of the glue and when you pass \type {true} you
get all five values. Otherwise you get a node which is a copy of the internal
value so you are responsible for its freeing at the \LUA\ end. When you set a
glue quantity you can either pass a \type {glue_spec} or upto five numbers.

For the registers you can use \type {getskip} (node), \type {getglue} (numbers)
\type {setskip} (node) and \type {setglue} (numbers).

There are also dedicated setters, getters and checkers:

\startfunctioncall
local d = tex.getdimen("foo")
if tex.isdimen("bar") then
    tex.setdimen("bar",d)
end
\stopfunctioncall

There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type
{box} and \type {attribute} registers.

\subsubsection{Integer parameters}

The integer parameters accept and return \LUA\ numbers.

Read|-|write:

\starttwocolumns
\starttyping
tex.adjdemerits
tex.binoppenalty
tex.brokenpenalty
tex.catcodetable
tex.clubpenalty
tex.day
tex.defaulthyphenchar
tex.defaultskewchar
tex.delimiterfactor
tex.displaywidowpenalty
tex.doublehyphendemerits
tex.endlinechar
tex.errorcontextlines
tex.escapechar
tex.exhyphenpenalty
tex.fam
tex.finalhyphendemerits
tex.floatingpenalty
tex.globaldefs
tex.hangafter
tex.hbadness
tex.holdinginserts
tex.hyphenpenalty
tex.interlinepenalty
tex.language
tex.lastlinefit
tex.lefthyphenmin
tex.linepenalty
tex.localbrokenpenalty
tex.localinterlinepenalty
tex.looseness
tex.mag
tex.maxdeadcycles
tex.month
tex.newlinechar
tex.outputpenalty
tex.pausing
tex.postdisplaypenalty
tex.predisplaydirection
tex.predisplaypenalty
tex.pretolerance
tex.relpenalty
tex.righthyphenmin
tex.savinghyphcodes
tex.savingvdiscards
tex.showboxbreadth
tex.showboxdepth
tex.time
tex.tolerance
tex.tracingassigns
tex.tracingcommands
tex.tracinggroups
tex.tracingifs
tex.tracinglostchars
tex.tracingmacros
tex.tracingnesting
tex.tracingonline
tex.tracingoutput
tex.tracingpages
tex.tracingparagraphs
tex.tracingrestores
tex.tracingscantokens
tex.tracingstats
tex.uchyph
tex.vbadness
tex.widowpenalty
tex.year
\stoptyping
\stoptwocolumns

Read|-|only:

\startthreecolumns
\starttyping
tex.deadcycles
tex.insertpenalties
tex.parshape
tex.prevgraf
tex.spacefactor
\stoptyping
\stopthreecolumns

\subsubsection{Dimension parameters}

The dimension parameters accept \LUA\ numbers (signifying scaled points) or
strings (with included dimension). The result is always a number in scaled
points.

Read|-|write:

\startthreecolumns
\starttyping
tex.boxmaxdepth
tex.delimitershortfall
tex.displayindent
tex.displaywidth
tex.emergencystretch
tex.hangindent
tex.hfuzz
tex.hoffset
tex.hsize
tex.lineskiplimit
tex.mathsurround
tex.maxdepth
tex.nulldelimiterspace
tex.overfullrule
tex.pagebottomoffset
tex.pageheight
tex.pageleftoffset
tex.pagerightoffset
tex.pagetopoffset
tex.pagewidth
tex.parindent
tex.predisplaysize
tex.scriptspace
tex.splitmaxdepth
tex.vfuzz
tex.voffset
tex.vsize
tex.prevdepth
tex.prevgraf
tex.spacefactor
\stoptyping
\stopthreecolumns

Read|-|only:

\startthreecolumns
\starttyping
tex.pagedepth
tex.pagefilllstretch
tex.pagefillstretch
tex.pagefilstretch
tex.pagegoal
tex.pageshrink
tex.pagestretch
tex.pagetotal
\stoptyping
\stopthreecolumns

Beware: as with all \LUA\ tables you can add values to them. So, the following is valid:

\starttyping
tex.foo = 123
\stoptyping

When you access a \TEX\ parameter a look up takes place. For read||only variables
that means that you will get something back, but when you set them you create a
new entry in the table thereby making the original invisible.

There are a few special cases that we make an exception for: \type {prevdepth},
\type {prevgraf} and \type {spacefactor}. These normally are accessed via the
\type {tex.nest} table:

\starttyping
tex.nest[tex.nest.ptr].prevdepth   = p
tex.nest[tex.nest.ptr].spacefactor = s
\stoptyping

However, the following also works:

\starttyping
tex.prevdepth   = p
tex.spacefactor = s
\stoptyping

Keep in mind that when you mess with node lists directly at the \LUA\ end you
might need to update the top of the nesting stack's \type {prevdepth} explicitly
as there is no way \LUATEX\ can guess your intentions. By using the accessor in
the \type {tex} tables, you get and set the values atthe top of the nest stack.

\subsubsection{Direction parameters}

The direction parameters are read|-|only and return a \LUA\ string.

\startthreecolumns
\starttyping
tex.bodydir
tex.mathdir
tex.pagedir
tex.pardir
tex.textdir
\stoptyping
\stopthreecolumns

\subsubsection{Glue parameters}

The glue parameters accept and return a userdata object that represents a \type
{glue_spec} node.

\startthreecolumns
\starttyping
tex.abovedisplayshortskip
tex.abovedisplayskip
tex.baselineskip
tex.belowdisplayshortskip
tex.belowdisplayskip
tex.leftskip
tex.lineskip
tex.parfillskip
tex.parskip
tex.rightskip
tex.spaceskip
tex.splittopskip
tex.tabskip
tex.topskip
tex.xspaceskip
\stoptyping
\stopthreecolumns

\subsubsection{Muglue parameters}

All muglue parameters are to be used read|-|only and return a \LUA\ string.

\startthreecolumns
\starttyping
tex.medmuskip
tex.thickmuskip
tex.thinmuskip
\stoptyping
\stopthreecolumns

\subsubsection{Tokenlist parameters}

The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are
converted to and from token lists using \type {\the} \type {\toks} style expansion:
all category codes are either space (10) or other (12). It follows that assigning
to some of these, like \quote {tex.output}, is actually useless, but it feels bad
to make exceptions in view of a coming extension that will accept full|-|blown
token strings.

\startthreecolumns
\starttyping
tex.errhelp
tex.everycr
tex.everydisplay
tex.everyeof
tex.everyhbox
tex.everyjob
tex.everymath
tex.everypar
tex.everyvbox
tex.output
\stoptyping
\stopthreecolumns

\subsection{Convert commands}

All \quote {convert} commands are read|-|only and return a \LUA\ string. The
supported commands at this moment are:

\starttwocolumns
\starttyping
tex.eTeXVersion
tex.eTeXrevision
tex.formatname
tex.jobname
tex.luatexbanner
tex.luatexrevision
tex.fontname(number)
tex.uniformdeviate(number)
tex.number(number)
tex.romannumeral(number)
tex.fontidentifier(number)
\stoptyping
\stoptwocolumns

If you are wondering why this list looks haphazard; these are all the cases of
the \quote {convert} internal command that do not require an argument, as well as
the ones that require only a simple numeric value.

The special (lua-only) case of \type {tex.fontidentifier} returns the \type
{csname} string that matches a font id number (if there is one).

if these are really needed in a macro package.

\subsection{Last item commands}

All \quote {last item} commands are read|-|only and return a number.

The supported commands at this moment are:

\startthreecolumns
\starttyping
tex.lastpenalty
tex.lastkern
tex.lastskip
tex.lastnodetype
tex.inputlineno
tex.lastxpos
tex.lastypos
tex.randomseed
tex.luatexversion
tex.eTeXminorversion
tex.eTeXversion
tex.currentgrouplevel
tex.currentgrouptype
tex.currentiflevel
tex.currentiftype
tex.currentifbranch
\stoptyping
\stopthreecolumns

\subsection{Attribute, count, dimension, skip and token registers}

\TEX's attributes (\type {\attribute}), counters (\type {\count}), dimensions (\type
{\dimen}), skips (\type {\skip}) and token (\type {\toks}) registers can be accessed
and written to using two times five virtual sub|-|tables of the \type {tex}
table:

\startthreecolumns
\starttyping
tex.attribute
tex.count
tex.dimen
tex.skip
tex.toks
\stoptyping
\stopthreecolumns

It is possible to use the names of relevant \type {\attributedef}, \type {\countdef},
\type {\dimendef}, \type {\skipdef}, or \type {\toksdef} control sequences as indices
to these tables:

\starttyping
tex.count.scratchcounter = 0
enormous = tex.dimen['maxdimen']
\stoptyping

In this case, \LUATEX\ looks up the value for you on the fly. You have to use a
valid \type {\countdef} (or \type {\attributedef}, or \type {\dimendef}, or \type
{\skipdef}, or \type {\toksdef}), anything else will generate an error (the intent
is to eventually also allow \type {<chardef tokens>} and even macros that expand
into a number).

The attribute and count registers accept and return \LUA\ numbers.

The dimension registers accept \LUA\ numbers (in scaled points) or strings (with
an included absolute dimension; \type {em} and \type {ex} and \type {px} are
forbidden). The result is always a number in scaled points.

The token registers accept and return \LUA\ strings. \LUA\ strings are converted
to and from token lists using \type {\the} \type {\toks} style expansion: all
category codes are either space (10) or other (12).

The skip registers accept and return \type {glue_spec} userdata node objects (see
the description of the node interface elsewhere in this manual).

As an alternative to array addressing, there are also accessor functions defined
for all cases, for example, here is the set of possibilities for \type {\skip}
registers:

\startfunctioncall
tex.setskip (["global",] <number> n, <node> s)
tex.setskip (["global",] <string> s, <node> s)
<node> s = tex.getskip (<number> n)
<node> s = tex.getskip (<string> s)
\stopfunctioncall

We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and
\type {toks}. Counters and dimen are represented by numbers, skips and muskips by
nodes, and toks by strings. For tokens registers we have an alternative where a
catcode table is specified:

\startfunctioncall
tex.scantoks(0,3,"$e=mc^2$")
tex.scantoks("global",0,"$\int\limits^1_2$")
\stopfunctioncall

In the function-based interface, it is possible to define values globally by
using the string \type {global} as the first function argument.

There are four extra skip related helpers:

\startfunctioncall
tex.setglue (["global"], <number> n,
    width, stretch, shrink, stretch_order, shrink_order)
tex.setglue (["global"], <string> s,
    width, stretch, shrink, stretch_order, shrink_order)
width, stretch, shrink, stretch_order, shrink_order =
    tex.getglue (<number> n)
width, stretch, shrink, stretch_order, shrink_order =
    tex.getglue (<string> s)
\stopfunctioncall

The other two are \type {tex.setmuglue} and \type {tex.getmuglue}.

\subsection{Character code registers}

\TEX's character code tables (\type {\lccode}, \type {\uccode}, \type {\sfcode}, \type
{\catcode}, \type {\mathcode}, \type {\delcode}) can be accessed and written to using
six virtual subtables of the \type {tex} table

\startthreecolumns
\starttyping
tex.lccode
tex.uccode
tex.sfcode
tex.catcode
tex.mathcode
tex.delcode
\stoptyping
\stopthreecolumns

The function call interfaces are roughly as above, but there are a few twists.
\type {sfcode}s are the simple ones:

\startfunctioncall
tex.setsfcode (["global",] <number> n, <number> s)
<number> s = tex.getsfcode (<number> n)
\stopfunctioncall

The function call interface for \type {lccode} and \type {uccode} additionally
allows you to set the associated sibling at the same time:

\startfunctioncall
tex.setlccode (["global"], <number> n, <number> lc)
tex.setlccode (["global"], <number> n, <number> lc, <number> uc)
<number> lc = tex.getlccode (<number> n)
tex.setuccode (["global"], <number> n, <number> uc)
tex.setuccode (["global"], <number> n, <number> uc, <number> lc)
<number> uc = tex.getuccode (<number> n)
\stopfunctioncall

The function call interface for \type {catcode} also allows you to specify a
category table to use on assignment or on query (default in both cases is the
current one):

\startfunctioncall
tex.setcatcode (["global"], <number> n, <number> c)
tex.setcatcode (["global"], <number> cattable, <number> n, <number> c)
<number> lc = tex.getcatcode (<number> n)
<number> lc = tex.getcatcode (<number> cattable, <number> n)
\stopfunctioncall

The interfaces for \type {delcode} and \type {mathcode} use small array tables to
set and retrieve values:

\startfunctioncall
tex.setmathcode (["global"], <number> n, <table> mval )
<table> mval = tex.getmathcode (<number> n)
tex.setdelcode (["global"], <number> n, <table> dval )
<table> dval = tex.getdelcode (<number> n)
\stopfunctioncall

Where the table for \type {mathcode} is an array of 3 numbers, like this:

\starttyping
{
    <number> class,
    <number> family,
    <number> character
}
\stoptyping

And the table for \type {delcode} is an array with 4 numbers, like this:

\starttyping
{
    <number> small_fam,
    <number> small_char,
    <number> large_fam,
    <number> large_char
}
\stoptyping

You can also avoid the table:

\startfunctioncall
tex.setmathcode (["global"], <number> n, <number> class,
    <number> family, <number> character)
class, family, char =
    tex.getmathcodes (<number> n)
tex.setdelcode (["global"], <number> n, <number> smallfam,
    <number> smallchar, <number> largefam, <number> largechar)
smallfam, smallchar, largefam, largechar =
    tex.getdelcodes (<number> n)
\stopfunctioncall

Normally, the third and fourth values in a delimiter code assignment will be zero
according to \type {\Udelcode} usage, but the returned table can have values there
(if the delimiter code was set using \type {\delcode}, for example). Unset \type
{delcode}'s can be recognized because \type {dval[1]} is $-1$.

\subsection{Box registers}

It is possible to set and query actual boxes, using the node interface as defined
in the \type {node} library:

\starttyping
tex.box
\stoptyping

for array access, or

\starttyping
tex.setbox(["global",] <number> n, <node> s)
tex.setbox(["global",] <string> cs, <node> s)
<node> n = tex.getbox(<number> n)
<node> n = tex.getbox(<string> cs)
\stoptyping

for function|-|based access. In the function-based interface, it is possible to
define values globally by using the string \type {global} as the first function
argument.

Be warned that an assignment like

\starttyping
tex.box[0] = tex.box[2]
\stoptyping

does not copy the node list, it just duplicates a node pointer. If \type {\box2}
will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes
invalid as well. To prevent this from happening, always use \type
{node.copy_list()} unless you are assigning to a temporary variable:

\starttyping
tex.box[0] = node.copy_list(tex.box[2])
\stoptyping

The following function will register a box for reuse (this is modelled after so
called xforms in \PDF). You can (re)use the box with \type {\useboxresource} or
by creating a rule node with subtype~2.

\starttyping
local index = tex.saveboxresource(n,attributes,resources,immediate,type)
\stoptyping

The optional second and third arguments are strings, the fourth is a boolean. The
fifth argument is a type. When set to non|-|zero the \type {/Type} entry is
omitted. A value of 1 or 3 still writes a \type {/BBox}, while 2 or 3 will write
a \type {/Matrix}.

You can generate the reference (a rule type) with:

\starttyping
local reused = tex.useboxresource(n,wd,ht,dp)
\stoptyping

The dimensions are optional and the final ones are returned as extra values. The
following is just a bonus (no dimensions returned means that the resource is
unknown):

\starttyping
local w, h, d = tex.getboxresourcedimensions(n)
\stoptyping

You can split a box:

\starttyping
local vlist = tex.splitbox(n,height,mode)
\stoptyping

The remainder is kept in the original box and a packaged vlist is returned. This
operation is comparable to the \type {\vsplit} operation. The mode can be \type
{additional} or \type {exactly} and concerns the split off box.

\subsection{Math parameters}

It is possible to set and query the internal math parameters using:

\startfunctioncall
tex.setmath(["global",] <string> n, <string> t, <number> n)
<number> n = tex.getmath(<string> n, <string> t)
\stopfunctioncall

As before an optional first parameter \type {global} indicates a global
assignment.

The first string is the parameter name minus the leading \quote {Umath}, and the
second string is the style name minus the trailing \quote {style}. Just to be
complete, the values for the math parameter name are:

\starttyping
quad                axis                operatorsize
overbarkern         overbarrule         overbarvgap
underbarkern        underbarrule        underbarvgap
radicalkern         radicalrule         radicalvgap
radicaldegreebefore radicaldegreeafter  radicaldegreeraise
stackvgap           stacknumup          stackdenomdown
fractionrule        fractionnumvgap     fractionnumup
fractiondenomvgap   fractiondenomdown   fractiondelsize
limitabovevgap      limitabovebgap      limitabovekern
limitbelowvgap      limitbelowbgap      limitbelowkern
underdelimitervgap  underdelimiterbgap
overdelimitervgap   overdelimiterbgap
subshiftdrop        supshiftdrop        subshiftdown
subsupshiftdown     subtopmax           supshiftup
supbottommin        supsubbottommax     subsupvgap
spaceafterscript    connectoroverlapmin
ordordspacing       ordopspacing        ordbinspacing     ordrelspacing
ordopenspacing      ordclosespacing     ordpunctspacing   ordinnerspacing
opordspacing        opopspacing         opbinspacing      oprelspacing
opopenspacing       opclosespacing      oppunctspacing    opinnerspacing
binordspacing       binopspacing        binbinspacing     binrelspacing
binopenspacing      binclosespacing     binpunctspacing   bininnerspacing
relordspacing       relopspacing        relbinspacing     relrelspacing
relopenspacing      relclosespacing     relpunctspacing   relinnerspacing
openordspacing      openopspacing       openbinspacing    openrelspacing
openopenspacing     openclosespacing    openpunctspacing  openinnerspacing
closeordspacing     closeopspacing      closebinspacing   closerelspacing
closeopenspacing    closeclosespacing   closepunctspacing closeinnerspacing
punctordspacing     punctopspacing      punctbinspacing   punctrelspacing
punctopenspacing    punctclosespacing   punctpunctspacing punctinnerspacing
innerordspacing     inneropspacing      innerbinspacing   innerrelspacing
inneropenspacing    innerclosespacing   innerpunctspacing innerinnerspacing
\stoptyping

The values for the style parameter name are:

\starttyping
display       crampeddisplay
text          crampedtext
script        crampedscript
scriptscript  crampedscriptscript
\stoptyping

The value is either a number (representing a dimension or number) or a glue spec
node representing a muskip for \type {ordordspacing} and similar spacing
parameters.

\subsection{Special list heads}

The virtual table \type {tex.lists} contains the set of internal registers that
keep track of building page lists.

\starttabulate[|lT|p|]
\NC \bf field           \NC \bf description \NC \NR
\NC page_ins_head       \NC circular list of pending insertions \NC \NR
\NC contrib_head        \NC the recent contributions \NC \NR
\NC page_head           \NC the current page content \NC \NR
%NC temp_head           \NC \NC \NR
\NC hold_head           \NC used for held-over items for next page \NC \NR
\NC adjust_head         \NC head of the current \type {\vadjust} list \NC \NR
\NC pre_adjust_head     \NC head of the current \type {\vadjust pre} list \NC \NR
%NC align_head          \NC \NC \NR
\NC page_discards_head  \NC head of the discarded items of a page break \NC \NR
\NC split_discards_head \NC head of the discarded items in a vsplit \NC \NR
\stoptabulate

\subsection{Semantic nest levels}

The virtual table \type {tex.nest} contains the currently active
semantic nesting state. It has two main parts: a zero-based array of userdata for
the semantic nest itself, and the numerical value \type {tex.nest.ptr}, which
gives the highest available index. Neither the array items in \type {tex.nest[]}
nor \type {tex.nest.ptr} can be assigned to (as this would confuse the
typesetting engine beyond repair), but you can assign to the individual values
inside the array items, e.g.\ \type {tex.nest[tex.nest.ptr].prevdepth}.

\type {tex.nest[tex.nest.ptr]} is the current nest state, \type {tex.nest[0]} the
outermost (main vertical list) level.

The known fields are:

\starttabulate[|lT|l|l|p|]
\NC \rmbf key   \NC \bf type \NC \bf modes \NC \bf explanation \NC \NR
\NC mode        \NC number   \NC all       \NC a number representing the main mode at this level:
                                               \type {0}    = no mode (this happens during \type {\write}),
                                               \type {1}    = vertical,
                                               \type {127}  = horizontal,
                                               \type {253}  = display math,
                                               \type {-1}   = internal vertical,
                                               \type {-127} = restricted horizontal,
                                               \type {-253} = inline math \NC \NR
\NC modeline    \NC number   \NC all       \NC source input line where this mode was entered in,
                                               negative inside the output routine \NC \NR
\NC head        \NC node     \NC all       \NC the head of the current list \NC \NR
\NC tail        \NC node     \NC all       \NC the tail of the current list \NC \NR
\NC prevgraf    \NC number   \NC vmode     \NC number of lines in the previous paragraph \NC \NR
\NC prevdepth   \NC number   \NC vmode     \NC depth of the previous paragraph \NC \NR
\NC spacefactor \NC number   \NC hmode     \NC the current space factor \NC \NR
\NC dirs        \NC node     \NC hmode     \NC used for temporary storage by the line break algorithm\NC \NR
\NC noad        \NC node     \NC mmode     \NC used for temporary storage of a pending fraction numerator,
                                               for \type {\over} etc. \NC \NR
\NC delimptr    \NC node     \NC mmode     \NC used for temporary storage of the previous math delimiter,
                                               for \type {\middle} \NC \NR
\NC mathdir     \NC boolean  \NC mmode     \NC true when during math processing the \type {\mathdir} is not
                                               the same as the surrounding \type {\textdir} \NC \NR
\NC mathstyle   \NC number   \NC mmode     \NC the current \type {\mathstyle} \NC \NR
\stoptabulate

\subsection[sec:luaprint]{Print functions}

The \type {tex} table also contains the three print functions that are the
major interface from \LUA\ scripting to \TEX.

The arguments to these three functions are all stored in an in|-|memory virtual
file that is fed to the \TEX\ scanner as the result of the expansion of
\type {\directlua}.

The total amount of returnable text from a \type {\directlua} command is only
limited by available system \RAM. However, each separate printed string has to
fit completely in \TEX's input buffer.

The result of using these functions from inside callbacks is undefined
at the moment.

\subsubsection{\type {tex.print}}

\startfunctioncall
tex.print(<string> s, ...)
tex.print(<number> n, <string> s, ...)
tex.print(<table> t)
tex.print(<number> n, <table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a separate input line. If there is a
table argument instead of a list of strings, this has to be a consecutive array
of strings to print (the first non-string value will stop the printing process).

The optional parameter can be used to print the strings using the catcode regime
defined by \type {\catcodetable}~\type {n}. If \type {n} is $-1$, the currently
active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are
the result of \type {\the} \type {\toks}: all category codes are 12 (other) except for
the space character, that has category code 10 (space). Otherwise, if \type {n}
is not a valid catcode table, then it is ignored, and the currently active
catcode regime is used instead.

The very last string of the very last \type {tex.print()} command in a \type
{\directlua} will not have the \type {\endlinechar} appended, all others do.

\subsubsection{\type {tex.sprint}}

\startfunctioncall
tex.sprint(<string> s, ...)
tex.sprint(<number> n, <string> s, ...)
tex.sprint(<table> t)
tex.sprint(<number> n, <table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a special kind of input line that
makes it suitable for use as a partial line input mechanism:

\startitemize[packed]
\startitem
    \TEX\ does not switch to the \quote {new line} state, so that leading spaces
    are not ignored.
\stopitem
\startitem
    No \type {\endlinechar} is inserted.
\stopitem
\startitem
    Trailing spaces are not removed.

    Note that this does not prevent \TEX\ itself from eating spaces as result of
    interpreting the line. For example, in

\starttyping
before\directlua{tex.sprint("\\relax")tex.sprint(" inbetween")}after
\stoptyping
    the space before \type {in between} will be gobbled as a result of the \quote
    {normal} scanning of \type {\relax}.
\stopitem
\stopitemize

If there is a table argument instead of a list of strings, this has to
be a consecutive array of strings to print (the first non-string value
will stop the printing process).

The optional argument sets the catcode regime, as with \type {tex.print()}.

\subsubsection{\type {tex.tprint}}

\startfunctioncall
tex.tprint({<number> n, <string> s, ...}, {...})
\stopfunctioncall

This function is basically a shortcut for repeated calls to \type
{tex.sprint(<number> n, <string> s, ...)}, once for each of the supplied argument
tables.

\subsubsection{\type {tex.cprint}}

This function takes a number indicating the to be used catcode, plus either a
table of strings or an argument list of strings that will be pushed into the
input stream.

\startfunctioncall
tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s
tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s
tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored
tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces
tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters
tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters
tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers
\stopfunctioncall

\subsubsection{\type {tex.write}}

\startfunctioncall
tex.write(<string> s, ...)
tex.write(<table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a special kind of input line that
makes it suitable for use as a quick way to dump information:

\startitemize
\item All catcodes on that line are either \quote{space} (for '~') or
     \quote{character} (for all others).
\item There is no \type {\endlinechar} appended.
\stopitemize

If there is a table argument instead of a list of strings, this has to be a
consecutive array of strings to print (the first non-string value will stop the
printing process).

\subsection{Helper functions}

\subsubsection{\type {tex.round}}

\startfunctioncall
<number> n = tex.round(<number> o)
\stopfunctioncall

Rounds \LUA\ number \type {o}, and returns a number that is in the range of a
valid \TEX\ register value. If the number starts out of range, it generates a
\quote {number to big} error as well.

\subsubsection{\type {tex.scale}}

\startfunctioncall
<number> n = tex.scale(<number> o, <number> delta)
<table> n = tex.scale(table o, <number> delta)
\stopfunctioncall

Multiplies the \LUA\ numbers \type {o} and \type {delta}, and returns a rounded
number that is in the range of a valid \TEX\ register value. In the table
version, it creates a copy of the table with all numeric top||level values scaled
in that manner. If the multiplied number(s) are of range, it generates
\quote{number to big} error(s) as well.

Note: the precision of the output of this function will depend on your computer's
architecture and operating system, so use with care! An interface to \LUATEX's
internal, 100\% portable scale function will be added at a later date.

\subsubsection{\type {tex.sp}}

\startfunctioncall
<number> n = tex.sp(<number> o)
<number> n = tex.sp(<string> s)
\stopfunctioncall

Converts the number \type {o} or a string \type {s} that represents an explicit
dimension into an integer number of scaled points.

For parsing the string, the same scanning and conversion rules are used that
\LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like
input language (this includes generating errors for bad values), expect for the
following:

\startitemize[n]
\startitem
    only explicit values are allowed, control sequences are not handled
\stopitem
\startitem
    infinite dimension units (\type {fil...}) are forbidden
\stopitem
\startitem
    \type {mu} units do not generate an error (but may not be useful either)
\stopitem
\stopitemize

\subsubsection{\type {tex.definefont}}

\startfunctioncall
tex.definefont(<string> csname, <number> fontid)
tex.definefont(<boolean> global, <string> csname, <number> fontid)
\stopfunctioncall

Associates \type {csname} with the internal font number \type {fontid}. The
definition is global if (and only if) \type {global} is specified and true (the
setting of \type {globaldefs} is not taken into account).

\subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}}

You can mess with the current line number:

\startfunctioncall
local n = tex.getlinenumber()
tex.setlinenumber(n+10)
\stopfunctioncall

which can be shortcut to:

\startfunctioncall
tex.setlinenumber(10,true)
\stopfunctioncall

This might be handy when you have a callback that read numbers from a file and
combines them in one line (in which case an error message probably has to refer
to the original line). Interference with \TEX's internal handling of numbers is
of course possible.

\subsubsection{\type {tex.error}}

\startfunctioncall
tex.error(<string> s)
tex.error(<string> s, <table> help)
\stopfunctioncall

This creates an error somewhat like the combination of \type {\errhelp} and \type
{\errmessage} would. During this error, deletions are disabled.

The array part of the \type {help} table has to contain strings, one for each
line of error help.

\subsubsection{\type {tex.hashtokens}}

\startfunctioncall
for i,v in pairs (tex.hashtokens()) do ... end
\stopfunctioncall

Returns a list of names. This can be useful for debugging, but note that this
also reports control sequences that may be unreachable at this moment due to
local redefinitions: it is strictly a dump of the hash table. You can use \type
{token.create} to inspect properties, for instance when the \type {command} key
in a created table equals \type {123}, you have the \type {cmdname} value \type
{undefined_cs}.

\subsection[luaprimitives]{Functions for dealing with primitives }

\subsubsection{\type {tex.enableprimitives}}

\startfunctioncall
tex.enableprimitives(<string> prefix, <table> primitive names)
\stopfunctioncall

This function accepts a prefix string and an array of primitive names.

For each combination of \quote {prefix} and \quote {name}, the \type
{tex.enableprimitives} first verifies that \quote {name} is an actual primitive
(it must be returned by one of the \type {tex.extraprimitives()} calls explained
below, or part of \TEX82, or \type {\directlua}). If it is not, \type
{tex.enableprimitives} does nothing and skips to the next pair.

But if it is, then it will construct a csname variable by concatenating the
\quote {prefix} and \quote {name}, unless the \quote {prefix} is already the
actual prefix of \quote {name}. In the latter case, it will discard the \quote
{prefix}, and just use \quote {name}.

Then it will check for the existence of the constructed csname. If the csname is
currently undefined (note: that is not the same as \type {\relax}), it will
globally define the csname to have the meaning: run code belonging to the
primitive \quote {name}. If for some reason the csname is already defined, it
does nothing and tries the next pair.

An example:

\starttyping
  tex.enableprimitives('LuaTeX', {'formatname'})
\stoptyping

will define \type {\LuaTeXformatname} with the same intrinsic meaning as the
documented primitive \type {\formatname}, provided that the control sequences \type
{\LuaTeXformatname} is currently undefined.

When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \type
{\directlua} are available, so no extra primitives {\bf at all}.

If you want to have all the new functionality available using their default
names, as it is now, you will have to add

\starttyping
  \ifx\directlua\undefined \else
     \directlua {tex.enableprimitives('',tex.extraprimitives ())}
  \fi
\stoptyping

near the beginning of your format generation file. Or you can choose different
prefixes for different subsets, as you see fit.

Calling some form of \type {tex.enableprimitives()} is highly important though,
because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\
code but not do much else. The defined csnames are (of course) saved in the
format and will be available at runtime.

\subsubsection{\type {tex.extraprimitives}}

\startfunctioncall
<table> t = tex.extraprimitives(<string> s, ...)
\stopfunctioncall

This function returns a list of the primitives that originate from the engine(s)
given by the requested string value(s). The possible values and their (current)
return values are:

\startluacode
function document.showprimitives(tag)
    for k, v in table.sortedpairs(tex.extraprimitives(tag)) do
        if v == ' ' then
            v = '\\normalcontrolspace'
        end
        context.type(v)
        context.space()
    end
end
\stopluacode

\starttabulate[|l|pl|]
\NC \bf name\NC \bf values \NC \NR
\NC tex     \NC \ctxlua{document.showprimitives('tex')    } \NC \NR
\NC core    \NC \ctxlua{document.showprimitives('core')   } \NC \NR
\NC etex    \NC \ctxlua{document.showprimitives('etex')   } \NC \NR
\NC luatex  \NC \ctxlua{document.showprimitives('luatex') } \NC \NR
\stoptabulate

Note that \type {luatex} does not contain \type {directlua}, as that is
considered to be a core primitive, along with all the \TEX82 primitives, so it is
part of the list that is returned from \type {'core'}.

% \type {'umath'} is a subset of \type {'luatex'} that covers the Unicode math
% primitives as it might be desired to handle the prefixing of that subset
% differently.

Running \type {tex.extraprimitives()} will give you the complete list of
primitives \type {-ini} startup. It is exactly equivalent to \type
{tex.extraprimitives("etex","luatex")}.

\subsubsection{\type {tex.primitives}}

\startfunctioncall
<table> t = tex.primitives()
\stopfunctioncall

This function returns a list of all primitives that \LUATEX\ knows about.

\subsection{Core functionality interfaces}

\subsubsection{\type {tex.badness}}

\startfunctioncall
<number> b = tex.badness(<number> t, <number> s)
\stopfunctioncall

This helper function is useful during linebreak calculations. \type {t} and \type
{s} are scaled values; the function returns the badness for when total \type {t}
is supposed to be made from amounts that sum to \type {s}. The returned number is
a reasonable approximation of $100(t/s)^3$;

\subsubsection{\type {tex.resetparagraph}}

This function resets the parameters that \TEX\ normally resets when a new paragraph
is seen.

\subsubsection{\type {tex.linebreak}}

\startfunctioncall
local <node> nodelist, <table> info =
       tex.linebreak(<node> listhead, <table> parameters)
\stopfunctioncall

The understood parameters are as follows:

\starttabulate[|l|l|p|]
\NC \bf name                 \NC \bf type        \NC \bf description \NC \NR
\NC pardir                   \NC string          \NC \NC \NR
\NC pretolerance             \NC number          \NC \NC \NR
\NC tracingparagraphs        \NC number          \NC \NC \NR
\NC tolerance                \NC number          \NC \NC \NR
\NC looseness                \NC number          \NC \NC \NR
\NC hyphenpenalty            \NC number          \NC \NC \NR
\NC exhyphenpenalty          \NC number          \NC \NC \NR
\NC pdfadjustspacing         \NC number          \NC \NC \NR
\NC adjdemerits              \NC number          \NC \NC \NR
\NC pdfprotrudechars         \NC number          \NC \NC \NR
\NC linepenalty              \NC number          \NC \NC \NR
\NC lastlinefit              \NC number          \NC \NC \NR
\NC doublehyphendemerits     \NC number          \NC \NC \NR
\NC finalhyphendemerits      \NC number          \NC \NC \NR
\NC hangafter                \NC number          \NC \NC \NR
\NC interlinepenalty         \NC number or table \NC if a table, then it is an array like \type {\interlinepenalties} \NC \NR
\NC clubpenalty              \NC number or table \NC if a table, then it is an array like \type {\clubpenalties} \NC \NR
\NC widowpenalty             \NC number or table \NC if a table, then it is an array like \type {\widowpenalties} \NC \NR
\NC brokenpenalty            \NC number          \NC \NC \NR
\NC emergencystretch         \NC number          \NC in scaled points \NC \NR
\NC hangindent               \NC number          \NC in scaled points \NC \NR
\NC hsize                    \NC number          \NC in scaled points \NC \NR
\NC leftskip                 \NC glue_spec node  \NC \NC \NR
\NC rightskip                \NC glue_spec node  \NC \NC \NR
\NC pdfignoreddimen          \NC number          \NC in scaled points \NC \NR
\NC parshape                 \NC table           \NC \NC \NR
\stoptabulate

Note that there is no interface for \type {\displaywidowpenalties}, you have to
pass the right choice for \type {widowpenalties} yourself.

The meaning of the various keys should be fairly obvious from the table (the
names match the \TEX\ and \PDFTEX\ primitives) except for the last 5 entries. The
four \type {pdf...line...} keys are ignored if their value equals \type
{pdfignoreddimen}.

It is your own job to make sure that \type {listhead} is a proper paragraph list:
this function does not add any nodes to it. To be exact, if you want to replace
the core line breaking, you may have to do the following (when you are not
actually working in the \type {pre_linebreak_filter} or \type {linebreak_filter}
callbacks, or when the original list starting at listhead was generated in
horizontal mode):

\startitemize
\startitem
    add an \quote {indent box} and perhaps a \type {local_par} node at the start
    (only if you need them)
\stopitem
\startitem
    replace any found final glue by an infinite penalty (or add such a penalty,
    if the last node is not a glue)
\stopitem
\startitem
    add a glue node for the \type {\parfillskip} after that penalty node
\stopitem
\startitem
    make sure all the \type {prev} pointers are OK
\stopitem
\stopitemize

The result is a node list, it still needs to be vpacked if you want to assign it
to a \type {\vbox}.

The returned \type {info} table contains four values that are all numbers:

\starttabulate[|l|p|]
\NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR
\NC prevgraf  \NC number of lines in the broken paragraph \NC \NR
\NC looseness \NC the actual looseness value in the broken paragraph \NC \NR
\NC demerits  \NC the total demerits of the chosen solution  \NC \NR
\stoptabulate

Note there are a few things you cannot interface using this function: You cannot
influence font expansion other than via \type {pdfadjustspacing}, because the
settings for that take place elsewhere. The same is true for hbadness and hfuzz
etc. All these are in the \type {hpack()} routine, and that fetches its own
variables via globals.

\subsubsection{\type {tex.shipout}}

\startfunctioncall
tex.shipout(<number> n)
\stopfunctioncall

Ships out box number \type {n} to the output file, and clears the box register.

\section[texconfig]{The \type {texconfig} table}

This is a table that is created empty. A startup \LUA\ script could
fill this table with a number of settings that are read out by
the executable after loading and executing the startup file.

\starttabulate[|lT|l|l|p|]
\NC \rmbf key             \NC \bf type \NC \bf default \NC \bf explanation \NC \NR
\NC kpse_init             \NC boolean  \NC true
\NC
    \type {false} totally disables \KPATHSEA\ initialisation, and enables
    interpretation of the following numeric key--value pairs. (only ever unset
    this if you implement {\it all\/} file find callbacks!)
\NC \NR
\NC
    shell_escape          \NC string   \NC \type {'f'} \NC
    Use \type {'y'} or \type {'t'} or \type {'1'} to enable \type {\write18}
    unconditionally, \type {'p'} to enable the commands that are listed in \type
    {shell_escape_commands}
\NC \NR
\NC
    shell_escape_commands \NC string \NC \NC Comma-separated list of command
    names that may be executed by \type {\write18} even if \type {shell_escape}
    is set to \type {'p'}. Do {\it not\/} use spaces around commas, separate any
    required command arguments by using a space, and use the \ASCII\ double quote
    (\type {"}) for any needed argument or path quoting
\NC \NR

\NC string_vacancies      \NC number   \NC  75000  \NC cf.\ web2c docs \NC \NR
\NC pool_free             \NC number   \NC   5000  \NC cf.\ web2c docs \NC \NR
\NC max_strings           \NC number   \NC  15000  \NC cf.\ web2c docs \NC \NR
\NC strings_free          \NC number   \NC    100  \NC cf.\ web2c docs \NC \NR
\NC nest_size             \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
\NC max_in_open           \NC number   \NC     15  \NC cf.\ web2c docs \NC \NR
\NC param_size            \NC number   \NC     60  \NC cf.\ web2c docs \NC \NR
\NC save_size             \NC number   \NC   4000  \NC cf.\ web2c docs \NC \NR
\NC stack_size            \NC number   \NC    300  \NC cf.\ web2c docs \NC \NR
\NC dvi_buf_size          \NC number   \NC  16384  \NC cf.\ web2c docs \NC \NR
\NC error_line            \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
\NC half_error_line       \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
\NC max_print_line        \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
\NC hash_extra            \NC number   \NC      0  \NC cf.\ web2c docs \NC \NR
\NC pk_dpi                \NC number   \NC     72  \NC cf.\ web2c docs \NC \NR
\NC trace_file_names      \NC boolean  \NC true
\NC
    \type {false} disables \TEX's normal file open|-|close feedback (the
    assumption is that callbacks will take care of that)
\NC \NR
\NC file_line_error       \NC boolean  \NC false
\NC
    do \type {file:line} style error messages
\NC \NR
\NC halt_on_error         \NC boolean  \NC false
\NC
    abort run on the first encountered error
\NC \NR
\NC formatname            \NC string   \NC
\NC
    if no format name was given on the command line, this key will be tested first
    instead of simply quitting
\NC \NR
\NC jobname               \NC string   \NC
\NC
    if no input file name was given on the command line, this key will be tested
    first instead of simply giving up
\NC \NR
\stoptabulate

Note: the numeric values that match web2c parameters are only used if \type
{kpse_init} is explicitly set to \type {false}. In all other cases, the normal
values from \type {texmf.cnf} are used.

\section{The \type {texio} library}

This library takes care of the low|-|level I/O interface: writing to the log file
and|/|or console.

\subsection{\type {texio.write}}

\startfunctioncall
texio.write(<string> target, <string> s, ...)
texio.write(<string> s, ...)
\stopfunctioncall

Without the \type {target} argument, writes all given strings to the same
location(s) \TEX\ writes messages to at this moment. If \type {\batchmode} is in
effect, it writes only to the log, otherwise it writes to the log and the
terminal. The optional \type {target} can be one of three possibilities: \type
{term}, \type {log} or \type {term and log}.

Note: If several strings are given, and if the first of these strings is or might
be one of the targets above, the \type {target} must be specified explicitly to
prevent \LUA\ from interpreting the first string as the target.

\subsection{\type {texio.write_nl}}

\startfunctioncall
texio.write_nl(<string> target, <string> s, ...)
texio.write_nl(<string> s, ...)
\stopfunctioncall

This function behaves like \type {texio.write}, but make sure that the given
strings will appear at the beginning of a new line. You can pass a single empty
string if you only want to move to the next line.

\subsection{\type {texio.setescape}}

You can disable \type {^^} escaping of control characters by passing a value of
zero.

\section{The \type {token} library}

\subsection{The scanner}

The token library provides means to intercept the input and deal with it at the
\LUA\ level. The library provides a basic scanner infrastructure that can be used
to write macros that accept a wide range of arguments. This interface is on
purpose kept general and as performance is quite ok one can build additional
parsers without too much overhead. It's up to macro package writers to see how
they can benefit from this as the main principle behind \LUATEX\ is to provide a
minimal set of tools and no solutions. The functions provided in the \type
{token} namespace are given in the next table:

\starttabulate[|lT|lT|p|]
\NC \bf function \NC \bf argument       \NC \bf result \NC \NR
\HL
\NC is_token     \NC token              \NC checks if the given argument is a token userdatum \NC \NR
\NC get_next     \NC                    \NC returns the next token in the input \NC \NR
\NC scan_keyword \NC string             \NC returns true if the given keyword is gobbled \NC \NR
\NC scan_int     \NC                    \NC returns a number \NC \NR
\NC scan_dimen   \NC infinity, mu-units \NC returns a number representing a dimension and or two numbers being the filler and order \NC \NR
\NC scan_glue    \NC mu-units           \NC returns a glue spec node \NC \NR
\NC scan_toks    \NC definer, expand    \NC returns a table of tokens token list (this can become a linked list in later releases) \NC \NR
\NC scan_code    \NC bitset             \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR
\NC scan_string  \NC                    \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR
\NC scan_word    \NC                    \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR
\NC scan_csname  \NC                    \NC returns \type {foo} after scanning \type {\foo} \NC \NR
\NC set_macro    \NC see below          \NC assign a macro \NC \NR
\NC create       \NC                    \NC returns a userdata token object of the given control sequence name (or character); this interface can change  \NC \NR
\stoptabulate

The scanners can be considered stable apart from the one scanning for a token.
This is because futures releases can return a linked list instead of a table (as
with nodes). The \type {scan_code} function takes an optional number, the \type
{keyword} function a normal \LUA\ string. The \type {infinity} boolean signals
that we also permit \type {fill} as dimension and the \type {mu-units} flags the
scanner that we expect math units. When scanning tokens we can indicate that we
are defining a macro, in which case the result will also provide information
about what arguments are expected and in the result this is separated from the
meaning by a separator token. The \type {expand} flag determines if the list will
be expanded.

The string scanner scans for something between curly braces and expands on the
way, or when it sees a control sequence it will return its meaning. Otherwise it
will scan characters with catcode \type {letter} or \type {other}. So, given the
following definition:

\startbuffer
\def\bar{bar}
\def\foo{foo-\bar}
\stopbuffer

\typebuffer \getbuffer

we get:

\starttabulate[|l|Tl|l|]
\NC \type {\directlua{token.scan_string()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} {foo} \NC full expansion \NR
\NC \type {\directlua{token.scan_string()}foo}   \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} foo   \NC letters and others \NR
\NC \type {\directlua{token.scan_string()}\foo}  \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")}\foo   \NC meaning \NR
\stoptabulate

The \type {\foo} case only gives the meaning, but one can pass an already
expanded definition (\type {\edef}'d). In the case of the braced variant one can of
course use the \type {\detokenize} and \type {\unexpanded} primitives as there we
do expand.

The \type {scan_word} scanner can be used to implement for instance a number scanner:

\starttyping
function token.scan_number(base)
    return tonumber(token.scan_word(),base)
end
\stoptyping

This scanner accepts any valid \LUA\ number so it is a way to pick up floats
in the input.

The creator function can be used as follows:

\starttyping
local t = token.create("relax")
\stoptyping

This gives back a token object that has the properties of the \type {\relax}
primitive. The possible properties of tokens are:

\starttabulate[|lT|p|]
\NC command    \NC a number representing the internal command number \NC \NR
\NC cmdname    \NC the type of the command (for instance the catcode in case of a
                   character or the classifier that determines the internal
                   treatment \NC \NR
\NC csname     \NC the associated control sequence (if applicable) \NC \NR
\NC id         \NC the unique id of the token \NC \NR
%NC tok        \NC \NC \NR % might change
\NC active     \NC a boolean indicating the active state of the token \NC \NR
\NC expandable \NC a boolean indicating if the token (macro) is expandable \NC \NR
\NC protected  \NC a boolean indicating if the token (macro) is protected \NC \NR
\stoptabulate

The numbers that represent a catcode are the same as in \TEX\ itself, so using
this information assumes that you know a bit about \TEX's internals. The other
numbers and names are used consistently but are not frozen. So, when you use them
for comparing you can best query a known primitive or character first to see the
values.

More interesting are the scanners. You can use the \LUA\ interface as follows:

\starttyping
\directlua {
    function mymacro(n)
        ...
    end
}

\def\mymacro#1{%
    \directlua {
        mymacro(\number\dimexpr#1)
    }%
}

\mymacro{12pt}
\mymacro{\dimen0}
\stoptyping

You can also do this:

\starttyping
\directlua {
    function mymacro()
        local d = token.scan_dimen()
        ...
    end
}

\def\mymacro{%
    \directlua {
        mymacro()
    }%
}

\mymacro 12pt
\mymacro \dimen0
\stoptyping

It is quite clear from looking at the code what the first method needs as
argument(s). For the second method you need to look at the \LUA\ code to see what
gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from
the input stream.

In the first case the input is tokenized and then turned into a string when it's
passed to \LUA\ where it gets interpreted. In the second case only a function
call gets interpreted but then the input is picked up by explicitly calling the
scanner functions. These return proper \LUA\ variables so no further conversion
has to be done. This is more efficient but in practice (given what \TEX\ has to
do) this effect should not be overestimated. For numbers and dimensions it saves a
bit but for passing strings conversion to and from tokens has to be done anyway
(although we can probably speed up the process in later versions if needed).

\subsection{Macros}

The \type {set_macro} function can get upto 4 arguments:

\starttyping
setmacro("csname","content")
setmacro("csname","content","global")
setmacro("csname")
\stoptyping

You can pass a catcodetable identifier as first argument:

\starttyping
setmacro(catcodetable,"csname","content")
setmacro(catcodetable,"csname","content","global")
setmacro(catcodetable,"csname")
\stoptyping

The results are like:

\starttyping
 \def\csname{content}
\gdef\csname{content}
 \def\csname{}
\stoptyping

\subsection{Pushing back}

There is a (for now) experimental putter:

\starttyping
local t1 = token.get_next()
local t2 = token.get_next()
local t3 = token.get_next()
local t4 = token.get_next()
-- watch out, we flush in sequence
token.put_next { t1, t2 }
-- but this one gets pushed in front
token.put_next ( t3, t4 )
\stoptyping

When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table
with tokens or a list of tokens.

\subsection{Nota bene}

When scanning for the next token you need to keep in mind that we're not scanning
like \TEX\ does: expanding, changing modes and doing things as it goes. When we
scan with \LUA\ we just pick up tokens. Say that we have:

\starttyping
\bar
\stoptyping

but \type {\bar} is undefined. Normally \TEX\ will then issue an error message.
However, when we have:

\starttyping
\def\foo{\bar}
\stoptyping

We get no error, unless we expand \type {\foo} while \type {\bar} is still
undefined. What happens is that as soon as \TEX\ sees an undefined macro it will
create a hash entry and when later it gets defined that entry will be reused. So,
\type {\bar} really exists but can be in an undefined state.

\startbuffer[demo]
bar        : \directlua{tex.print(token.scan_csname())}\bar
foo        : \directlua{tex.print(token.scan_csname())}\foo
myfirstbar : \directlua{tex.print(token.scan_csname())}\myfirstbar
\stopbuffer

\startlines
\getbuffer[demo]
\stoplines

This was entered as:

\typebuffer[demo]

The reason that you see \type {bar} reported and not \type {myfirstbar} is that
\type {\bar} was already used in a previous paragraph.

If we now say:

\startbuffer
\def\foo{}
\stopbuffer

\typebuffer \getbuffer

we get:

\startlines
\getbuffer[demo]
\stoplines

And if we say

\startbuffer
\def\foo{\bar}
\stopbuffer

\typebuffer \getbuffer

we get:

\startlines
\getbuffer[demo]
\stoplines

When scanning from \LUA\ we are not in a mode that defines (undefined) macros at
all. There we just get the real primitive undefined macro token.

\startbuffer
\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\myfirstbar
\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mysecondbar
\directlua{local t = token.get_next() tex.print(t.id.." "..t.tok)}\mythirdbar
\stopbuffer

\startlines
\getbuffer
\stoplines

This was generated with:

\typebuffer

So, we do get a unique token because after all we need some kind of \LUA\ object
that can be used and garbage collected, but it is basically the same one,
representing an undefined control sequence.

\section{The \type {kpse} library}

This library provides two separate, but nearly identical interfaces to the
\KPATHSEA\ file search functionality: there is a \quote {normal} procedural
interface that shares its kpathsea instance with \LUATEX\ itself, and an object
oriented interface that is completely on its own.

\subsection{\type {kpse.set_program_name} and \type {kpse.new}}

Before the search library can be used at all, its database has to be initialized.
There are three possibilities, two of which belong to the procedural interface.

First, when \LUATEX\ is used to typeset documents, this initialization happens
automatically and the \KPATHSEA\ executable and program names are set to \type
{luatex} (that is, unless explicitly prohibited by the user's startup script.
See~\in {section} [init] for more details).

Second, in \TEXLUA\ mode, the initialization has to be done explicitly via the
\type {kpse.set_program_name} function, which sets the \KPATHSEA\ executable
(and optionally program) name.

\startfunctioncall
kpse.set_program_name(<string> name)
kpse.set_program_name(<string> name, <string> progname)
\stopfunctioncall

The second argument controls the use of the \quote {dotted} values in the \type
{texmf.cnf} configuration file, and defaults to the first argument.

Third, if you prefer the object oriented interface, you have to call a different
function. It has the same arguments, but it returns a userdata variable.

\startfunctioncall
local kpathsea = kpse.new(<string> name)
local kpathsea = kpse.new(<string> name, <string> progname)
\stopfunctioncall

Apart from these two functions, the calling conventions of the interfaces are
identical. Depending on the chosen interface, you either call \type
{kpse.find_file()} or \type {kpathsea:find_file()}, with identical arguments and
return vales.

\subsection{\type {find_file}}

The most often used function in the library is \type {find_file}:

\startfunctioncall
<string> f = kpse.find_file(<string> filename)
<string> f = kpse.find_file(<string> filename, <string> ftype)
<string> f = kpse.find_file(<string> filename, <boolean> mustexist)
<string> f = kpse.find_file(<string> filename, <string> ftype, <boolean> mustexist)
<string> f = kpse.find_file(<string> filename, <string> ftype, <number> dpi)
\stopfunctioncall

Arguments:

\startitemize[intro]

\sym{filename}

the name of the file you want to find, with or without extension.

\sym{ftype}

maps to the \type {-format} argument of \KPSEWHICH. The supported \type {ftype}
values are the same as the ones supported by the standalone \type {kpsewhich}
program: \startluacode
    local list = {
        "afm",
        "base",
        "bib",
        "bitmap font",
        "bst",
        "cid maps",
        "clua",
        "cmap files",
        "cnf",
        "cweb",
        "dvips config",
        "enc files",
        "fmt",
        "font feature files",
        "gf",
        "graphic|/|figure",
        "ist",
        "lig files",
        "ls-R",
        "lua",
        "map",
        "mem",
        "MetaPost support",
        "mf",
        "mfpool",
        "mft",
        "misc fonts",
        "mlbib",
        "mlbst",
        "mp",
        "mppool",
        "ocp",
        "ofm",
        "opentype fonts",
        "opl",
        "other binary files",
        "other text files",
        "otp",
        "ovf",
        "ovp",
        "pdftex config",
        "pk",
        "PostScript header",
        "subfont definition files",
        "tex",
        "TeX system documentation",
        "TeX system sources",
        "texmfscripts",
        "texpool",
        "tfm",
        "Troff fonts",
        "truetype fonts",
        "type1 fonts",
        "type42 fonts",
        "vf",
        "web",
        "web2c files",
    }
    table.sort(list)
    context("{\\tttf \letterpercent, t}",list)
\stopluacode

The default type is \type {tex}. Note: this is different from \KPSEWHICH, which
tries to deduce the file type itself from looking at the supplied extension.

\sym{mustexist}

is similar to \KPSEWHICH's \type {-must-exist}, and the default is \type {false}.
If you specify \type {true} (or a non|-|zero integer), then the \KPSE\ library
will search the disk as well as the \type {ls-R} databases.

\sym{dpi}

This is used for the size argument of the formats \type {pk}, \type {gf}, and
\type {bitmap font}. \stopitemize


\subsection{\type {lookup}}

A more powerful (but slower) generic method for finding files is also available.
It returns a string for each found file.

\startfunctioncall
<string> f, ... = kpse.lookup(<string> filename, <table> options)
\stopfunctioncall

The options match commandline arguments from \type {kpsewhich}:

\starttabulate[|l|l|p|]
\NC \rmbf key \NC \rmbf type \NC \rmbf description \NC \NR
\NC debug     \NC number     \NC set debugging flags for this lookup\NC     \NR
\NC format    \NC string     \NC use specific file type (see list above)\NC \NR
\NC dpi       \NC number     \NC use this resolution for this lookup; default 600\NC \NR
\NC path      \NC string     \NC search in the given path\NC \NR
\NC all       \NC boolean    \NC output all matches, not just the first\NC \NR
\NC mustexist \NC boolean    \NC search the disk as well as ls-R if necessary\NC \NR
\NC mktexpk   \NC boolean    \NC disable/enable mktexpk generation for this lookup\NC \NR
\NC mktextex  \NC boolean    \NC disable/enable mktextex generation for this lookup\NC \NR
\NC mktexmf   \NC boolean    \NC disable/enable mktexmf generation for this lookup\NC \NR
\NC mktextfm  \NC boolean    \NC disable/enable mktextfm generation for this lookup\NC \NR
\NC subdir    \NC string
                  or table   \NC only output matches whose directory part
                                 ends with the given string(s) \NC \NR
\stoptabulate

\subsection{\type {init_prog}}

Extra initialization for programs that need to generate bitmap fonts.

\startfunctioncall
kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode)
kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode, <string> fallback)
\stopfunctioncall

\subsection{\type {readable_file}}

Test if an (absolute) file name is a readable file.

\startfunctioncall
<string> f = kpse.readable_file(<string> name)
\stopfunctioncall

The return value is the actual absolute filename you should use, because the disk
name is not always the same as the requested name, due to aliases and
system|-|specific handling under e.g.\ \MSDOS. Returns \type {nil} if the file
does not exist or is not readable.

\subsection{\type {expand_path}}

Like kpsewhich's \type {-expand-path}:

\startfunctioncall
<string> r = kpse.expand_path(<string> s)
\stopfunctioncall

\subsection{\type {expand_var}}

Like kpsewhich's  \type {-expand-var}:

\startfunctioncall
<string> r = kpse.expand_var(<string> s)
\stopfunctioncall

\subsection{\type {expand_braces}}

Like kpsewhich's \type {-expand-braces}:

\startfunctioncall
<string> r = kpse.expand_braces(<string> s)
\stopfunctioncall

\subsection{\type {show_path}}

Like kpsewhich's \type {-show-path}:

\startfunctioncall
<string> r = kpse.show_path(<string> ftype)
\stopfunctioncall


\subsection{\type {var_value}}

Like kpsewhich's \type {-var-value}:

\startfunctioncall
<string> r = kpse.var_value(<string> s)
\stopfunctioncall

\subsection{\type {version}}

Returns the kpathsea version string.

\startfunctioncall
<string> r = kpse.version()
\stopfunctioncall

\stopchapter

\stopcomponent