new beta-0.90.0

[luatex.git] / manual / luatex-libraries.tex

% language=uk

\environment luatex-style
\environment luatex-logos

% HH: to be checked

\startcomponent luatex-libraries

\startchapter[reference=libraries,title={\LUATEX\ \LUA\ Libraries}]

The implied use of the built|-|in \LUA\ modules \type {epdf}, \type {fontloader},
\type {mplib}, and \type {pdfscanner} is deprecated. If you want to use these,
please start your source file with a proper \type {require} line. In the future,
\LUATEX\ will switch to loading these modules on demand.

The interfacing between \TEX\ and \LUA\ is facilitated by a set of library
modules. The \LUA\ libraries in this chapter are all defined and initialized by
the \LUATEX\ executable. Together, they allow \LUA\ scripts to query and change a
number of \TEX's internal variables, run various internal \TEX\ functions, and
set up \LUATEX's hooks to execute \LUA\ code.

The following sections are in alphabetical order. For any callback (and
manipulation of nodes) the following is true: you have a lot of freedom which
also means that you can mess up the node lists and nodes themselves. So, a bit of
defensive programming doesn't hurt. A crash can happen when you spoil things or
when \LUATEX\ can recognize the issue, a panic exit will happen. Don't bother the
team with such issues.

\section{The \type {callback} library}

This library has functions that register, find and list callbacks. Callbacks are
\LUA\ functions that are called in well defined places. There are two kind of
callbacks: those that mix with existing functionality, and those that (when
enabled) replace functionality. In mosty cases the second category is expected to
behave similar to the built in functionality because in a next step specific
data is expected. For instance, you can replace the hyphenation routine. The
function gets a list that can be hyphenated (or not). The final list should be
valid and is (normally) used for constructing a paragraph. Another function can
replace the ligature builder and|/|or kerner. Doing something else is possible
but in the end might not give the user the expected outcome.

The first thing you need to do is registering a callback:

\startfunctioncall
id, error = callback.register (<string> callback_name, <function> func)
id, error = callback.register (<string> callback_name, nil)
id, error = callback.register (<string> callback_name, false)
\stopfunctioncall

Here the \syntax {callback_name} is a predefined callback name, see below. The
function returns the internal \type {id} of the callback or \type {nil}, if the
callback could not be registered. In the latter case, \type {error} contains an
error message, otherwise it is \type {nil}.

\LUATEX\ internalizes the callback function in such a way that it does not matter
if you redefine a function accidentally.

Callback assignments are always global. You can use the special value \type {nil}
instead of a function for clearing the callback.

For some minor speed gain, you can assign the boolean \type {false} to the
non|-|file related callbacks, doing so will prevent \LUATEX\ from executing
whatever it would execute by default (when no callback function is registered at
all). Be warned: this may cause all sorts of grief unless you know {\em exactly}
what you are doing!

\startfunctioncall
<table> info = callback.list()
\stopfunctioncall

The keys in the table are the known callback names, the value is a boolean where
\type {true} means that the callback is currently set (active).

\startfunctioncall
<function> f = callback.find (callback_name)
\stopfunctioncall

If the callback is not set, \type {callback.find} returns \type {nil}.

\subsection{File discovery callbacks}

The behaviour documented in this subsection is considered stable in the sense that
there will not be backward|-|incompatible changes any more.

\subsubsection{\type {find_read_file} and \type {find_write_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<number> id_number, <string> asked_name)
\stopfunctioncall

Arguments:

\startitemize

\sym{id_number}

This number is zero for the log or \type {\input} files. For \TEX's \type {\read}
or \type {\write} the number is incremented by one, so \type {\read0} becomes~1.

\sym{asked_name}

This is the user|-|supplied filename, as found by \type {\input}, \type {\openin}
or \type {\openout}.

\stopitemize

Return value:

\startitemize

\sym{actual_name}

This is the filename used. For the very first file that is read in by \TEX, you
have to make sure you return an \type {actual_name} that has an extension and
that is suitable for use as \type {jobname}. If you don't, you will have to
manually fix the name of the log file and output file after \LUATEX\ is finished,
and an eventual format filename will become mangled. That is because these file
names depend on the jobname.

You have to return \type {nil} if the file cannot be found.

\stopitemize

\subsubsection{\type {find_font_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<string> asked_name)
\stopfunctioncall

The \type {asked_name} is an \OTF\ or \TFM\ font metrics file.

Return \type {nil} if the file cannot be found.

\subsubsection{\type {find_output_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<string> asked_name)
\stopfunctioncall

The \type {asked_name} is the \PDF\ or \DVI\ file for writing.

\subsubsection{\type {find_format_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<string> asked_name)
\stopfunctioncall

The \type {asked_name} is a format file for reading (the format file for writing
is always opened in the current directory).

\subsubsection{\type {find_vf_file}}

Like \type {find_font_file}, but for virtual fonts. This applies to both \ALEPH's
\OVF\ files and traditional Knuthian \VF\ files.

\subsubsection{\type {find_map_file}}

Like \type {find_font_file}, but for map files.

\subsubsection{\type {find_enc_file}}

Like \type {find_font_file}, but for enc files.

\subsubsection{\type {find_sfd_file}}

Like \type {find_font_file}, but for subfont definition files.

\subsubsection{\type {find_pk_file}}

Like \type {find_font_file}, but for pk bitmap files. This callback takes two
arguments: \type {name} and \type {dpi}. In your callback you can decide to
look for:

\starttyping
<base res>dpi/<fontname>.<actual res>pk
\stoptyping

but other strategies are possible. It is up to you to find a \quote {reasonable}
bitmap file to go with that specification.

\subsubsection{\type {find_data_file}}

Like \type {find_font_file}, but for embedded files (\type {\pdfobj file '...'}).

\subsubsection{\type {find_opentype_file}}

Like \type {find_font_file}, but for \OPENTYPE\ font files.

\subsubsection{\type {find_truetype_file} and \type {find_type1_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<string> asked_name)
\stopfunctioncall

The \type {asked_name} is a font file. This callback is called while \LUATEX\ is
building its internal list of needed font files, so the actual timing may
surprise you. Your return value is later fed back into the matching \type
{read_file} callback.

Strangely enough, \type {find_type1_file} is also used for \OPENTYPE\ (\OTF)
fonts.

\subsubsection{\type {find_image_file}}

Your callback function should have the following conventions:

\startfunctioncall
<string> actual_name = function (<string> asked_name)
\stopfunctioncall

The \type {asked_name} is an image file. Your return value is used to open a file
from the hard disk, so make sure you return something that is considered the name
of a valid file by your operating system.

\subsection[iocallback]{File reading callbacks}

The behavior documented in this subsection is considered stable in the sense that
there will not be backward-incompatible changes any more.

\subsubsection{\type {open_read_file}}

Your callback function should have the following conventions:

\startfunctioncall
<table> env = function (<string> file_name)
\stopfunctioncall

Argument:

\startitemize

\sym{file_name}

The filename returned by a previous \type {find_read_file} or the return value of
\type {kpse.find_file()} if there was no such callback defined.

\stopitemize

Return value:

\startitemize

\sym{env}

This is a table containing at least one required and one optional callback
function for this file. The required field is \type {reader} and the associated
function will be called once for each new line to be read, the optional one is
\type {close} that will be called once when \LUATEX\ is done with the file.

\LUATEX\ never looks at the rest of the table, so you can use it to store your
private per|-|file data. Both the callback functions will receive the table as
their only argument.

\stopitemize

\subsubsubsection{\type {reader}}

\LUATEX\ will run this function whenever it needs a new input line from the file.

\startfunctioncall
function(<table> env)
    return <string> line
end
\stopfunctioncall

Your function should return either a string or \type {nil}. The value \type {nil}
signals that the end of file has occurred, and will make \TEX\ call the optional
\type {close} function next.

\subsubsubsection{\type {close}}

\LUATEX\ will run this optional function when it decides to close the file.

\startfunctioncall
function(<table> env)
end
\stopfunctioncall

Your function should not return any value.

\subsubsection{General file readers}

There is a set of callbacks for the loading of binary data files. These all use
the same interface:

\startfunctioncall
function(<string> name)
    return <boolean> success, <string> data, <number> data_size
end
\stopfunctioncall

The \type {name} will normally be a full path name as it is returned by either
one of the file discovery callbacks or the internal version of \type
{kpse.find_file()}.

\startitemize

\sym{success}

Return \type {false} when a fatal error occurred (e.g.\ when the file cannot be
found, after all).

\sym{data}

The bytes comprising the file.

\sym{data_size}

The length of the \type {data}, in bytes.

\stopitemize

Return an empty string and zero if the file was found but there was a
reading problem.

The list of functions is as follows:

\starttabulate[|l|p|]
\NC \type {read_font_file}     \NC ofm or tfm files \NC \NR
\NC \type {read_vf_file}       \NC virtual fonts \NC \NR
\NC \type {read_map_file}      \NC map files \NC \NR
\NC \type {read_enc_file}      \NC encoding files \NC \NR
\NC \type {read_sfd_file}      \NC subfont definition files \NC \NR
\NC \type {read_pk_file}       \NC pk bitmap files \NC \NR
\NC \type {read_data_file}     \NC embedded files (\type {\pdfobj file ...}) \NC \NR
\NC \type {read_truetype_file} \NC \TRUETYPE\ font files \NC \NR
\NC \type {read_type1_file}    \NC \TYPEONE\ font files \NC \NR
\NC \type {read_opentype_file} \NC \OPENTYPE\ font files \NC \NR
\stoptabulate

\subsection{Data processing callbacks}

\subsubsection{\type {process_input_buffer}}

This callback allows you to change the contents of the line input buffer just
before \LUATEX\ actually starts looking at it.

\startfunctioncall
function(<string> buffer)
    return <string> adjusted_buffer
end
\stopfunctioncall

If you return \type {nil}, \LUATEX\ will pretend like your callback never
happened. You can gain a small amount of processing time from that. This callback
does not replace any internal code.

\subsubsection{\type {process_output_buffer}}

This callback allows you to change the contents of the line output buffer just
before \LUATEX\ actually starts writing it to a file as the result of a \type
{\write} command. It is only called for output to an actual file (that is,
excluding the log, the terminal, and \type {\write18} calls).

\startfunctioncall
function(<string> buffer)
    return <string> adjusted_buffer
end
\stopfunctioncall

If you return \type {nil}, \LUATEX\ will pretend like your callback never
happened. You can gain a small amount of processing time from that. This callback
does not replace any internal code.

\subsubsection{\type {process_jobname}}

This callback allows you to change the jobname given by \type {\jobname} in \TEX\
and \type {tex.jobname} in Lua. It does not affect the internal job name or the
name of the output or log files.

\startfunctioncall
function(<string> jobname)
    return <string> adjusted_jobname
end
\stopfunctioncall

The only argument is the actual job name; you should not use \type {tex.jobname}
inside this function or infinite recursion may occur. If you return \type {nil},
\LUATEX\ will pretend your callback never happened. This callback does not
replace any internal code.

\subsection{Node list processing callbacks}

The description of nodes and node lists is in~\in{chapter}[nodes].

\subsubsection{\type {contribute_filter}}

This callback is called when \LUATEX\ adds contents to list:

\startfunctioncall
function(<string> extrainfo)
end
\stopfunctioncall

The string reports the group code. From this you can deduce from
what list you can give a treat.

\starttabulate
\NC \bf group codes        \NC \bf pointer \NC \NR
\HL
\NC\type {pre_box}         \NC \type {contrib_head}    \NC \NR
\NC\type {pre_adjust_tail} \NC \type {pre_adjust_head} \NC \NR
\NC\type {just}            \NC \type {just_box}        \NC \NR
\NC\type {adjust_tail}     \NC \type {adjust_head}     \NC \NR
\stoptabulate

\subsubsection{\type {buildpage_filter}}

This callback is called whenever \LUATEX\ is ready to move stuff to the main
vertical list. You can use this callback to do specialized manipulation of the
page building stage like imposition or column balancing.

\startfunctioncall
function(<string> extrainfo)
end
\stopfunctioncall

The string \type {extrainfo} gives some additional information about what \TEX's
state is with respect to the \quote {current page}. The possible values are:

\starttabulate[|lT|p|]
\NC \ssbf value     \NC \bf explanation                           \NC \NR
\NC alignment       \NC a (partial) alignment is being added      \NC \NR
\NC after_output    \NC an output routine has just finished       \NC \NR
\NC box             \NC a typeset box is being added              \NC \NR
%NC pre_box         \NC interline material is being added         \NC \NR
%NC adjust          \NC \type {\vadjust} material is being added  \NC \NR
\NC new_graf        \NC the beginning of a new paragraph          \NC \NR
\NC vmode_par       \NC \type {\par} was found in vertical mode   \NC \NR
\NC hmode_par       \NC \type {\par} was found in horizontal mode \NC \NR
\NC insert          \NC an insert is added                        \NC \NR
\NC penalty         \NC a penalty (in vertical mode)              \NC \NR
\NC before_display  \NC immediately before a display starts       \NC \NR
\NC after_display   \NC a display is finished                     \NC \NR
\NC end             \NC \LUATEX\ is terminating (it's all over)   \NC \NR
\stoptabulate

This callback does not replace any internal code.

\subsubsection{\type {pre_linebreak_filter}}

This callback is called just before \LUATEX\ starts converting a list of nodes
into a stack of \type {\hbox}es, after the addition of \type {\parfillskip}.

\startfunctioncall
function(<node> head, <string> groupcode)
    return true | false | <node> newhead
end
\stopfunctioncall

The string called \type {groupcode} identifies the nodelist's context within
\TEX's processing. The range of possibilities is given in the table below, but
not all of those can actually appear in \type {pre_linebreak_filter}, some are
for the \type {hpack_filter} and \type {vpack_filter} callbacks that will be
explained in the next two paragraphs.

\starttabulate[|lT|p|]
\NC \ssbf value   \NC \bf explanation                                 \NC \NR
\NC <empty>       \NC main vertical list                              \NC \NR
\NC hbox          \NC \type {\hbox} in horizontal mode                \NC \NR
\NC adjusted_hbox \NC \type {\hbox} in vertical mode                  \NC \NR
\NC vbox          \NC \type {\vbox}                                   \NC \NR
\NC vtop          \NC \type {\vtop}                                   \NC \NR
\NC align         \NC \type {\halign} or \type {\valign}              \NC \NR
\NC disc          \NC discretionaries                                 \NC \NR
\NC insert        \NC packaging an insert                             \NC \NR
\NC vcenter       \NC \type {\vcenter}                                \NC \NR
\NC local_box     \NC \type {\localleftbox} or \type {\localrightbox} \NC \NR
\NC split_off     \NC top of a \type {\vsplit}                        \NC \NR
\NC split_keep    \NC remainder of a \type {\vsplit}                  \NC \NR
\NC align_set     \NC alignment cell                                  \NC \NR
\NC fin_row       \NC alignment row                                   \NC \NR
\stoptabulate

As for all the callbacks that deal with nodes, the return value can be one of
three things:

\startitemize
\startitem
    boolean \type {true} signals succesfull processing
\stopitem
\startitem
    \type {<node>} signals that the \quote {head} node should be replaced by the
    returned node
\stopitem
\startitem
    boolean \type {false} signals that the \quote {head} node list should be
    ignored and flushed from memory
\stopitem
\stopitemize

This callback does not replace any internal code.

\subsubsection{\type {linebreak_filter}}

This callback replaces \LUATEX's line breaking algorithm.

\startfunctioncall
function(<node> head, <boolean> is_display)
    return <node> newhead
end
\stopfunctioncall

The returned node is the head of the list that will be added to the main vertical
list, the boolean argument is true if this paragraph is interrupted by a
following math display.

If you return something that is not a \type {<node>}, \LUATEX\ will apply the
internal linebreak algorithm on the list that starts at \type {<head>}.
Otherwise, the \type {<node>} you return is supposed to be the head of a list of
nodes that are all allowed in vertical mode, and at least one of those has to
represent a hbox. Failure to do so will result in a fatal error.

Setting this callback to \type {false} is possible, but dangerous, because it is
possible you will end up in an unfixable \quote {deadcycles loop}.

\subsubsection{\type {append_to_vlist_filter}}

This callback is called whenever \LUATEX\ adds a box to a vertical list:

\startfunctioncall
function(<node> box, <string> locationcode, <number prevdepth>,
    <boolean> mirrored)
    return list, prevdepth
end
\stopfunctioncall

It is ok to return nothing in which case you also need to flush the box or deal
with it yourself. The prevdepth is also optional. Locations are \type {box},
\type {alignment}, \type {equation}, \type {equation_number} and \type
{post_linebreak}.

\subsubsection{\type {post_linebreak_filter}}

This callback is called just after \LUATEX\ has converted a list of nodes into a
stack of \type {\hbox}es.

\startfunctioncall
function(<node> head, <string> groupcode)
    return true | false | <node> newhead
end
\stopfunctioncall

This callback does not replace any internal code.

\subsubsection{\type {hpack_filter}}

This callback is called when \TEX\ is ready to start boxing some horizontal mode
material. Math items and line boxes are ignored at the moment.

\startfunctioncall
function(<node> head, <string> groupcode, <number> size,
         <string> packtype [, <string> direction] [, <node> attributelist])
    return true | false | <node> newhead
end
\stopfunctioncall

The \type {packtype} is either \type {additional} or \type {exactly}. If \type
{additional}, then the \type {size} is a \type {\hbox spread ...} argument. If
\type {exactly}, then the \type {size} is a \type {\hbox to ...}. In both cases,
the number is in scaled points.

The \type {direction} is either one of the three-letter direction specifier
strings, or \type {nil}.

This callback does not replace any internal code.

\subsubsection{\type {vpack_filter}}

This callback is called when \TEX\ is ready to start boxing some vertical mode
material. Math displays are ignored at the moment.

This function is very similar to the \type {hpack_filter}. Besides the fact
that it is called at different moments, there is an extra variable that matches
\TEX's \type {\maxdepth} setting.

\startfunctioncall
function(<node> head, <string> groupcode, <number> size, <string> packtype,
        <number> maxdepth [, <string> direction] [, <node> attributelist]))
    return true | false | <node> newhead
end
\stopfunctioncall

This callback does not replace any internal code.

\subsubsection{\type {hpack_quality}}

This callback can be used to intercept the overfull messages that can result from
packing a horizontal list (as happens in the par builder). The function takes a
few arguments:

\startfunctioncall
function(<string> incident, <number> detail, <node> head, <number> first,
         <number> last)
    return <node> whatever
end
\stopfunctioncall

The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
\type {tight}. The detail is either the amount of overflow in case of \type
{overfull}, or the badness otherwise. The head is the list that is constructed
(when protrusion or expansion is enabled, this is an intermediate list).
Optionally you can return a node, for instance an overfull rule indicator. That
node will be appended to the list (just like \TEX's own rule would).

\subsubsection{\type {vpack_quality}}

This callback can be used to intercept the overfull messages that can result from
packing a vertical list (as happens in the page builder). The function takes a
few arguments:

\startfunctioncall
function(<string> incident, <number> detail, <node> head, <number> first,
         <number> last)
end
\stopfunctioncall

The incident is one of \type {overfull}, \type {underfull}, \type {loose} or
\type {tight}. The detail is either the amount of overflow in case of \type
{overfull}, or the badness otherwise. The head is the list that is constructed.

\subsubsection{\type {process_rule}}

This is an experimental callback. It can be used with rules of subtype~4
(user). The callback gets three arguments: the node, the width and the
height. The callback can use \type {pdf.print} to write code to the \PDF\
file but beware of not messing up the final result. No checking is done.

\subsubsection{\type {pre_output_filter}}

This callback is called when \TEX\ is ready to start boxing the box 255 for \type
{\output}.

\startfunctioncall
function(<node> head, <string> groupcode, <number> size, <string> packtype,
        <number> maxdepth [, <string> direction])
    return true | false | <node> newhead
end
\stopfunctioncall

This callback does not replace any internal code.

\subsubsection{\type {hyphenate}}

\startfunctioncall
function(<node> head, <node> tail)
end
\stopfunctioncall

No return values. This callback has to insert discretionary nodes in the node
list it receives.

Setting this callback to \type {false} will prevent the internal discretionary
insertion pass.

\subsubsection{\type {ligaturing}}

\startfunctioncall
function(<node> head, <node> tail)
end
\stopfunctioncall

No return values. This callback has to apply ligaturing to the node list it
receives.

You don't have to worry about return values because the \type {head} node that is
passed on to the callback is guaranteed not to be a glyph_node (if need be, a
temporary node will be prepended), and therefore it cannot be affected by the
mutations that take place. After the callback, the internal value of the \quote
{tail of the list} will be recalculated.

The \type {next} of \type {head} is guaranteed to be non-nil.

The \type {next} of \type {tail} is guaranteed to be nil, and therefore the
second callback argument can often be ignored. It is provided for orthogonality,
and because it can sometimes be handy when special processing has to take place.

Setting this callback to \type {false} will prevent the internal ligature
creation pass.

You must not ruin the node list. For instance, the head normally is a local par node,
and the tail a glue. Messing too much can push \LUATEX\ into panic mode.

\subsubsection{\type {kerning}}

\startfunctioncall
function(<node> head, <node> tail)
end
\stopfunctioncall

No return values. This callback has to apply kerning between the nodes in the
node list it receives. See \type {ligaturing} for calling conventions.

Setting this callback to \type {false} will prevent the internal kern insertion
pass.

You must not ruin the node list. For instance, the head normally is a local par node,
and the tail a glue. Messing too much can push \LUATEX\ into panic mode.

\subsubsection{\type {insert_local_par}}

Each paragraph starts with a local par node that keeps track of for instance
the direction. You can hook a callback into the creator:

\startfunctioncall
function(<node> local_par, <string> location)
end
\stopfunctioncall

There is no return value and you should make sure that the node stays valid
as otherwise \TEX\ can get confused.

\subsubsection{\type {mlist_to_hlist}}

This callback replaces \LUATEX's math list to node list conversion algorithm.

\startfunctioncall
function(<node> head, <string> display_type, <boolean> need_penalties)
    return <node> newhead
end
\stopfunctioncall

The returned node is the head of the list that will be added to the vertical or
horizontal list, the string argument is either \quote {text} or \quote {display}
depending on the current math mode, the boolean argument is \type {true} if
penalties have to be inserted in this list, \type {false} otherwise.

Setting this callback to \type {false} is bad, it will almost certainly result in
an endless loop.

\subsection{Information reporting callbacks}

\subsubsection{\type {pre_dump}}

\startfunctioncall
function()
end
\stopfunctioncall

This function is called just before dumping to a format file starts. It does not
replace any code and there are neither arguments nor return values.

\subsubsection{\type {start_run}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback replaces the code that prints \LUATEX's banner. Note that for
successful use, this callback has to be set in the \LUA\ initialization script,
otherwise it will be seen only after the run has already started.

\subsubsection{\type {stop_run}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback replaces the code that prints \LUATEX's statistics and \quote
{output written to} messages.

\subsubsection{\type {start_page_number}}

\startfunctioncall
function()
end
\stopfunctioncall

Replaces the code that prints the \type {[} and the page number at the begin of
\type {\shipout}. This callback will also override the printing of box information
that normally takes place when \type {\tracingoutput} is positive.

\subsubsection{\type {stop_page_number}}

\startfunctioncall
function()
end
\stopfunctioncall

Replaces the code that prints the \type {]} at the end of \type {\shipout}.

\subsubsection{\type {show_error_hook}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback is run from inside the \TEX\ error function, and the idea is to
allow you to do some extra reporting on top of what \TEX\ already does (none of
the normal actions are removed). You may find some of the values in the \type
{status} table useful. This callback does not replace any internal code.

\subsubsection{\type {show_error_message}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback replaces the code that prints the error message. The usual
interaction after the message is not affected.

\subsubsection{\type {show_lua_error_hook}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback replaces the code that prints the extra \LUA\ error message.

\subsubsection{\type {start_file}}

\startfunctioncall
function(category,filename)
end
\stopfunctioncall

This callback replaces the code that prints \LUATEX's when a file is opened like
\type {(filename} for regular files. The category is a number:

\starttabulate[|||]
\NC 1 \NC a normal data file, like a \TEX\ source \NC \NR
\NC 2 \NC a font map coupling font names to resources \NC \NR
\NC 3 \NC an image file (\type {png}, \type {pdf}, etc) \NC \NR
\NC 4 \NC an embedded font subset \NC \NR
\NC 5 \NC a fully embedded font \NC \NR
\stoptabulate

\subsubsection{\type {stop_file}}

\startfunctioncall
function(category)
end
\stopfunctioncall

This callback replaces the code that prints \LUATEX's when a file is closed like
the \type {)} for regular files.

\subsection{PDF-related callbacks}

\subsubsection{\type {finish_pdffile}}

\startfunctioncall
function()
end
\stopfunctioncall

This callback is called when all document pages are already written to the \PDF\
file and \LUATEX\ is about to finalize the output document structure. Its
intended use is final update of \PDF\ dictionaries such as \type {/Catalog} or
\type {/Info}. The callback does not replace any code. There are neither
arguments nor return values.

\subsubsection{\type {finish_pdfpage}}

\startfunctioncall
function(shippingout)
end
\stopfunctioncall

This callback is called after the \PDF\ page stream has been assembled and before
the page object gets finalized.

\subsection{Font-related callbacks}

\subsubsection{\type {define_font}}

\startfunctioncall
function(<string> name, <number> size, <number> id)
    return <table> font | <number> id
end
\stopfunctioncall

The string \type {name} is the filename part of the font specification, as given
by the user.

The number \type {size} is a bit special:

\startitemize[packed]
\startitem
    If it is positive, it specifies an \quote{at size} in scaled points.
\stopitem
\startitem
    If it is negative, its absolute value represents a \quote {scaled} setting
    relative to the designsize of the font.
\stopitem
\stopitemize

The \type {id} is the internal number assigned to the font.

The internal structure of the \type {font} table that is to be returned is
explained in \in {chapter} [fonts]. That table is saved internally, so you can
put extra fields in the table for your later \LUA\ code to use. In alternative,
\type {retval} can be a previously defined fontid. This is useful if a previous
definition can be reused instead of creating a whole new font structure.

Setting this callback to \type {false} is pointless as it will prevent font
loading completely but will nevertheless generate errors.

\section{The \type {epdf} library}

The \type {epdf} library provides \LUA\ bindings to many \PDF\ access functions
that are defined by the poppler \PDF\ viewer library (written in C$+{}+$ by
Kristian H\o gsberg, based on xpdf by Derek Noonburg). Within \LUATEX\ (and
\PDFTEX), xpdf functionality is being used since long time to embed \PDF\ files.
The \type {epdf} library shall allow to scrutinize an external \PDF\ file. It
gives access to its document structure: catalog, cross|-|reference table,
individual pages, objects, annotations, info, and metadata. The \LUATEX\ team is
evaluating the possibility of reducing the binding to a basic low level \PDF\
primitives and delegate the complete set of functions to an external shared
object module.

The \type {epdf} library is still in alpha state: \PDF\ access is currently
read|-|only. Iit's not yet possible to alter a \PDF\ file or to assemble it from
scratch, and many function bindings are still missing, and it is unlikely that we
to support that at all. At some point we might also decide to limit the interface
to a reasonable subset.

For a start, a \PDF\ file is opened by \type {epdf.open()} with file name, e.g.:

\starttyping
doc = epdf.open("foo.pdf")
\stoptyping

This normally returns a \type {PDFDoc} userdata variable; but if the file could
not be opened successfully, instead of a fatal error just the value \type {nil} is
returned.

All Lua functions in the \type {epdf} library are named after the poppler
functions listed in the poppler header files for the various classes, e.g., files
\type {PDFDoc.h}, \type {Dict.h}, and \type {Array.h}. These files can be found
in the poppler subdirectory within the \LUATEX\ sources. Which functions are
already implemented in the \type {epdf} library can be found in the \LUATEX\
source file \type {lepdflib.cc}. For using the \type {epdf} library, knowledge of
the \PDF\ file architecture is indispensable.

There are many different userdata types defined by the \type {epdf} library,
currently these are \type {AnnotBorderStyle}, \type {AnnotBorder}, \type
{Annots}, \type {Annot}, \type {Array}, \type {Attribute}, \type {Catalog}, \type
{Dict}, \type {EmbFile}, \type {GString}, \type {LinkDest}, \type {Links}, \type
{Link}, \type {ObjectStream}, \type {Object}, \type {PDFDoc}, \type
{PDFRectangle}, \type {Page}, \type {Ref}, \type {Stream}, \type {StructElement},
\type {StructTreeRoot} \type {TextSpan}, \type {XRefEntry} and \type {XRef}.

All these userdata names and the Lua access functions closely resemble the
classes naming from the poppler header files, including the choice of mixed upper
and lower case letters. The Lua function calls use object|-|oriented syntax,
e.g., the following calls return the \type {Page} object for page~1:

\starttyping
pageref = doc:getCatalog():getPageRef(1)
pageobj = doc:getXRef():fetch(pageref.num, pageref.gen)
\stoptyping

But writing such chained calls is risky, as an intermediate function may return
\type {nil} on error; therefore between function calls there should be Lua type
checks (e.g., against \type {nil}) done. If a non-object item is requested (e.g.,
a \type {Dict} item by calling \type {page:getPieceInfo()}, cf.~\type {Page.h})
but not available, the Lua functions return \type {nil} (without error). If a
function should return an \type {Object}, but it's not existing, a \type {Null}
object is returned instead (also without error; this is in|-|line with poppler
behavior).

All library objects have a \type {__gc} metamethod for garbage collection. The
\type {__tostring} metamethod gives the type name for each object.

All object constructors:

\startfunctioncall
<PDFDoc>       = epdf.open(<string> PDF filename)
<Annot>        = epdf.Annot(<XRef>, <Dict>, <Catalog>, <Ref>)
<Annots>       = epdf.Annots(<XRef>, <Catalog>, <Object>)
<Array>        = epdf.Array(<XRef>)
<Attribute>    = epdf.Attribute(<Type>,<Object>)| epdf.Attribute(<string>, <int>, <Object>)
<Dict>         = epdf.Dict(<XRef>)
<Object>       = epdf.Object()
<PDFRectangle> = epdf.PDFRectangle()
\stopfunctioncall

The functions \type {StructElement_Type}, \type {Attribute_Type} and \type
{AttributeOwner_Type} return a hash table \type {{<string>,<integer>}}.

\type {Annot} methods:

\startfunctioncall
<boolean>     = <Annot>:isOK()
<Object>      = <Annot>:getAppearance()
<AnnotBorder> = <Annot>:getBorder()
<boolean>     = <Annot>:match(<Ref>)
\stopfunctioncall

\type {AnnotBorderStyle} methods:

\startfunctioncall
<number> = <AnnotBorderStyle>:getWidth()
\stopfunctioncall

\type {Annots} methods:

\startfunctioncall
<integer> = <Annots>:getNumAnnots()
<Annot>   = <Annots>:getAnnot(<integer>)
\stopfunctioncall

\type {Array} methods:

\startfunctioncall
            <Array>:incRef()
            <Array>:decRef()
<integer> = <Array>:getLength()
            <Array>:add(<Object>)
<Object>  = <Array>:get(<integer>)
<Object>  = <Array>:getNF(<integer>)
<string>  = <Array>:getString(<integer>)
\stopfunctioncall

\type {Attribute} methods:

\startfunctioncall
<boolean>  = <Attribute>:isOk()
<integer>  = <Attribute>:getType()
<integer>  = <Attribute>:getOwner()
<string>   = <Attribute>:getTypeName()
<string>   = <Attribute>:getOwnerName()
<Object>   = <Attribute>:getValue()
<Object>   = <Attribute>:getDefaultValue
<string>   = <Attribute>:getName()
<integer>  = <Attribute>:getRevision()
             <Attribute>:setRevision(<unsigned integer>)
<boolean>  = <Attribute>:istHidden()
             <Attribute>:setHidden(<boolean>)
<string>   = <Attribute>:getFormattedValue()
<string>   = <Attribute>:setFormattedValue(<string>)
\stopfunctioncall

\type {Catalog} methods:

\startfunctioncall
<boolean>  = <Catalog>:isOK()
<integer>  = <Catalog>:getNumPages()
<Page>     = <Catalog>:getPage(<integer>)
<Ref>      = <Catalog>:getPageRef(<integer>)
<string>   = <Catalog>:getBaseURI()
<string>   = <Catalog>:readMetadata()
<Object>   = <Catalog>:getStructTreeRoot()
<integer>  = <Catalog>:findPage(<integer> object number, <integer> object generation)
<LinkDest> = <Catalog>:findDest(<string> name)
<Object>   = <Catalog>:getDests()
<integer>  = <Catalog>:numEmbeddedFiles()
<EmbFile>  = <Catalog>:embeddedFile(<integer>)
<integer>  = <Catalog>:numJS()
<string>   = <Catalog>:getJS(<integer>)
<Object>   = <Catalog>:getOutline()
<Object>   = <Catalog>:getAcroForm()
\stopfunctioncall

\type {EmbFile} methods:

\startfunctioncall
<string>   = <EmbFile>:name()
<string>   = <EmbFile>:description()
<integer>  = <EmbFile>:size()
<string>   = <EmbFile>:modDate()
<string>   = <EmbFile>:createDate()
<string>   = <EmbFile>:checksum()
<string>   = <EmbFile>:mimeType()
<Object>   = <EmbFile>:streamObject()
<boolean>  = <EmbFile>:isOk()
\stopfunctioncall

\type {Dict} methods:

\startfunctioncall
            <Dict>:incRef()
            <Dict>:decRef()
<integer> = <Dict>:getLength()
            <Dict>:add(<string>, <Object>)
            <Dict>:set(<string>, <Object>)
            <Dict>:remove(<string>)
<boolean> = <Dict>:is(<string>)
<Object>  = <Dict>:lookup(<string>)
<Object>  = <Dict>:lookupNF(<string>)
<integer> = <Dict>:lookupInt(<string>, <string>)
<string>  = <Dict>:getKey(<integer>)
<Object>  = <Dict>:getVal(<integer>)
<Object>  = <Dict>:getValNF(<integer>)
<boolean> = <Dict>:hasKey(<string>)
\stopfunctioncall

\type {Link} methods:

\startfunctioncall
<boolean>  = <Link>:isOK()
<boolean>  = <Link>:inRect(<number>, <number>)
\stopfunctioncall

\type {LinkDest} methods:

\startfunctioncall
<boolean>  = <LinkDest>:isOK()
<integer>  = <LinkDest>:getKind()
<string>   = <LinkDest>:getKindName()
<boolean>  = <LinkDest>:isPageRef()
<integer>  = <LinkDest>:getPageNum()
<Ref>      = <LinkDest>:getPageRef()
<number>   = <LinkDest>:getLeft()
<number>   = <LinkDest>:getBottom()
<number>   = <LinkDest>:getRight()
<number>   = <LinkDest>:getTop()
<number>   = <LinkDest>:getZoom()
<boolean>  = <LinkDest>:getChangeLeft()
<boolean>  = <LinkDest>:getChangeTop()
<boolean>  = <LinkDest>:getChangeZoom()
\stopfunctioncall

\type {Links} methods:

\startfunctioncall
<integer>  = <Links>:getNumLinks()
<Link>     = <Links>:getLink(<integer>)
\stopfunctioncall

\type {Object} methods:

\startfunctioncall
            <Object>:initBool(<boolean>)
            <Object>:initInt(<integer>)
            <Object>:initReal(<number>)
            <Object>:initString(<string>)
            <Object>:initName(<string>)
            <Object>:initNull()
            <Object>:initArray(<XRef>)
            <Object>:initDict(<XRef>)
            <Object>:initStream(<Stream>)
            <Object>:initRef(<integer> object number, <integer> object generation)
            <Object>:initCmd(<string>)
            <Object>:initError()
            <Object>:initEOF()
<Object>  = <Object>:fetch(<XRef>)
<integer> = <Object>:getType()
<string>  = <Object>:getTypeName()
<boolean> = <Object>:isBool()
<boolean> = <Object>:isInt()
<boolean> = <Object>:isReal()
<boolean> = <Object>:isNum()
<boolean> = <Object>:isString()
<boolean> = <Object>:isName()
<boolean> = <Object>:isNull()
<boolean> = <Object>:isArray()
<boolean> = <Object>:isDict()
<boolean> = <Object>:isStream()
<boolean> = <Object>:isRef()
<boolean> = <Object>:isCmd()
<boolean> = <Object>:isError()
<boolean> = <Object>:isEOF()
<boolean> = <Object>:isNone()
<boolean> = <Object>:getBool()
<integer> = <Object>:getInt()
<number>  = <Object>:getReal()
<number>  = <Object>:getNum()
<string>  = <Object>:getString()
<string>  = <Object>:getName()
<Array>   = <Object>:getArray()
<Dict>    = <Object>:getDict()
<Stream>  = <Object>:getStream()
<Ref>     = <Object>:getRef()
<integer> = <Object>:getRefNum()
<integer> = <Object>:getRefGen()
<string>  = <Object>:getCmd()
<integer> = <Object>:arrayGetLength()
          = <Object>:arrayAdd(<Object>)
<Object>  = <Object>:arrayGet(<integer>)
<Object>  = <Object>:arrayGetNF(<integer>)
<integer> = <Object>:dictGetLength(<integer>)
          = <Object>:dictAdd(<string>, <Object>)
          = <Object>:dictSet(<string>, <Object>)
<Object>  = <Object>:dictLookup(<string>)
<Object>  = <Object>:dictLookupNF(<string>)
<string>  = <Object>:dictgetKey(<integer>)
<Object>  = <Object>:dictgetVal(<integer>)
<Object>  = <Object>:dictgetValNF(<integer>)
<boolean> = <Object>:streamIs(<string>)
          = <Object>:streamReset()
<integer> = <Object>:streamGetChar()
<integer> = <Object>:streamLookChar()
<integer> = <Object>:streamGetPos()
          = <Object>:streamSetPos(<integer>)
<Dict>    = <Object>:streamGetDict()
\stopfunctioncall

\type {Page} methods:

\startfunctioncall
<boolean>      = <Page>:isOk()
<integer>      = <Page>:getNum()
<PDFRectangle> = <Page>:getMediaBox()
<PDFRectangle> = <Page>:getCropBox()
<boolean>      = <Page>:isCropped()
<number>       = <Page>:getMediaWidth()
<number>       = <Page>:getMediaHeight()
<number>       = <Page>:getCropWidth()
<number>       = <Page>:getCropHeight()
<PDFRectangle> = <Page>:getBleedBox()
<PDFRectangle> = <Page>:getTrimBox()
<PDFRectangle> = <Page>:getArtBox()
<integer>      = <Page>:getRotate()
<string>       = <Page>:getLastModified()
<Dict>         = <Page>:getBoxColorInfo()
<Dict>         = <Page>:getGroup()
<Stream>       = <Page>:getMetadata()
<Dict>         = <Page>:getPieceInfo()
<Dict>         = <Page>:getSeparationInfo()
<Dict>         = <Page>:getResourceDict()
<Object>       = <Page>:getAnnots()
<Links>        = <Page>:getLinks(<Catalog>)
<Object>       = <Page>:getContents()
\stopfunctioncall

\type {PDFDoc} methods:

\startfunctioncall
<boolean>  = <PDFDoc>:isOk()
<integer>  = <PDFDoc>:getErrorCode()
<string>   = <PDFDoc>:getErrorCodeName()
<string>   = <PDFDoc>:getFileName()
<XRef>     = <PDFDoc>:getXRef()
<Catalog>  = <PDFDoc>:getCatalog()
<number>   = <PDFDoc>:getPageMediaWidth()
<number>   = <PDFDoc>:getPageMediaHeight()
<number>   = <PDFDoc>:getPageCropWidth()
<number>   = <PDFDoc>:getPageCropHeight()
<integer>  = <PDFDoc>:getNumPages()
<string>   = <PDFDoc>:readMetadata()
<Object>   = <PDFDoc>:getStructTreeRoot()
<integer>  = <PDFDoc>:findPage(<integer> object number, <integer> object generation)
<Links>    = <PDFDoc>:getLinks(<integer>)
<LinkDest> = <PDFDoc>:findDest(<string>)
<boolean>  = <PDFDoc>:isEncrypted()
<boolean>  = <PDFDoc>:okToPrint()
<boolean>  = <PDFDoc>:okToChange()
<boolean>  = <PDFDoc>:okToCopy()
<boolean>  = <PDFDoc>:okToAddNotes()
<boolean>  = <PDFDoc>:isLinearized()
<Object>   = <PDFDoc>:getDocInfo()
<Object>   = <PDFDoc>:getDocInfoNF()
<integer>  = <PDFDoc>:getPDFMajorVersion()
<integer>  = <PDFDoc>:getPDFMinorVersion()
\stopfunctioncall

\type {PDFRectangle} methods:

\startfunctioncall
<boolean>  = <PDFRectangle>:isValid()
\stopfunctioncall

%\type {Ref} methods:
%
%\startfunctioncall
%\stopfunctioncall

\type {Stream} methods:

\startfunctioncall
<integer>  = <Stream>:getKind()
<string>   = <Stream>:getKindName()
           = <Stream>:reset()
           = <Stream>:close()
<integer>  = <Stream>:getChar()
<integer>  = <Stream>:lookChar()
<integer>  = <Stream>:getRawChar()
<integer>  = <Stream>:getUnfilteredChar()
           = <Stream>:unfilteredReset()
<integer>  = <Stream>:getPos()
<boolean>  = <Stream>:isBinary()
<Stream>   = <Stream>:getUndecodedStream()
<Dict>     = <Stream>:getDict()
\stopfunctioncall

\type {StructElement} methods:

\startfunctioncall
<string>         = <StructElement>:getTypeName()
<integer>        = <StructElement>:getType()
<boolean>        = <StructElement>:isOk()
<boolean>        = <StructElement>:isBlock()
<boolean>        = <StructElement>:isInline()
<boolean>        = <StructElement>:isGrouping()
<boolean>        = <StructElement>:isContent()
<boolean>        = <StructElement>:isObjectRef()
<integer>        = <StructElement>:getMCID()
<Ref>            = <StructElement>:getObjectRef()
<Ref>            = <StructElement>:getParentRef()
<boolean>        = <StructElement>:hasPageRef()
<Ref>            = <StructElement>:getPageRef()
<StructTreeRoot> = <StructElement>:getStructTreeRoot()
<string>         = <StructElement>:getID()
<string>         = <StructElement>:getLanguage()
<integer>        = <StructElement>:getRevision()
                   <StructElement>:setRevision(<unsigned integer>)
<string>         = <StructElement>:getTitle()
<string>         = <StructElement>:getExpandedAbbr()
<integer>        = <StructElement>:getNumChildren()
<StructElement>  = <StructElement>:getChild()
                 = <StructElement>:appendChild<StructElement>)
<integer>        = <StructElement>:getNumAttributes()
<Attribute>      = <StructElement>:geAttribute(<integer>)
<string>         = <StructElement>:appendAttribute(<Attribute>)
<Attribute>      = <StructElement>:findAttribute(<Attribute::Type>,boolean,Attribute::Owner)
<string>         = <StructElement>:getAltText()
<string>         = <StructElement>:getActualText()
<string>         = <StructElement>:getText(<boolean>)
<table>          = <StructElement>:getTextSpans()
\stopfunctioncall

\type {StructTreeRoot} methods:

\startfunctioncall
<StructElement> = <StructTreeRoot>:findParentElement
<PDFDoc>        = <StructTreeRoot>:getDoc
<Dict>          = <StructTreeRoot>:getRoleMap
<Dict>          = <StructTreeRoot>:getClassMap
<integer>       = <StructTreeRoot>:getNumChildren
<StructElement> = <StructTreeRoot>:getChild
                  <StructTreeRoot>:appendChild
<StructElement> = <StructTreeRoot>:findParentElement
\stopfunctioncall

\type {TextSpan} han only one method:

\startfunctioncall
<string> = <TestSpan>:getText()
\stopfunctioncall

\type {XRef} methods:

\startfunctioncall
<boolean>  = <XRef>:isOk()
<integer>  = <XRef>:getErrorCode()
<boolean>  = <XRef>:isEncrypted()
<boolean>  = <XRef>:okToPrint()
<boolean>  = <XRef>:okToPrintHighRes()
<boolean>  = <XRef>:okToChange()
<boolean>  = <XRef>:okToCopy()
<boolean>  = <XRef>:okToAddNotes()
<boolean>  = <XRef>:okToFillForm()
<boolean>  = <XRef>:okToAccessibility()
<boolean>  = <XRef>:okToAssemble()
<Object>   = <XRef>:getCatalog()
<Object>   = <XRef>:fetch(<integer> object number, <integer> object generation)
<Object>   = <XRef>:getDocInfo()
<Object>   = <XRef>:getDocInfoNF()
<integer>  = <XRef>:getNumObjects()
<integer>  = <XRef>:getRootNum()
<integer>  = <XRef>:getRootGen()
<integer>  = <XRef>:getSize()
<Object>   = <XRef>:getTrailerDict()
\stopfunctioncall

There is an experimental function \type {epdf.openMemStream} that takes three
arguments:

\starttabulate
\NC \type {stream}  \NC this is a (in low level \LUA\ speak) light userdata
                        object, i.e.\ a pointer to a sequence of bytes \NC \NR
\NC \type {length}  \NC this is the length of the stream in bytes \NC \NR
\NC \type {name}    \NC this is a unique identifier that us used for hashing the
                        stream, so that mulltiple doesn't use more memory \NC \NR
\stoptabulate

Instead of a light userdata stream you can also pass a \LUA\ string, in which
case the given length is (at most) the string length.

The returned object can be used in the \type {img} library instead of a filename.
Both the memory stream and it's use in the image library is experimental and can
change. In case you wonder where this can be used: when you use the swiglib
library for \type {graphicmagick}, it can return such a userdata object. This
permits conversion in memory and passing the result directly to the backend. This
might save some runtime in one|-|pass workflows. This feature is currently not
meant for production.

\section{The \type {font} library}

The font library provides the interface into the internals of the font system,
and also it contains helper functions to load traditional \TEX\ font metrics
formats. Other font loading functionality is provided by the \type {fontloader}
library that will be discussed in the next section.

\subsection{Loading a \TFM\ file}

The behavior documented in this subsection is considered stable in the sense that
there will not be backward-incompatible changes any more.

\startfunctioncall
<table> fnt = font.read_tfm(<string> name, <number> s)
\stopfunctioncall

The number is a bit special:

\startitemize
\startitem
    If it is positive, it specifies an \quote {at size} in scaled points.
\stopitem
\startitem
    If it is negative, its absolute value represents a \quote {scaled}
    setting relative to the designsize of the font.
\stopitem
\stopitemize

The internal structure of the metrics font table that is returned is explained in
\in {chapter} [fonts].

\subsection{Loading a \VF\ file}

The behavior documented in this subsection is considered stable in the sense that
there will not be backward-incompatible changes any more.

\startfunctioncall
<table> vf_fnt = font.read_vf(<string> name, <number> s)
\stopfunctioncall

The meaning of the number \type {s} and the format of the returned table are
similar to the ones in the \type {read_tfm()} function.

\subsection{The fonts array}

The whole table of \TEX\ fonts is accessible from \LUA\ using a virtual array.

\starttyping
font.fonts[n] = { ... }
<table> f = font.fonts[n]
\stoptyping

See \in {chapter} [fonts] for the structure of the tables. Because this is a
virtual array, you cannot call \type {pairs} on it, but see below for the \type
{font.each} iterator.

The two metatable functions implementing the virtual array are:

\startfunctioncall
<table> f = font.getfont(<number> n)
font.setfont(<number> n, <table> f)
\stopfunctioncall

Note that at the moment, each access to the \type {font.fonts} or call to \type
{font.getfont} creates a \LUA\ table for the whole font. This process can be quite
slow. In a later version of \LUATEX, this interface will change (it will start
using userdata objects instead of actual tables).

Also note the following: assignments can only be made to fonts that have already
been defined in \TEX, but have not been accessed {\it at all\/} since that
definition. This limits the usability of the write access to \type {font.fonts}
quite a lot, a less stringent ruleset will likely be implemented later.

\subsection{Checking a font's status}

You can test for the status of a font by calling this function:

\startfunctioncall
<boolean> f = font.frozen(<number> n)
\stopfunctioncall

The return value is one of \type {true} (unassignable), \type {false} (can be
changed) or \type {nil} (not a valid font at all).

\subsection{Defining a font directly}

You can define your own font into \type {font.fonts} by calling this function:

\startfunctioncall
<number> i = font.define(<table> f)
\stopfunctioncall

The return value is the internal id number of the defined font (the index into
\type {font.fonts}). If the font creation fails, an error is raised. The table
is a font structure, as explained in \in {chapter} [fonts].

\subsection{Projected next font id}

\startfunctioncall
<number> i = font.nextid()
\stopfunctioncall

This returns the font id number that would be returned by a \type {font.define}
call if it was executed at this spot in the code flow. This is useful for virtual
fonts that need to reference themselves.

\subsection{Font id}

\startfunctioncall
<number> i = font.id(<string> csname)
\stopfunctioncall

This returns the font id associated with \type {csname} string, or $-1$ if \type
{csname} is not defined.

\subsection{Currently active font}

\startfunctioncall
<number> i = font.current()
font.current(<number> i)
\stopfunctioncall

This gets or sets the currently used font number.

\subsection{Maximum font id}

\startfunctioncall
<number> i = font.max()
\stopfunctioncall

This is the largest used index in \type {font.fonts}.

\subsection{Iterating over all fonts}

\startfunctioncall
for i,v in font.each() do
  ...
end
\stopfunctioncall

This is an iterator over each of the defined \TEX\ fonts. The first returned
value is the index in \type {font.fonts}, the second the font itself, as a \LUA\
table. The indices are listed incrementally, but they do not always form an array
of consecutive numbers: in some cases there can be holes in the sequence.

\section{The \type {fontloader} library}

\subsection{Getting quick information on a font}

\startfunctioncall
<table> info = fontloader.info(<string> filename)
\stopfunctioncall

This function returns either \type {nil}, or a \type {table}, or an array of
small tables (in the case of a \TRUETYPE\ collection). The returned table(s) will
contain some fairly interesting information items from the font(s) defined by the
file:

\starttabulate[|lT|l|p|]
\NC \ssbf key    \NC \bf type \NC \bf explanation \NC \NR
\NC fontname     \NC string   \NC the \POSTSCRIPT\ name of the font\NC \NR
\NC fullname     \NC string   \NC the formal name of the font\NC \NR
\NC familyname   \NC string   \NC the family name this font belongs to\NC \NR
\NC weight       \NC string   \NC a string indicating the color value of the font\NC \NR
\NC version      \NC string   \NC the internal font version\NC \NR
\NC italicangle  \NC float    \NC the slant angle\NC \NR
\NC units_per_em \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR
\NC pfminfo      \NC table    \NC (see \in{section}[fontloaderpfminfotable])\NC \NR
\stoptabulate

Getting information through this function is (sometimes much) more efficient than
loading the font properly, and is therefore handy when you want to create a
dictionary of available fonts based on a directory contents.

\subsection{Loading an \OPENTYPE\ or \TRUETYPE\ file}
If you want to use an \OPENTYPE\ font, you have to get the metric information
from somewhere. Using the \type {fontloader} library, the simplest way to get
that information is thus:

\starttyping
function load_font (filename)
  local metrics = nil
  local font = fontloader.open(filename)
  if font then
     metrics = fontloader.to_table(font)
     fontloader.close(font)
  end
  return metrics
end

myfont = load_font('/opt/tex/texmf/fonts/data/arial.ttf')
\stoptyping

The main function call is

\startfunctioncall
<userdata> f, <table> w = fontloader.open(<string> filename)
<userdata> f, <table> w = fontloader.open(<string> filename, <string> fontname)
\stopfunctioncall

The first return value is a userdata representation of the font. The second
return value is a table containing any warnings and errors reported by fontloader
while opening the font. In normal typesetting, you would probably ignore the
second argument, but it can be useful for debugging purposes.

For \TRUETYPE\ collections (when filename ends in 'ttc') and \DFONT\ collections,
you have to use a second string argument to specify which font you want from the
collection. Use the \type {fontname} strings that are returned by \type
{fontloader.info} for that.

To turn the font into a table, \type {fontloader.to_table} is used on the font
returned by \type {fontloader.open}.

\startfunctioncall
<table> f = fontloader.to_table(<userdata> font)
\stopfunctioncall

This table cannot be used directly by \LUATEX\ and should be turned into another
one as described in~\in {chapter} [fonts]. Do not forget to store the \type
{fontname} value in the \type {psname} field of the metrics table to be returned
to \LUATEX, otherwise the font inclusion backend will not be able to find the
correct font in the collection.

See \in {section} [fontloadertables] for details on the userdata object returned
by \type {fontloader.open()} and the layout of the \type {metrics} table returned
by \type {fontloader.to_table()}.

The font file is parsed and partially interpreted by the font loading routines
from \FONTFORGE. The file format can be \OPENTYPE, \TRUETYPE, \TRUETYPE\
Collection, \CFF, or \TYPEONE.

There are a few advantages to this approach compared to reading the actual font
file ourselves:

\startitemize

\startitem
    The font is automatically re|-|encoded, so that the \type {metrics} table for
    \TRUETYPE\ and \OPENTYPE\ fonts is using \UNICODE\ for the character indices.
\stopitem

\startitem
    Many features are pre|-|processed into a format that is easier to handle than
    just the bare tables would be.
\stopitem

\startitem
    \POSTSCRIPT|-|based \OPENTYPE\ fonts do not store the character height and
    depth in the font file, so the character boundingbox has to be calculated in
    some way.
\stopitem

\startitem
    In the future, it may be interesting to allow \LUA\ scripts access to
    the font program itself, perhaps even creating or changing the font.
\stopitem

\stopitemize

A loaded font is discarded with:

\startfunctioncall
fontloader.close(<userdata> font)
\stopfunctioncall

\subsection{Applying a \quote{feature file}}

You can apply a \quote{feature file} to a loaded font:

\startfunctioncall
<table> errors = fontloader.apply_featurefile(<userdata> font, <string> filename)
\stopfunctioncall

A \quote {feature file} is a textual representation of the features in an
\OPENTYPE\ font. See

\starttyping
http://www.adobe.com/devnet/opentype/afdko/topic_feature_file_syntax.html
\stoptyping

and

\starttyping
http://fontforge.sourceforge.net/featurefile.html
\stoptyping

for a more detailed description of feature files.

If the function fails, the return value is a table containing any errors reported
by fontloader while applying the feature file. On success, \type {nil} is
returned.

\subsection{Applying an \quote{\AFM\ file}}

You can apply an \quote {\AFM\ file} to a loaded font:

\startfunctioncall
<table> errors = fontloader.apply_afmfile(<userdata> font, <string> filename)
\stopfunctioncall

An \AFM\ file is a textual representation of (some of) the meta information
in a \TYPEONE\ font. See

\starttyping
ftp://ftp.math.utah.edu/u/ma/hohn/linux/postscript/5004.AFM_Spec.pdf
\stoptyping

for more information about \AFM\ files.

Note: If you \type {fontloader.open()} a \TYPEONE\ file named \type {font.pfb},
the library will automatically search for and apply \type {font.afm} if it exists
in the same directory as the file \type {font.pfb}. In that case, there is no
need for an explicit call to \type {apply_afmfile()}.

If the function fails, the return value is a table containing any errors reported
by fontloader while applying the AFM file. On success, \type {nil} is returned.

\subsection[fontloadertables]{Fontloader font tables}

As mentioned earlier, the return value of \type {fontloader.open()} is a userdata
object. One way to have access to the actual metrics is to call \type
{fontloader.to_table()} on this object, returning the table structure that is
explained in the following subsections.

However, it turns out that the result from \type {fontloader.to_table()}
sometimes needs very large amounts of memory (depending on the font's complexity
and size) so it is possible to access the userdata object directly.

\startitemize
\startitem
    All top|-|level keys that would be returned by \type {to_table()}
    can also be accessed directly.
\stopitem
\startitem
\startitem
    The top|-|level key \quote {glyphs} returns a {\it virtual\/} array that
    allows indices from \type {f.glyphmin} to (\type {f.glyphmax}).
\stopitem
\startitem
    The items in that virtual array (the actual glyphs) are themselves also
    userdata objects, and each has accessors for all of the keys explained in the
    section \quote {Glyph items} below.
\stopitem
    The top|-|level key \quote {subfonts} returns an {\it actual} array of userdata
    objects, one for each of the subfonts (or nil, if there are no subfonts).
\stopitem
\stopitemize

A short example may be helpful. This code generates a printout of all
the glyph names in the font \type {PunkNova.kern.otf}:

\starttyping
local f = fontloader.open('PunkNova.kern.otf')
print (f.fontname)
local i = 0
if f.glyphcnt > 0 then
    for i=f.glyphmin,f.glyphmax do
       local g = f.glyphs[i]
       if g then
          print(g.name)
       end
       i = i + 1
    end
end
fontloader.close(f)
\stoptyping

In this case, the \LUATEX\ memory requirement stays below 100MB on the test
computer, while the internal structure generated by \type {to_table()} needs more
than 2GB of memory (the font itself is 6.9MB in disk size).

Only the top|-|level font, the subfont table entries, and the glyphs are virtual
objects, everything else still produces normal \LUA\ values and tables.

If you want to know the valid fields in a font or glyph structure, call the \type
{fields} function on an object of a particular type (either glyph or font):

\startfunctioncall
<table> fields = fontloader.fields(<userdata> font)
<table> fields = fontloader.fields(<userdata> font_glyph)
\stopfunctioncall

For instance:

\startfunctioncall
local fields = fontloader.fields(f)
local fields = fontloader.fields(f.glyphs[0])
\stopfunctioncall

\subsubsection{Table types}

\subsubsubsection{Top-level}

The top|-|level keys in the returned table are (the explanations in this part of
the documentation are not yet finished):

\starttabulate[|lT|l|p|]
\NC \ssbf key                    \NC \bf type \NC \bf explanation \NC \NR
\NC table_version                \NC number   \NC indicates the metrics version (currently~0.3)\NC \NR
\NC fontname                     \NC string   \NC \POSTSCRIPT\ font name\NC \NR
\NC fullname                     \NC string   \NC official (human-oriented) font name\NC \NR
\NC familyname                   \NC string   \NC family name\NC \NR
\NC weight                       \NC string   \NC weight indicator\NC \NR
\NC copyright                    \NC string   \NC copyright information\NC \NR
\NC filename                     \NC string   \NC the file name\NC \NR
\NC version                      \NC string   \NC font version\NC \NR
\NC italicangle                  \NC float    \NC slant angle\NC \NR
\NC units_per_em                 \NC number   \NC 1000 for \POSTSCRIPT-based fonts, usually 2048 for \TRUETYPE\NC \NR
\NC ascent                       \NC number   \NC height of ascender in \type {units_per_em}\NC \NR
\NC descent                      \NC number   \NC depth of descender in \type {units_per_em}\NC \NR
\NC upos                         \NC float    \NC \NC \NR
\NC uwidth                       \NC float    \NC \NC \NR
\NC uniqueid                     \NC number   \NC \NC \NR
\NC glyphs                       \NC array    \NC \NC \NR
\NC glyphcnt                     \NC number   \NC number of included glyphs\NC \NR
\NC glyphmax                     \NC number   \NC maximum used index the glyphs array\NC \NR
\NC glyphmin                     \NC number   \NC minimum used index the glyphs array\NC \NR
\NC notdef_loc                   \NC number   \NC location of the \type {.notdef} glyph
                                                  or \type {-1} when not present \NC \NR
\NC hasvmetrics                  \NC number   \NC \NC \NR
\NC onlybitmaps                  \NC number   \NC \NC \NR
\NC serifcheck                   \NC number   \NC \NC \NR
\NC isserif                      \NC number   \NC \NC \NR
\NC issans                       \NC number   \NC \NC \NR
\NC encodingchanged              \NC number   \NC \NC \NR
\NC strokedfont                  \NC number   \NC \NC \NR
\NC use_typo_metrics             \NC number   \NC \NC \NR
\NC weight_width_slope_only      \NC number   \NC \NC \NR
\NC head_optimized_for_cleartype \NC number   \NC \NC \NR
\NC uni_interp                   \NC enum     \NC \type {unset}, \type {none}, \type {adobe},
                                                  \type {greek}, \type {japanese}, \type {trad_chinese},
                                                  \type {simp_chinese}, \type {korean}, \type {ams}\NC \NR
\NC origname                     \NC string   \NC the file name, as supplied by the user\NC \NR
\NC map                          \NC table    \NC \NC \NR
\NC private                      \NC table    \NC \NC \NR
\NC xuid                         \NC string   \NC \NC \NR
\NC pfminfo                      \NC table    \NC \NC \NR
\NC names                        \NC table    \NC \NC \NR
\NC cidinfo                      \NC table    \NC \NC \NR
\NC subfonts                     \NC array    \NC \NC \NR
\NC commments                    \NC string   \NC \NC \NR
\NC fontlog                      \NC string   \NC \NC \NR
\NC cvt_names                    \NC string   \NC \NC \NR
\NC anchor_classes               \NC table    \NC \NC \NR
\NC ttf_tables                   \NC table    \NC \NC \NR
\NC ttf_tab_saved                \NC table    \NC \NC \NR
\NC kerns                        \NC table    \NC \NC \NR
\NC vkerns                       \NC table    \NC \NC \NR
\NC texdata                      \NC table    \NC \NC \NR
\NC lookups                      \NC table    \NC \NC \NR
\NC gpos                         \NC table    \NC \NC \NR
\NC gsub                         \NC table    \NC \NC \NR
\NC mm                           \NC table    \NC \NC \NR
\NC chosenname                   \NC string   \NC \NC \NR
\NC macstyle                     \NC number   \NC \NC \NR
\NC fondname                     \NC string   \NC \NC \NR
%NC design_size                  \NC number   \NC \NC \NR
\NC fontstyle_id                 \NC number   \NC \NC \NR
\NC fontstyle_name               \NC table    \NC \NC \NR
%NC design_range_bottom          \NC number   \NC \NC \NR
%NC design_range_top             \NC number   \NC \NC \NR
\NC strokewidth                  \NC float    \NC \NC \NR
\NC mark_classes                 \NC table    \NC \NC \NR
\NC creationtime                 \NC number   \NC \NC \NR
\NC modificationtime             \NC number   \NC \NC \NR
\NC os2_version                  \NC number   \NC \NC \NR
\NC sfd_version                  \NC number   \NC \NC \NR
\NC math                         \NC table    \NC \NC \NR
\NC validation_state             \NC table    \NC \NC \NR
\NC horiz_base                   \NC table    \NC \NC \NR
\NC vert_base                    \NC table    \NC \NC \NR
\NC extrema_bound                \NC number   \NC \NC \NR
\NC truetype                     \NC boolean  \NC signals a \TRUETYPE\ font \NC \NR
\stoptabulate

\subsubsubsection{Glyph items}

The \type {glyphs} is an array containing the per|-|character
information (quite a few of these are only present if nonzero).

\starttabulate[|lT|l|p|]
\NC \ssbf key         \NC \bf type \NC \bf explanation \NC \NR
\NC name              \NC string   \NC the glyph name \NC \NR
\NC unicode           \NC number   \NC unicode code point, or -1 \NC \NR
\NC boundingbox       \NC array    \NC array of four numbers, see note below \NC \NR
\NC width             \NC number   \NC only for horizontal fonts \NC \NR
\NC vwidth            \NC number   \NC only for vertical fonts \NC \NR
\NC tsidebearing      \NC number   \NC only for vertical ttf/otf fonts, and only if nonzero \NC \NR
\NC lsidebearing      \NC number   \NC only if nonzero and not equal to boundingbox[1] \NC \NR
\NC class             \NC string   \NC one of "none", "base", "ligature", "mark", "component"
                                       (if not present, the glyph class is \quote {automatic}) \NC \NR
\NC kerns             \NC array    \NC only for horizontal fonts, if set \NC \NR
\NC vkerns            \NC array    \NC only for vertical fonts, if set \NC \NR
\NC dependents        \NC array    \NC linear array of glyph name strings, only if nonempty\NC \NR
\NC lookups           \NC table    \NC only if nonempty \NC \NR
\NC ligatures         \NC table    \NC only if nonempty \NC \NR
\NC anchors           \NC table    \NC only if set \NC \NR
\NC comment           \NC string   \NC only if set \NC \NR
\NC tex_height        \NC number   \NC only if set \NC \NR
\NC tex_depth         \NC number   \NC only if set \NC \NR
\NC italic_correction \NC number   \NC only if set \NC \NR
\NC top_accent        \NC number   \NC only if set \NC \NR
\NC is_extended_shape \NC number   \NC only if this character is part of a math extension list \NC \NR
\NC altuni            \NC table    \NC alternate \UNICODE\ items \NC \NR
\NC vert_variants     \NC table    \NC \NC \NR
\NC horiz_variants    \NC table    \NC \NC \NR
\NC mathkern          \NC table    \NC \NC \NR
\stoptabulate

On \type {boundingbox}: The boundingbox information for \TRUETYPE\ fonts and
\TRUETYPE-based \OTF\ fonts is read directly from the font file.
\POSTSCRIPT-based fonts do not have this information, so the boundingbox of
traditional \POSTSCRIPT\ fonts is generated by interpreting the actual bezier
curves to find the exact boundingbox. This can be a slow process, so the
boundingboxes of \POSTSCRIPT-based \OTF\ fonts (and raw \CFF\ fonts) are
calculated using an approximation of the glyph shape based on the actual glyph
points only, instead of taking the whole curve into account. This means that
glyphs that have missing points at extrema will have a too|-|tight boundingbox,
but the processing is so much faster that in our opinion the tradeoff is worth
it.

The \type {kerns} and \type {vkerns} are linear arrays of small hashes:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC char      \NC string   \NC \NC \NR
\NC off       \NC number   \NC \NC \NR
\NC lookup    \NC string   \NC \NC \NR
\stoptabulate

The \type {lookups} is a hash, based on lookup subtable names, with
the value of each key inside that a linear array of small hashes:

% TODO: fix this description
\starttabulate[|lT|l|p|]
\NC \ssbf key     \NC \bf type \NC \bf explanation \NC \NR
\NC type          \NC enum     \NC \type {position}, \type {pair}, \type
                                   {substitution}, \type {alternate}, \type
                                   {multiple}, \type {ligature}, \type {lcaret},
                                   \type {kerning}, \type {vkerning}, \type
                                   {anchors}, \type {contextpos}, \type
                                   {contextsub}, \type {chainpos}, \type
                                   {chainsub}, \type {reversesub}, \type {max},
                                   \type {kernback}, \type {vkernback} \NC \NR
\NC specification \NC table    \NC extra data \NC \NR
\stoptabulate

For the first seven values of \type {type}, there can be additional
sub|-|information, stored in the sub-table \type {specification}:

\starttabulate[|lT|l|p|]
\NC \ssbf value  \NC \bf type \NC \bf explanation \NC \NR
\NC position     \NC table    \NC a table of the \type {offset_specs} type \NC \NR
\NC pair         \NC table    \NC one string: \type {paired}, and an array of one
                                  or two \type {offset_specs} tables: \type
                                  {offsets} \NC \NR
\NC substitution \NC table    \NC one string: \type {variant} \NC \NR
\NC alternate    \NC table    \NC one string: \type {components} \NC \NR
\NC multiple     \NC table    \NC one string: \type {components} \NC \NR
\NC ligature     \NC table    \NC two strings: \type {components}, \type {char} \NC \NR
\NC lcaret       \NC array    \NC linear array of numbers \NC \NR
\stoptabulate

Tables for \type {offset_specs} contain up to four number|-|valued fields: \type
{x} (a horizontal offset), \type {y} (a vertical offset), \type {h} (an advance
width correction) and \type {v} (an advance height correction).

The \type {ligatures} is a linear array of small hashes:

\starttabulate[|lT|l|p|]
\NC \ssbf key  \NC \bf type \NC \bf explanation \NC \NR
\NC lig        \NC table    \NC uses the same substructure as a single item in
                                the \type {lookups} table explained above \NC \NR
\NC char       \NC string   \NC \NC \NR
\NC components \NC array    \NC linear array of named components \NC \NR
\NC ccnt       \NC number   \NC \NC \NR
\stoptabulate

The \type {anchor} table is indexed by a string signifying the anchor type, which
is one of

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC mark      \NC table    \NC placement mark \NC \NR
\NC basechar  \NC table    \NC mark for attaching combining items to a base char \NC \NR
\NC baselig   \NC table    \NC mark for attaching combining items to a ligature \NC \NR
\NC basemark  \NC table    \NC generic mark for attaching combining items to connect to \NC \NR
\NC centry    \NC table    \NC cursive entry point \NC \NR
\NC cexit     \NC table    \NC cursive exit point \NC \NR
\stoptabulate

The content of these is a short array of defined anchors, with the
entry keys being the anchor names. For all except \type {baselig}, the
value is a single table with this definition:

\starttabulate[|lT|l|p|]
\NC \ssbf key    \NC \bf type \NC \bf explanation \NC \NR
\NC x            \NC number   \NC x location \NC \NR
\NC y            \NC number   \NC y location \NC \NR
\NC ttf_pt_index \NC number   \NC truetype point index, only if given \NC \NR
\stoptabulate

For \type {baselig}, the value is a small array of such anchor sets sets, one for
each constituent item of the ligature.

For clarification, an anchor table could for example look like this :

\starttyping
['anchor'] = {
    ['basemark'] = {
        ['Anchor-7'] = { ['x']=170, ['y']=1080 }
    },
    ['mark'] ={
        ['Anchor-1'] = { ['x']=160, ['y']=810 },
        ['Anchor-4'] = { ['x']=160, ['y']=800 }
    },
    ['baselig'] = {
        [1] = { ['Anchor-2'] = { ['x']=160, ['y']=650 } },
        [2] = { ['Anchor-2'] = { ['x']=460, ['y']=640 } }
        }
    }
\stoptyping

Note: The \type {baselig} table can be sparse!

\subsubsubsection{map table}

The top|-|level map is a list of encoding mappings. Each of those is a table
itself.

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC enccount  \NC number   \NC \NC \NR
\NC encmax    \NC number   \NC \NC \NR
\NC backmax   \NC number   \NC \NC \NR
\NC remap     \NC table    \NC \NC \NR
\NC map       \NC array    \NC non|-|linear array of mappings\NC \NR
\NC backmap   \NC array    \NC non|-|linear array of backward mappings\NC \NR
\NC enc       \NC table    \NC \NC \NR
\stoptabulate

The \type {remap} table is very small:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC firstenc  \NC number   \NC \NC \NR
\NC lastenc   \NC number   \NC \NC \NR
\NC infont    \NC number   \NC \NC \NR
\stoptabulate

The \type {enc} table is a bit more verbose:

\starttabulate[|lT|l|p|]
\NC \ssbf key        \NC \bf type \NC \bf explanation \NC \NR
\NC enc_name         \NC string   \NC \NC \NR
\NC char_cnt         \NC number   \NC \NC \NR
\NC char_max         \NC number   \NC \NC \NR
\NC unicode          \NC array    \NC of \UNICODE\ position numbers\NC \NR
\NC psnames          \NC array    \NC of \POSTSCRIPT\ glyph names\NC \NR
\NC builtin          \NC number   \NC \NC \NR
\NC hidden           \NC number   \NC \NC \NR
\NC only_1byte       \NC number   \NC \NC \NR
\NC has_1byte        \NC number   \NC \NC \NR
\NC has_2byte        \NC number   \NC \NC \NR
\NC is_unicodebmp    \NC number   \NC only if nonzero\NC \NR
\NC is_unicodefull   \NC number   \NC only if nonzero\NC \NR
\NC is_custom        \NC number   \NC only if nonzero\NC \NR
\NC is_original      \NC number   \NC only if nonzero\NC \NR
\NC is_compact       \NC number   \NC only if nonzero\NC \NR
\NC is_japanese      \NC number   \NC only if nonzero\NC \NR
\NC is_korean        \NC number   \NC only if nonzero\NC \NR
\NC is_tradchinese   \NC number   \NC only if nonzero [name?]\NC \NR
\NC is_simplechinese \NC number   \NC only if nonzero\NC \NR
\NC low_page         \NC number   \NC \NC \NR
\NC high_page        \NC number   \NC \NC \NR
\NC iconv_name       \NC string   \NC \NC \NR
\NC iso_2022_escape  \NC string   \NC \NC \NR
\stoptabulate

\subsubsubsection{private table}

This is the font's private \POSTSCRIPT\ dictionary, if any. Keys and values are
both strings.

\subsubsubsection{cidinfo table}

\starttabulate[|lT|l|p|]
\NC \ssbf key  \NC \bf type \NC \bf explanation \NC \NR
\NC registry   \NC string   \NC \NC \NR
\NC ordering   \NC string   \NC \NC \NR
\NC supplement \NC number   \NC \NC \NR
\NC version    \NC number   \NC \NC \NR
\stoptabulate

\subsubsubsection[fontloaderpfminfotable]{pfminfo table}

The \type {pfminfo} table contains most of the OS/2 information:

\starttabulate[|lT|l|p|]
\NC \ssbf key        \NC \bf type \NC \bf explanation \NC \NR
\NC pfmset           \NC number   \NC \NC \NR
\NC winascent_add    \NC number   \NC \NC \NR
\NC windescent_add   \NC number   \NC \NC \NR
\NC hheadascent_add  \NC number   \NC \NC \NR
\NC hheaddescent_add \NC number   \NC \NC \NR
\NC typoascent_add   \NC number   \NC \NC \NR
\NC typodescent_add  \NC number   \NC \NC \NR
\NC subsuper_set     \NC number   \NC \NC \NR
\NC panose_set       \NC number   \NC \NC \NR
\NC hheadset         \NC number   \NC \NC \NR
\NC vheadset         \NC number   \NC \NC \NR
\NC pfmfamily        \NC number   \NC \NC \NR
\NC weight           \NC number   \NC \NC \NR
\NC width            \NC number   \NC \NC \NR
\NC avgwidth         \NC number   \NC \NC \NR
\NC firstchar        \NC number   \NC \NC \NR
\NC lastchar         \NC number   \NC \NC \NR
\NC fstype           \NC number   \NC \NC \NR
\NC linegap          \NC number   \NC \NC \NR
\NC vlinegap         \NC number   \NC \NC \NR
\NC hhead_ascent     \NC number   \NC \NC \NR
\NC hhead_descent    \NC number   \NC \NC \NR
\NC os2_typoascent   \NC number   \NC \NC \NR
\NC os2_typodescent  \NC number   \NC \NC \NR
\NC os2_typolinegap  \NC number   \NC \NC \NR
\NC os2_winascent    \NC number   \NC \NC \NR
\NC os2_windescent   \NC number   \NC \NC \NR
\NC os2_subxsize     \NC number   \NC \NC \NR
\NC os2_subysize     \NC number   \NC \NC \NR
\NC os2_subxoff      \NC number   \NC \NC \NR
\NC os2_subyoff      \NC number   \NC \NC \NR
\NC os2_supxsize     \NC number   \NC \NC \NR
\NC os2_supysize     \NC number   \NC \NC \NR
\NC os2_supxoff      \NC number   \NC \NC \NR
\NC os2_supyoff      \NC number   \NC \NC \NR
\NC os2_strikeysize  \NC number   \NC \NC \NR
\NC os2_strikeypos   \NC number   \NC \NC \NR
\NC os2_family_class \NC number   \NC \NC \NR
\NC os2_xheight      \NC number   \NC \NC \NR
\NC os2_capheight    \NC number   \NC \NC \NR
\NC os2_defaultchar  \NC number   \NC \NC \NR
\NC os2_breakchar    \NC number   \NC \NC \NR
\NC os2_vendor       \NC string   \NC \NC \NR
\NC codepages        \NC table    \NC A two-number array of encoded code pages\NC \NR
\NC unicoderages     \NC table    \NC A four-number array of encoded unicode ranges\NC \NR
\NC panose           \NC table    \NC \NC \NR
\stoptabulate

The \type {panose} subtable has exactly 10 string keys:

\starttabulate[|lT|l|p|]
\NC \ssbf key       \NC \bf type \NC \bf explanation \NC \NR
\NC familytype      \NC string   \NC Values as in the \OPENTYPE\ font
                                     specification: \type {Any}, \type {No Fit},
                                     \type {Text and Display}, \type {Script},
                                     \type {Decorative}, \type {Pictorial} \NC
                                     \NR
\NC serifstyle      \NC string   \NC See the \OPENTYPE\ font specification for
                                     values \NC \NR
\NC weight          \NC string   \NC id. \NC \NR
\NC proportion      \NC string   \NC id. \NC \NR
\NC contrast        \NC string   \NC id. \NC \NR
\NC strokevariation \NC string   \NC id. \NC \NR
\NC armstyle        \NC string   \NC id. \NC \NR
\NC letterform      \NC string   \NC id. \NC \NR
\NC midline         \NC string   \NC id. \NC \NR
\NC xheight         \NC string   \NC id. \NC \NR
\stoptabulate

\subsubsubsection[fontloadernamestable]{names table}

Each item has two top|-|level keys:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC lang      \NC string   \NC language for this entry \NC \NR
\NC names     \NC table    \NC \NC \NR
\stoptabulate

The \type {names} keys are the actual \TRUETYPE\ name strings. The possible keys
are:

\starttabulate[|lT|p|]
\NC \ssbf key       \NC \bf explanation \NC \NR
\NC copyright       \NC \NC \NR
\NC family          \NC \NC \NR
\NC subfamily       \NC \NC \NR
\NC uniqueid        \NC \NC \NR
\NC fullname        \NC \NC \NR
\NC version         \NC \NC \NR
\NC postscriptname  \NC \NC \NR
\NC trademark       \NC \NC \NR
\NC manufacturer    \NC \NC \NR
\NC designer        \NC \NC \NR
\NC descriptor      \NC \NC \NR
\NC venderurl       \NC \NC \NR
\NC designerurl     \NC \NC \NR
\NC license         \NC \NC \NR
\NC licenseurl      \NC \NC \NR
\NC idontknow       \NC \NC \NR
\NC preffamilyname  \NC \NC \NR
\NC prefmodifiers   \NC \NC \NR
\NC compatfull      \NC \NC \NR
\NC sampletext      \NC \NC \NR
\NC cidfindfontname \NC \NC \NR
\NC wwsfamily       \NC \NC \NR
\NC wwssubfamily    \NC \NC \NR
\stoptabulate

\subsubsubsection{anchor_classes table}

The anchor_classes classes:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC name      \NC string   \NC a descriptive id of this anchor class\NC \NR
\NC lookup    \NC string   \NC \NC \NR
\NC type      \NC string   \NC one of \type {mark}, \type {mkmk}, \type {curs}, \type {mklg} \NC \NR
\stoptabulate

% type is actually a lookup subtype, not a feature name. Officially, these
% strings should be gpos_mark2mark etc.

\subsubsubsection{gpos table}

The \type {gpos} table has one array entry for each lookup. (The \type {gpos_}
prefix is somewhat redundant.)

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC type      \NC string   \NC one of \type {gpos_single}, \type {gpos_pair},
                               \type {gpos_cursive}, \type {gpos_mark2base},\crlf
                               \type {gpos_mark2ligature}, \type
                               {gpos_mark2mark}, \type {gpos_context},\crlf \type
                               {gpos_contextchain} \NC \NR
\NC flags     \NC table    \NC \NC \NR
\NC name      \NC string   \NC \NC \NR
\NC features  \NC array    \NC \NC \NR
\NC subtables \NC array    \NC \NC \NR
\stoptabulate

The flags table has a true value for each of the lookup flags that is actually
set:

\starttabulate[|lT|l|p|]
\NC \ssbf key            \NC \bf type \NC \bf explanation \NC \NR
\NC r2l                  \NC boolean  \NC \NC \NR
\NC ignorebaseglyphs     \NC boolean  \NC \NC \NR
\NC ignoreligatures      \NC boolean  \NC \NC \NR
\NC ignorecombiningmarks \NC boolean  \NC \NC \NR
\NC mark_class           \NC string   \NC \NC \NR
\stoptabulate

The features subtable items of gpos have:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC tag       \NC string   \NC \NC \NR
\NC scripts   \NC table    \NC \NC \NR
\stoptabulate

The scripts table within features has:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type         \NC \bf explanation \NC \NR
\NC script    \NC string           \NC \NC \NR
\NC langs     \NC array of strings \NC \NC \NR
\stoptabulate

The subtables table has:

\starttabulate[|lT|l|p|]
\NC \ssbf key        \NC \bf type \NC \bf explanation \NC \NR
\NC name             \NC string   \NC \NC \NR
\NC suffix           \NC string   \NC (only if used)\NC \NR % used by gpos_single to get a default
\NC anchor_classes   \NC number   \NC (only if used)\NC \NR
\NC vertical_kerning \NC number   \NC (only if used)\NC \NR
\NC kernclass        \NC table    \NC (only if used)\NC \NR
\stoptabulate

The kernclass with subtables table has:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC firsts    \NC array of strings  \NC \NC \NR
\NC seconds   \NC array of strings   \NC \NC \NR
\NC lookup    \NC string or array \NC associated lookup(s) \NC \NR
\NC offsets   \NC array of numbers  \NC \NC \NR
\stoptabulate

Note: the kernclass (as far as we can see) always has one entry so it could be one level
deep instead. Also the seconds start at \type {[2]} which is close to the fontforge
internals so we keep that too.

\subsubsubsection{gsub table}

This has identical layout to the \type {gpos} table, except for the
type:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC type      \NC string   \NC one of \type {gsub_single}, \type {gsub_multiple},
                               \type {gsub_alternate}, \type
                               {gsub_ligature},\crlf \type {gsub_context}, \type
                               {gsub_contextchain}, \type
                               {gsub_reversecontextchain} \NC \NR
\stoptabulate

\subsubsubsection{ttf_tables and ttf_tab_saved tables}

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC tag       \NC string   \NC \NC \NR
\NC len       \NC number   \NC \NC \NR
\NC maxlen    \NC number   \NC \NC \NR
\NC data      \NC number   \NC \NC \NR
\stoptabulate

\subsubsubsection{mm table}

\starttabulate[|lT|l|p|]
\NC \ssbf key      \NC \bf type \NC \bf explanation \NC \NR
\NC axes           \NC table    \NC array of axis names \NC \NR
\NC instance_count \NC number   \NC \NC \NR
\NC positions      \NC table    \NC array of instance positions
                                    (\#axes * instances )\NC \NR
\NC defweights     \NC table    \NC array of default weights for instances \NC \NR
\NC cdv            \NC string   \NC \NC \NR
\NC ndv            \NC string   \NC \NC \NR
\NC axismaps       \NC table    \NC \NC \NR
\stoptabulate

The \type {axismaps}:

\starttabulate[|lT|l|p|]
\NC \ssbf key            \NC \bf type \NC \bf explanation \NC \NR
\NC blends               \NC table     \NC an array of blend points \NC \NR
\NC designs              \NC table     \NC an array of design values \NC \NR
\NC min                  \NC number   \NC \NC \NR
\NC def                  \NC number   \NC \NC \NR
\NC max                  \NC number   \NC \NC \NR
\stoptabulate

\subsubsubsection{mark_classes table}

The keys in this table are mark class names, and the values are a
space|-|separated string of glyph names in this class.

\subsubsubsection{math table}

\starttabulate[|lT|p|]
\NC ScriptPercentScaleDown                   \NC \NC \NR
\NC ScriptScriptPercentScaleDown             \NC \NC \NR
\NC DelimitedSubFormulaMinHeight             \NC \NC \NR
\NC DisplayOperatorMinHeight                 \NC \NC \NR
\NC MathLeading                              \NC \NC \NR
\NC AxisHeight                               \NC \NC \NR
\NC AccentBaseHeight                         \NC \NC \NR
\NC FlattenedAccentBaseHeight                \NC \NC \NR
\NC SubscriptShiftDown                       \NC \NC \NR
\NC SubscriptTopMax                          \NC \NC \NR
\NC SubscriptBaselineDropMin                 \NC \NC \NR
\NC SuperscriptShiftUp                       \NC \NC \NR
\NC SuperscriptShiftUpCramped                \NC \NC \NR
\NC SuperscriptBottomMin                     \NC \NC \NR
\NC SuperscriptBaselineDropMax               \NC \NC \NR
\NC SubSuperscriptGapMin                     \NC \NC \NR
\NC SuperscriptBottomMaxWithSubscript        \NC \NC \NR
\NC SpaceAfterScript                         \NC \NC \NR
\NC UpperLimitGapMin                         \NC \NC \NR
\NC UpperLimitBaselineRiseMin                \NC \NC \NR
\NC LowerLimitGapMin                         \NC \NC \NR
\NC LowerLimitBaselineDropMin                \NC \NC \NR
\NC StackTopShiftUp                          \NC \NC \NR
\NC StackTopDisplayStyleShiftUp              \NC \NC \NR
\NC StackBottomShiftDown                     \NC \NC \NR
\NC StackBottomDisplayStyleShiftDown         \NC \NC \NR
\NC StackGapMin                              \NC \NC \NR
\NC StackDisplayStyleGapMin                  \NC \NC \NR
\NC StretchStackTopShiftUp                   \NC \NC \NR
\NC StretchStackBottomShiftDown              \NC \NC \NR
\NC StretchStackGapAboveMin                  \NC \NC \NR
\NC StretchStackGapBelowMin                  \NC \NC \NR
\NC FractionNumeratorShiftUp                 \NC \NC \NR
\NC FractionNumeratorDisplayStyleShiftUp     \NC \NC \NR
\NC FractionDenominatorShiftDown             \NC \NC \NR
\NC FractionDenominatorDisplayStyleShiftDown \NC \NC \NR
\NC FractionNumeratorGapMin                  \NC \NC \NR
\NC FractionNumeratorDisplayStyleGapMin      \NC \NC \NR
\NC FractionRuleThickness                    \NC \NC \NR
\NC FractionDenominatorGapMin                \NC \NC \NR
\NC FractionDenominatorDisplayStyleGapMin    \NC \NC \NR
\NC SkewedFractionHorizontalGap              \NC \NC \NR
\NC SkewedFractionVerticalGap                \NC \NC \NR
\NC OverbarVerticalGap                       \NC \NC \NR
\NC OverbarRuleThickness                     \NC \NC \NR
\NC OverbarExtraAscender                     \NC \NC \NR
\NC UnderbarVerticalGap                      \NC \NC \NR
\NC UnderbarRuleThickness                    \NC \NC \NR
\NC UnderbarExtraDescender                   \NC \NC \NR
\NC RadicalVerticalGap                       \NC \NC \NR
\NC RadicalDisplayStyleVerticalGap           \NC \NC \NR
\NC RadicalRuleThickness                     \NC \NC \NR
\NC RadicalExtraAscender                     \NC \NC \NR
\NC RadicalKernBeforeDegree                  \NC \NC \NR
\NC RadicalKernAfterDegree                   \NC \NC \NR
\NC RadicalDegreeBottomRaisePercent          \NC \NC \NR
\NC MinConnectorOverlap                      \NC \NC \NR
\NC FractionDelimiterSize                    \NC \NC \NR
\NC FractionDelimiterDisplayStyleSize        \NC \NC \NR
\stoptabulate

\subsubsubsection{validation_state table}

\starttabulate[|lT|p|]
\NC \ssbf key         \NC \bf explanation \NC \NR
\NC bad_ps_fontname   \NC \NC \NR
\NC bad_glyph_table   \NC \NC \NR
\NC bad_cff_table     \NC \NC \NR
\NC bad_metrics_table \NC \NC \NR
\NC bad_cmap_table    \NC \NC \NR
\NC bad_bitmaps_table \NC \NC \NR
\NC bad_gx_table      \NC \NC \NR
\NC bad_ot_table      \NC \NC \NR
\NC bad_os2_version   \NC \NC \NR
\NC bad_sfnt_header   \NC \NC \NR
\stoptabulate

\subsubsubsection{horiz_base and vert_base table}

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC tags      \NC table    \NC an array of script list tags\NC \NR
\NC scripts   \NC table    \NC \NC \NR
\stoptabulate

The \type {scripts} subtable:

\starttabulate[|lT|l|p|]
\NC \ssbf key        \NC \bf type \NC \bf explanation \NC \NR
\NC baseline         \NC table   \NC \NC \NR
\NC default_baseline \NC number  \NC \NC \NR
\NC lang             \NC table   \NC \NC \NR
\stoptabulate


The \type {lang} subtable:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC tag       \NC string   \NC a script tag \NC \NR
\NC ascent    \NC number   \NC \NC \NR
\NC descent   \NC number   \NC \NC \NR
\NC features  \NC table    \NC \NC \NR
\stoptabulate

The \type {features} points to an array of tables with the same layout except
that in those nested tables, the tag represents a language.

\subsubsubsection{altuni table}

An array of alternate \UNICODE\ values. Inside that array are hashes with:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC unicode   \NC number   \NC this glyph is also used for this unicode \NC \NR
\NC variant   \NC number   \NC the alternative is driven by this unicode selector \NC \NR
\stoptabulate

\subsubsubsection{vert_variants and horiz_variants table}

\starttabulate[|lT|l|p|]
\NC \ssbf key         \NC \bf type \NC \bf explanation \NC \NR
\NC variants          \NC string   \NC \NC \NR
\NC italic_correction \NC number   \NC \NC \NR
\NC parts             \NC table    \NC \NC \NR
\stoptabulate

The \type {parts} table is an array of smaller tables:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC component \NC string   \NC \NC \NR
\NC extender  \NC number   \NC \NC \NR
\NC start     \NC number   \NC \NC \NR
\NC end       \NC number   \NC \NC \NR
\NC advance   \NC number   \NC \NC \NR
\stoptabulate


\subsubsubsection{mathkern table}

\starttabulate[|lT|l|p|]
\NC \ssbf key    \NC \bf type \NC \bf explanation \NC \NR
\NC top_right    \NC table    \NC \NC \NR
\NC bottom_right \NC table    \NC \NC \NR
\NC top_left     \NC table    \NC \NC \NR
\NC bottom_left  \NC table    \NC \NC \NR
\stoptabulate

Each of the subtables is an array of small hashes with two keys:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC height    \NC number   \NC \NC \NR
\NC kern      \NC number   \NC \NC \NR
\stoptabulate

\subsubsubsection{kerns table}

Substructure is identical to the per|-|glyph subtable.

\subsubsubsection{vkerns table}

Substructure is identical to the per|-|glyph subtable.

\subsubsubsection{texdata table}

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC type      \NC string   \NC \type {unset}, \type {text}, \type {math}, \type {mathext} \NC \NR
\NC params    \NC array    \NC 22 font numeric parameters \NC \NR
\stoptabulate

\subsubsubsection{lookups table}

Top|-|level \type {lookups} is quite different from the ones at character level.
The keys in this hash are strings, the values the actual lookups, represented as
dictionary tables.

\starttabulate[|lT|l|p|]
\NC \ssbf key     \NC \bf type \NC \bf explanation \NC \NR
\NC type          \NC string   \NC \NC \NR
\NC format        \NC enum     \NC one of \type {glyphs}, \type {class}, \type {coverage}, \type {reversecoverage} \NC \NR
\NC tag           \NC string   \NC \NC \NR
\NC current_class \NC array    \NC \NC \NR
\NC before_class  \NC array    \NC \NC \NR
\NC after_class   \NC array    \NC \NC \NR
\NC rules         \NC array    \NC an array of rule items\NC \NR
\stoptabulate

Rule items have one common item and one specialized item:

\starttabulate[|lT|l|p|]
\NC \ssbf key       \NC \bf type \NC \bf explanation \NC \NR
\NC lookups         \NC array    \NC a linear array of lookup names\NC \NR
\NC glyphs          \NC array    \NC only if the parent's format is \type {glyphs}\NC \NR
\NC class           \NC array    \NC only if the parent's format is \type {class}\NC \NR
\NC coverage        \NC array    \NC only if the parent's format is \type {coverage}\NC \NR
\NC reversecoverage \NC array    \NC only if the parent's format is \type {reversecoverage}\NC \NR
\stoptabulate

A glyph table is:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC names     \NC string   \NC \NC \NR
\NC back      \NC string   \NC \NC \NR
\NC fore      \NC string   \NC \NC \NR
\stoptabulate

A class table is:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC current   \NC array    \NC of numbers \NC \NR
\NC before    \NC array    \NC of numbers  \NC \NR
\NC after     \NC array    \NC of numbers  \NC \NR
\stoptabulate

coverage:

\starttabulate[|lT|l|p|]
\NC \ssbf key \NC \bf type \NC \bf explanation \NC \NR
\NC current   \NC array    \NC of strings \NC \NR
\NC before    \NC array    \NC of strings\NC \NR
\NC after     \NC array    \NC of strings \NC \NR
\stoptabulate

reversecoverage:

\starttabulate[|lT|l|p|]
\NC \ssbf key    \NC \bf type \NC \bf explanation \NC \NR
\NC current      \NC array    \NC of strings \NC \NR
\NC before       \NC array    \NC of strings\NC \NR
\NC after        \NC array    \NC of strings \NC \NR
\NC replacements \NC string   \NC \NC \NR
\stoptabulate

\section{The \type {img} library}

The \type {img} library can be used as an alternative to \type {\pdfximage} and
\type {\pdfrefximage}, and the associated \quote {satellite} commands like \type
{\pdfximagebbox}. Image objects can also be used within virtual fonts via the
\type {image} command listed in~\in {section} [virtualfonts].

\subsection{\type {img.new}}

\startfunctioncall
<image> var = img.new()
<image> var = img.new(<table> image_spec)
\stopfunctioncall

This function creates a userdata object of type \quote {image}. The \type
{image_spec} argument is optional. If it is given, it must be a table, and that
table must contain a \type {filename} key. A number of other keys can also be
useful, these are explained below.

You can either say

\starttyping
a = img.new()
\stoptyping

followed by

\starttyping
a.filename = "foo.png"
\stoptyping

or you can put the file name (and some or all of the other keys) into a table
directly, like so:

\starttyping
a = img.new({filename='foo.pdf', page=1})
\stoptyping

The generated \type {<image>} userdata object allows access to a set of
user|-|specified values as well as a set of values that are normally filled in
and updated automatically by \LUATEX\ itself. Some of those are derived from the
actual image file, others are updated to reflect the \PDF\ output status of the
object.

There is one required user-specified field: the file name (\type {filename}). It
can optionally be augmented by the requested image dimensions (\type {width},
\type {depth}, \type {height}), user|-|specified image attributes (\type {attr}),
the requested \PDF\ page identifier (\type {page}), the requested boundingbox
(\type {pagebox}) for \PDF\ inclusion, the requested color space object (\type
{colorspace}).

The function \type {img.new} does not access the actual image file, it just
creates the \type {<image>} userdata object and initializes some memory
structures. The \type {<image>} object and its internal structures are
automatically garbage collected.

Once the image is scanned, all the values in the \type {<image>} except \type
{width}, \type {height} and \type {depth}, become frozen, and you cannot change
them any more.

You can use \type {pdf.setignoreunknownimages(1)} (or at the \TEX\ end the \type
{\pdfvariable} \type {ignoreunknownimages}) to get around a quit when no known
image type is found (based on name or preamble). Beware: this will not catch
invalid images and we cannot guarantee side effects. A zero dimension image is
still included when requested. No special flags are set. A proper workflow will
not rely in such a catch but make sure that images are valid.

\subsection{\type {img.keys}}

\startfunctioncall
<table> keys = img.keys()
\stopfunctioncall

This function returns a list of all the possible \type {image_spec} keys, both
user-supplied and automatic ones.

% hahe: i need to add r/w ro column...
\starttabulate[|l|l|p|]
\NC \bf field name \NC \bf type \NC description \NC \NR
\NC attr           \NC string   \NC the image attributes for \LUATEX \NC \NR
\NC bbox           \NC table    \NC table with 4 boundingbox dimensions
                                    \type {llx}, \type {lly}, \type {urx},
                                    and \type {ury} overruling the \type {pagebox}
                                    entry\NC \NR
\NC colordepth     \NC number   \NC the number of bits used by the color space\NC \NR
\NC colorspace     \NC number   \NC the color space object number \NC \NR
\NC depth          \NC number   \NC the image depth for \LUATEX\
                                    (in scaled points)\NC \NR
\NC filename       \NC string   \NC the image file name \NC \NR
\NC filepath       \NC string   \NC the full (expanded) file name of the image\NC \NR
\NC height         \NC number   \NC the image height for \LUATEX\
                                    (in scaled points)\NC \NR
\NC imagetype      \NC string   \NC one of \type {pdf}, \type {png}, \type {jpg}, \type {jp2},
                                    \type {jbig2}, or \type {nil} \NC \NR
\NC index          \NC number   \NC the \PDF\ image name suffix \NC \NR
\NC objnum         \NC number   \NC the \PDF\ image object number \NC \NR
\NC page           \NC ??       \NC the identifier for the requested image page
                                    (type is number or string,
                                    default is the number 1)\NC \NR
\NC pagebox        \NC string   \NC the requested bounding box, one of
                                    \type {none}, \type {media}, \type {crop},
                                    \type {bleed}, \type {trim}, \type {art} \NC \NR
\NC pages          \NC number   \NC the total number of available pages \NC \NR
\NC rotation       \NC number   \NC the image rotation from included \PDF\ file,
                                    in multiples of 90~deg. \NC \NR
\NC stream         \NC string   \NC the raw stream data for an \type {/Xobject}
                                    \type {/Form} object\NC \NR
\NC transform      \NC number   \NC the image transform, integer number 0..7\NC \NR
\NC width          \NC number   \NC the image width for \LUATEX\
                                    (in scaled points)\NC \NR
\NC xres           \NC number   \NC the horizontal natural image resolution
                                    (in \DPI) \NC \NR
\NC xsize          \NC number   \NC the natural image width \NC \NR
\NC yres           \NC number   \NC the vertical natural image resolution
                                    (in \DPI) \NC \NR
\NC ysize          \NC number   \NC the natural image height \NC \NR
\NC visiblefileame \NC string   \NC when set, this name will find its way in the
                                    \PDF\ file as \type {PTEX} specification; when
                                    an empty string is assigned nothing is written
                                    to file, otherwise the natural filename is taken \NC \NR
\stoptabulate

A running (undefined) dimension in \type {width}, \type {height}, or \type
{depth} is represented as \type {nil} in \LUA, so if you want to load an image at
its \quote {natural} size, you do not have to specify any of those three fields.

The \type {stream} parameter allows to fabricate an \type {/XObject} \type
{/Form} object from a string giving the stream contents, e.g., for a filled
rectangle:

\startfunctioncall
a.stream = "0 0 20 10 re f"
\stopfunctioncall

When writing the image, an \type {/Xobject} \type {/Form} object is created, like
with embedded \PDF\ file writing. The object is written out only once. The \type
{stream} key requires that also the \type {bbox} table is given. The \type
{stream} key conflicts with the \type {filename} key. The \type {transform} key
works as usual also with \type {stream}.

The \type {bbox} key needs a table with four boundingbox values, e.g.:

\startfunctioncall
a.bbox = {"30bp", 0, "225bp", "200bp"}
\stopfunctioncall

This replaces and overrules any given \type {pagebox} value; with given \type
{bbox} the box dimensions coming with an embedded \PDF\ file are ignored. The
\type {xsize} and \type {ysize} dimensions are set accordingly, when the image is
scaled. The \type {bbox} parameter is ignored for non-\PDF\ images.

The \type {transform} allows to mirror and rotate the image in steps of 90~deg.
The default value~$0$ gives an unmirrored, unrotated image. Values $1-3$ give
counterclockwise rotation by $90$, $180$, or $270$~degrees, whereas with values
$4-7$ the image is first mirrored and then rotated counterclockwise by $90$,
$180$, or $270$~degrees. The \type {transform} operation gives the same visual
result as if you would externally preprocess the image by a graphics tool and
then use it by \LUATEX. If a \PDF\ file to be embedded already contains a \type
{/Rotate} specification, the rotation result is the combination of the \type
{/Rotate} rotation followed by the \type {transform} operation.

\subsection{\type {img.scan}}

\startfunctioncall
<image> var = img.scan(<image> var)
<image> var = img.scan(<table> image_spec)
\stopfunctioncall

When you say \type {img.scan(a)} for a new image, the file is scanned, and
variables such as \type {xsize}, \type {ysize}, image \type {type}, number of
\type {pages}, and the resolution are extracted. Each of the \type {width}, \type
{height}, \type {depth} fields are set up according to the image dimensions, if
they were not given an explicit value already. An image file will never be
scanned more than once for a given image variable. With all subsequent \type
{img.scan(a)} calls only the dimensions are again set up (if they have been
changed by the user in the meantime).

For ease of use, you can do right-away a

\starttyping
<image> a = img.scan ({ filename = "foo.png" })
\stoptyping

without a prior \type {img.new}.

Nothing is written yet at this point, so you can do \type {a=img.scan}, retrieve
the available info like image width and height, and then throw away \type {a}
again by saying \type {a=nil}. In that case no image object will be reserved in
the PDF, and the used memory will be cleaned up automatically.

\subsection{\type {img.copy}}

\startfunctioncall
<image> var = img.copy(<image> var)
<image> var = img.copy(<table> image_spec)
\stopfunctioncall

If you say \type {a = b}, then both variables point to the same \type {<image>}
object. if you want to write out an image with different sizes, you can do a
\type {b=img.copy(a)}.

Afterwards, \type {a} and \type {b} still reference the same actual image
dictionary, but the dimensions for \type {b} can now be changed from their
initial values that were just copies from \type {a}.

\subsection{\type {img.write}}

\startfunctioncall
<image> var = img.write(<image> var)
<image> var = img.write(<table> image_spec)
\stopfunctioncall

By \type {img.write(a)} a \PDF\ object number is allocated, and a whatsit node of
subtype \type {pdf_refximage} is generated and put into the output list. By this
the image \type {a} is placed into the page stream, and the image file is written
out into an image stream object after the shipping of the current page is
finished.

Again you can do a terse call like

\starttyping
img.write ({ filename = "foo.png" })
\stoptyping

The \type {<image>} variable is returned in case you want it for later
processing.

\subsection{\type {img.immediatewrite}}

\startfunctioncall
<image> var = img.immediatewrite(<image> var)
<image> var = img.immediatewrite(<table> image_spec)
\stopfunctioncall

By \type {img.immediatewrite(a)} a \PDF\ object number is allocated, and the
image file for image \type {a} is written out immediately into the \PDF\ file as
an image stream object (like with \type {\immediate}\type {\pdfximage}). The object
number of the image stream dictionary is then available by the \type {objnum}
key. No \type {pdf_refximage} whatsit node is generated. You will need an
\type {img.write(a)} or \type {img.node(a)} call to let the image appear on the
page, or reference it by another trick; else you will have a dangling image
object in the \PDF\ file.

Also here you can do a terse call like

\starttyping
a = img.immediatewrite ({ filename = "foo.png" })
\stoptyping

The \type {<image>} variable is returned and you will most likely need it.

\subsection{\type {img.node}}

\startfunctioncall
<node> n = img.node(<image> var)
<node> n = img.node(<table> image_spec)
\stopfunctioncall

This function allocates a \PDF\ object number and returns a whatsit node of
subtype \type {pdf_refximage}, filled with the image parameters \type {width},
\type {height}, \type {depth}, and \type {objnum}. Also here you can do a terse
call like:

\starttyping
n = img.node ({ filename = "foo.png" })
\stoptyping

This example outputs an image:

\starttyping
node.write(img.node{filename="foo.png"})
\stoptyping

\subsection{\type {img.types}}

\startfunctioncall
<table> types = img.types()
\stopfunctioncall

This function returns a list with the supported image file type names, currently
these are \type {pdf}, \type {png}, \type {jpg}, \type {jp2} (JPEG~2000), and
\type {jbig2}.

\subsection{\type {img.boxes}}

\startfunctioncall
<table> boxes = img.boxes()
\stopfunctioncall

This function returns a list with the supported \PDF\ page box names, currently
these are \type {media}, \type {crop}, \type {bleed}, \type {trim}, and \type
{art} (all in lowercase letters).

\section{The \type {kpse} library}

This library provides two separate, but nearly identical interfaces to the
\KPATHSEA\ file search functionality: there is a \quote {normal} procedural
interface that shares its kpathsea instance with \LUATEX\ itself, and an object
oriented interface that is completely on its own.

\subsection{\type {kpse.set_program_name} and \type {kpse.new}}

Before the search library can be used at all, its database has to be initialized.
There are three possibilities, two of which belong to the procedural interface.

First, when \LUATEX\ is used to typeset documents, this initialization happens
automatically and the \KPATHSEA\ executable and program names are set to \type
{luatex} (that is, unless explicitly prohibited by the user's startup script.
See~\in {section} [init] for more details).

Second, in \TEXLUA\ mode, the initialization has to be done explicitly via the
\type {kpse.set_program_name} function, which sets the \KPATHSEA\ executable
(and optionally program) name.

\startfunctioncall
kpse.set_program_name(<string> name)
kpse.set_program_name(<string> name, <string> progname)
\stopfunctioncall

The second argument controls the use of the \quote {dotted} values in the \type
{texmf.cnf} configuration file, and defaults to the first argument.

Third, if you prefer the object oriented interface, you have to call a different
function. It has the same arguments, but it returns a userdata variable.

\startfunctioncall
local kpathsea = kpse.new(<string> name)
local kpathsea = kpse.new(<string> name, <string> progname)
\stopfunctioncall

Apart from these two functions, the calling conventions of the interfaces are
identical. Depending on the chosen interface, you either call \type
{kpse.find_file()} or \type {kpathsea:find_file()}, with identical arguments and
return vales.

\subsection{\type {find_file}}

The most often used function in the library is find_file:

\startfunctioncall
<string> f = kpse.find_file(<string> filename)
<string> f = kpse.find_file(<string> filename, <string> ftype)
<string> f = kpse.find_file(<string> filename, <boolean> mustexist)
<string> f = kpse.find_file(<string> filename, <string> ftype, <boolean> mustexist)
<string> f = kpse.find_file(<string> filename, <string> ftype, <number> dpi)
\stopfunctioncall

Arguments:
\startitemize[intro]

\sym{filename}

the name of the file you want to find, with or without extension.

\sym{ftype}

maps to the \type {-format} argument of \KPSEWHICH. The supported \type {ftype}
values are the same as the ones supported by the standalone \type {kpsewhich}
program:

\startsimplecolumns
\starttyping
gf
pk
bitmap font
tfm
afm
base
bib
bst
cnf
ls-R
fmt
map
mem
mf
mfpool
mft
mp
mppool
MetaPost support
ocp
ofm
opl
otp
ovf
ovp
graphic/figure
tex
TeX system documentation
texpool
TeX system sources
PostScript header
Troff fonts
type1 fonts
vf
dvips config
ist
truetype fonts
type42 fonts
web2c files
other text files
other binary files
misc fonts
web
cweb
enc files
cmap files
subfont definition files
opentype fonts
pdftex config
lig files
texmfscripts
lua
font feature files
cid maps
mlbib
mlbst
clua
\stoptyping
\stopsimplecolumns

The default type is \type {tex}. Note: this is different from \KPSEWHICH, which
tries to deduce the file type itself from looking at the supplied extension.

\sym{mustexist}

is similar to \KPSEWHICH's \type {-must-exist}, and the default is \type {false}.
If you specify \type {true} (or a non|-|zero integer), then the \KPSE\ library
will search the disk as well as the \type {ls-R} databases.

\sym{dpi}

This is used for the size argument of the formats \type {pk}, \type {gf}, and
\type {bitmap font}. \stopitemize


\subsection{\type {lookup}}

A more powerful (but slower) generic method for finding files is also available.
It returns a string for each found file.

\startfunctioncall
<string> f, ... = kpse.lookup(<string> filename, <table> options)
\stopfunctioncall

The options match commandline arguments from \type {kpsewhich}:

\starttabulate[|l|l|p|]
\NC \ssbf key \NC \ssbf type \NC \ssbf description \NC \NR
\NC debug     \NC number     \NC set debugging flags for this lookup\NC     \NR
\NC format    \NC string     \NC use specific file type (see list above)\NC \NR
\NC dpi       \NC number     \NC use this resolution for this lookup; default 600\NC \NR
\NC path      \NC string     \NC search in the given path\NC \NR
\NC all       \NC boolean    \NC output all matches, not just the first\NC \NR
\NC mustexist \NC boolean    \NC search the disk as well as ls-R if necessary\NC \NR
\NC mktexpk   \NC boolean    \NC disable/enable mktexpk generation for this lookup\NC \NR
\NC mktextex  \NC boolean    \NC disable/enable mktextex generation for this lookup\NC \NR
\NC mktexmf   \NC boolean    \NC disable/enable mktexmf generation for this lookup\NC \NR
\NC mktextfm  \NC boolean    \NC disable/enable mktextfm generation for this lookup\NC \NR
\NC subdir    \NC string
                  or table   \NC only output matches whose directory part
                                 ends with the given string(s) \NC \NR
\stoptabulate

\subsection{\type {init_prog}}

Extra initialization for programs that need to generate bitmap fonts.

\startfunctioncall
kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode)
kpse.init_prog(<string> prefix, <number> base_dpi, <string> mfmode, <string> fallback)
\stopfunctioncall

\subsection{\type {readable_file}}

Test if an (absolute) file name is a readable file.

\startfunctioncall
<string> f = kpse.readable_file(<string> name)
\stopfunctioncall

The return value is the actual absolute filename you should use, because the disk
name is not always the same as the requested name, due to aliases and
system|-|specific handling under e.g.\ \MSDOS. Returns \type {nil} if the file
does not exist or is not readable.

\subsection{\type {expand_path}}

Like kpsewhich's \type {-expand-path}:

\startfunctioncall
<string> r = kpse.expand_path(<string> s)
\stopfunctioncall

\subsection{\type {expand_var}}

Like kpsewhich's  \type {-expand-var}:

\startfunctioncall
<string> r = kpse.expand_var(<string> s)
\stopfunctioncall

\subsection{\type {expand_braces}}

Like kpsewhich's \type {-expand-braces}:

\startfunctioncall
<string> r = kpse.expand_braces(<string> s)
\stopfunctioncall

\subsection{\type {show_path}}

Like kpsewhich's \type {-show-path}:

\startfunctioncall
<string> r = kpse.show_path(<string> ftype)
\stopfunctioncall


\subsection{\type {var_value}}

Like kpsewhich's \type {-var-value}:

\startfunctioncall
<string> r = kpse.var_value(<string> s)
\stopfunctioncall

\subsection{\type {version}}

Returns the kpathsea version string.

\startfunctioncall
<string> r = kpse.version()
\stopfunctioncall


\section{The \type {lang} library}

This library provides the interface to \LUATEX's structure
representing a language, and the associated functions.

\startfunctioncall
<language> l = lang.new()
<language> l = lang.new(<number> id)
\stopfunctioncall

This function creates a new userdata object. An object of type \type {<language>}
is the first argument to most of the other functions in the \type {lang}
library. These functions can also be used as if they were object methods, using
the colon syntax.

Without an argument, the next available internal id number will be assigned to
this object. With argument, an object will be created that links to the internal
language with that id number.

\startfunctioncall
<number> n = lang.id(<language> l)
\stopfunctioncall

returns the internal \type {\language} id number this object refers to.

\startfunctioncall
<string> n = lang.hyphenation(<language> l)
lang.hyphenation(<language> l, <string> n)
\stopfunctioncall

Either returns the current hyphenation exceptions for this language, or adds new
ones. The syntax of the string is explained in~\in {section}
[patternsexceptions].

\startfunctioncall
lang.clear_hyphenation(<language> l)
\stopfunctioncall

Clears the exception dictionary (string) for this language.

\startfunctioncall
<string> n = lang.clean(<language> l, <string> o)
<string> n = lang.clean(<string> o)
\stopfunctioncall

Creates a hyphenation key from the supplied hyphenation value. The syntax of the
argument string is explained in~\in {section} [patternsexceptions]. This function
is useful if you want to do something else based on the words in a dictionary
file, like spell|-|checking.

\startfunctioncall
<string> n = lang.patterns(<language> l)
lang.patterns(<language> l, <string> n)
\stopfunctioncall

Adds additional patterns for this language object, or returns the current set.
The syntax of this string is explained in~\in {section} [patternsexceptions].

\startfunctioncall
lang.clear_patterns(<language> l)
\stopfunctioncall

Clears the pattern dictionary for this language.

\startfunctioncall
<number> n = lang.prehyphenchar(<language> l)
lang.prehyphenchar(<language> l, <number> n)
\stopfunctioncall

Gets or sets the \quote {pre|-|break} hyphen character for implicit hyphenation
in this language (initially the hyphen, decimal 45).

\startfunctioncall
<number> n = lang.posthyphenchar(<language> l)
lang.posthyphenchar(<language> l, <number> n)
\stopfunctioncall

Gets or sets the \quote {post|-|break} hyphen character for implicit hyphenation
in this language (initially null, decimal~0, indicating emptiness).

\startfunctioncall
<number> n = lang.preexhyphenchar(<language> l)
lang.preexhyphenchar(<language> l, <number> n)
\stopfunctioncall

Gets or sets the \quote {pre|-|break} hyphen character for explicit hyphenation
in this language (initially null, decimal~0, indicating emptiness).

\startfunctioncall
<number> n = lang.postexhyphenchar(<language> l)
lang.postexhyphenchar(<language> l, <number> n)
\stopfunctioncall

Gets or sets the \quote {post|-|break} hyphen character for explicit hyphenation
in this language (initially null, decimal~0, indicating emptiness).

\startfunctioncall
<boolean> success = lang.hyphenate(<node> head)
<boolean> success = lang.hyphenate(<node> head, <node> tail)
\stopfunctioncall

Inserts hyphenation points (discretionary nodes) in a node list. If \type {tail}
is given as argument, processing stops on that node. Currently, \type {success}
is always true if \type {head} (and \type {tail}, if specified) are proper nodes,
regardless of possible other errors.

Hyphenation works only on \quote {characters}, a special subtype of all the glyph
nodes with the node subtype having the value \type {1}. Glyph modes with
different subtypes are not processed. See \in {section~} [charsandglyphs] for
more details.

The following two commands can be used to set or query hj codes:

\startfunctioncall
lang.sethjcode(<language> l, <number> char, <number> usedchar)
<number> usedchar = lang.gethjcode(<language> l, <number> char)
\stopfunctioncall

When you set a hjcode the current sets get initialized unless the set was already
initialized due to \type {\savinghyphcodes} being larger than zero.

\section{The \type {lua} library}

This library contains one read|-|only  item:

\starttyping
<string> s = lua.version
\stoptyping

This returns the \LUA\ version identifier string. The value is currently
\directlua {tex.print(lua.version)}.

\subsection{\LUA\ bytecode registers}

\LUA\ registers can be used to communicate \LUA\ functions across \LUA\ chunks.
The accepted values for assignments are functions and \type {nil}. Likewise, the
retrieved value is either a function or \type {nil}.

\starttyping
lua.bytecode[<number> n] = <function> f
lua.bytecode[<number> n]()
\stoptyping

The contents of the \type {lua.bytecode} array is stored inside the format file
as actual \LUA\ bytecode, so it can also be used to preload \LUA\ code.

Note: The function must not contain any upvalues. Currently, functions containing
upvalues can be stored (and their upvalues are set to \type {nil}), but this is
an artifact of the current \LUA\ implementation and thus subject to change.

The associated function calls are

\startfunctioncall
<function> f = lua.getbytecode(<number> n)
lua.setbytecode(<number> n, <function> f)
\stopfunctioncall

Note: Since a \LUA\ file loaded using \type {loadfile(filename)} is essentially
an anonymous function, a complete file can be stored in a bytecode register like
this:

\startfunctioncall
lua.bytecode[n] = loadfile(filename)
\stopfunctioncall

Now all definitions (functions, variables) contained in the file can be
created by executing this bytecode register:

\startfunctioncall
lua.bytecode[n]()
\stopfunctioncall

Note that the path of the file is stored in the \LUA\ bytecode to be used in
stack backtraces and therefore dumped into the format file if the above code is
used in \INITEX. If it contains private information, i.e. the user name, this
information is then contained in the format file as well. This should be kept in
mind when preloading files into a bytecode register in \INITEX.

\subsection{\LUA\ chunk name registers}

There is an array of 65536 (0--65535) potential chunk names for use with the
\type {\directlua} and \type {\latelua} primitives.

\startfunctioncall
lua.name[<number> n] = <string> s
<string> s = lua.name[<number> n]
\stopfunctioncall

If you want to unset a \LUA\ name, you can assign \type {nil} to it.

\section{The \type {mplib} library}

The \MP\ library interface registers itself in the table \type {mplib}. It is
based on \MPLIB\ version \ctxlua {context(mplib.version())}.

\subsection{\type {mplib.new}}

To create a new \METAPOST\ instance, call

\startfunctioncall
<mpinstance> mp = mplib.new({...})
\stopfunctioncall

This creates the \type {mp} instance object. The argument hash can have a number
of different fields, as follows:

\starttabulate[|lT|l|p|p|]
\NC \ssbf name  \NC \bf type \NC \bf description          \NC \bf default       \NC \NR
\NC error_line  \NC number   \NC error line width         \NC 79                \NC \NR
\NC print_line  \NC number   \NC line length in ps output \NC 100               \NC \NR
\NC random_seed \NC number   \NC the initial random seed  \NC variable          \NC \NR
\NC interaction \NC string   \NC the interaction mode,
                                 one of
                                 \type {batch},
                                 \type {nonstop},
                                 \type {scroll},
                                 \type {errorstop}        \NC \type {errorstop} \NC \NR
\NC job_name    \NC string   \NC \type {--jobname}        \NC \type {mpout}     \NC \NR
\NC find_file   \NC function \NC a function to find files \NC only local files  \NC \NR
\stoptabulate

The \type {find_file} function should be of this form:

\starttyping
<string> found = finder (<string> name, <string> mode, <string> type)
\stoptyping

with:

\starttabulate[|lT|l|p|]
\NC \bf name \NC \bf the requested file \NC \NR
\NC mode     \NC the file mode: \type {r} or \type {w} \NC \NR
\NC type     \NC the kind of file, one of: \type {mp}, \type {tfm}, \type {map},
                 \type {pfb}, \type {enc} \NC \NR
\stoptabulate

Return either the full path name of the found file, or \type {nil} if the file
cannot be found.

Note that the new version of \MPLIB\ no longer uses binary mem files, so the way
to preload a set of macros is simply to start off with an \type {input} command
in the first \type {mp:execute()} call.

\subsection{\type {mp:statistics}}

You can request statistics with:

\startfunctioncall
<table> stats = mp:statistics()
\stopfunctioncall

This function returns the vital statistics for an \MPLIB\ instance. There are
four fields, giving the maximum number of used items in each of four allocated
object classes:

\starttabulate[|lT|l|p|]
\NC main_memory \NC number \NC memory size \NC \NR
\NC hash_size   \NC number \NC hash size\NC \NR
\NC param_size  \NC number \NC simultaneous macro parameters\NC \NR
\NC max_in_open \NC number \NC input file nesting levels\NC \NR
\stoptabulate

Note that in the new version of \MPLIB, this is informational only. The objects
are all allocated dynamically, so there is no chance of running out of space
unless the available system memory is exhausted.

\subsection{\type {mp:execute}}

You can ask the \METAPOST\ interpreter to run a chunk of code by calling

\startfunctioncall
<table> rettable = mp:execute('metapost language chunk')
\stopfunctioncall

for various bits of \METAPOST\ language input. Be sure to check the \type
{rettable.status} (see below) because when a fatal \METAPOST\ error occurs the
\MPLIB\ instance will become unusable thereafter.

Generally speaking, it is best to keep your chunks small, but beware that all
chunks have to obey proper syntax, like each of them is a small file. For
instance, you cannot split a single statement over multiple chunks.

In contrast with the normal stand alone \type {mpost} command, there is {\em no}
implied \quote{input} at the start of the first chunk.

\subsection{\type {mp:finish}}

\startfunctioncall
<table> rettable = mp:finish()
\stopfunctioncall

If for some reason you want to stop using an \MPLIB\ instance while processing is
not yet actually done, you can call \type {mp:finish}. Eventually, used memory
will be freed and open files will be closed by the \LUA\ garbage collector, but
an explicit \type {mp:finish} is the only way to capture the final part of the
output streams.

\subsection{Result table}

The return value of \type {mp:execute} and \type {mp:finish} is a table with a
few possible keys (only \type {status} is always guaranteed to be present).

\starttabulate[|l|l|p|]
\NC log    \NC string \NC output to the \quote {log} stream \NC \NR
\NC term   \NC string \NC output to the \quote {term} stream \NC \NR
\NC error  \NC string \NC output to the \quote {error} stream
                          (only used for \quote {out of memory}) \NC \NR
\NC status \NC number \NC the return value:
                          \type {0} = good,
                          \type {1} = warning,
                          \type {2} = errors,
                          \type {3} = fatal error \NC \NR
\NC fig    \NC table  \NC an array of generated figures (if any) \NC \NR
\stoptabulate

When \type {status} equals~3, you should stop using this \MPLIB\ instance
immediately, it is no longer capable of processing input.

If it is present, each of the entries in the \type {fig} array is a userdata
representing a figure object, and each of those has a number of object methods
you can call:

\starttabulate[|l|l|p|]
\NC boundingbox  \NC function \NC returns the bounding box, as an array of 4
                                  values\NC \NR
\NC postscript   \NC function \NC returns a string that is the ps output of the
                                  \type {fig}. this function accepts two optional
                                  integer arguments for specifying the values of
                                  \type {prologues} (first argument) and \type
                                  {procset} (second argument)\NC \NR
\NC svg          \NC function \NC returns a string that is the svg output of the
                                  \type {fig}. This function accepts an optional
                                  integer argument for specifying the value of
                                  \type {prologues}\NC \NR
\NC objects      \NC function \NC returns the actual array of graphic objects in
                                  this \type {fig} \NC \NR
\NC copy_objects \NC function \NC returns a deep copy of the array of graphic
                                  objects in this \type {fig} \NC \NR
\NC filename     \NC function \NC the filename this \type {fig}'s \POSTSCRIPT\
                                  output would have written to in stand alone
                                  mode \NC \NR
\NC width        \NC function \NC the \type {fontcharwd} value \NC \NR
\NC height       \NC function \NC the \type {fontcharht} value \NC \NR
\NC depth        \NC function \NC the \type {fontchardp} value \NC \NR
\NC italcorr     \NC function \NC the \type {fontcharit} value \NC \NR
\NC charcode     \NC function \NC the (rounded) \type {charcode} value \NC \NR
\stoptabulate

Note: you can call \type {fig:objects()} only once for any one \type {fig}
object!

When the boundingbox represents a \quote {negated rectangle}, i.e.\ when the
first set of coordinates is larger than the second set, the picture is empty.

Graphical objects come in various types that each has a different list of
accessible values. The types are: \type {fill}, \type {outline}, \type {text},
\type {start_clip}, \type {stop_clip}, \type {start_bounds}, \type {stop_bounds},
\type {special}.

There is helper function (\type {mplib.fields(obj)}) to get the list of
accessible values for a particular object, but you can just as easily use the
tables given below.

All graphical objects have a field \type {type} that gives the object type as a
string value; it is not explicit mentioned in the following tables. In the
following, \type {number}s are \POSTSCRIPT\ points represented as a floating
point number, unless stated otherwise. Field values that are of type \type
{table} are explained in the next section.

\subsubsection{fill}

\starttabulate[|l|l|p|]
\NC path       \NC table  \NC the list of knots \NC \NR
\NC htap       \NC table  \NC the list of knots for the reversed trajectory \NC \NR
\NC pen        \NC table  \NC knots of the pen \NC \NR
\NC color      \NC table  \NC the object's color \NC \NR
\NC linejoin   \NC number \NC line join style (bare number)\NC \NR
\NC miterlimit \NC number \NC miterlimit\NC \NR
\NC prescript  \NC string \NC the prescript text \NC \NR
\NC postscript \NC string \NC the postscript text \NC \NR
\stoptabulate

The entries \type {htap} and \type {pen} are optional.

There is helper function (\type {mplib.pen_info(obj)}) that returns a table
containing a bunch of vital characteristics of the used pen (all values are
floats):

\starttabulate[|l|l|p|]
\NC width \NC number \NC width of the pen \NC \NR
\NC sx    \NC number \NC $x$ scale        \NC \NR
\NC rx    \NC number \NC $xy$ multiplier  \NC \NR
\NC ry    \NC number \NC $yx$ multiplier  \NC \NR
\NC sy    \NC number \NC $y$ scale        \NC \NR
\NC tx    \NC number \NC $x$ offset       \NC \NR
\NC ty    \NC number \NC $y$ offset       \NC \NR
\stoptabulate

\subsubsection{outline}

\starttabulate[|l|l|p|]
\NC path       \NC table  \NC the list of knots \NC \NR
\NC pen        \NC table  \NC knots of the pen \NC \NR
\NC color      \NC table  \NC the object's color \NC \NR
\NC linejoin   \NC number \NC line join style (bare number) \NC \NR
\NC miterlimit \NC number \NC miterlimit \NC \NR
\NC linecap    \NC number \NC line cap style (bare number) \NC \NR
\NC dash       \NC table  \NC representation of a dash list \NC \NR
\NC prescript  \NC string \NC the prescript text \NC \NR
\NC postscript \NC string \NC the postscript text \NC \NR
\stoptabulate

The entry \type {dash} is optional.

\subsubsection{text}

\starttabulate[|l|l|p|]
\NC text       \NC string \NC the text \NC \NR
\NC font       \NC string \NC font tfm name \NC \NR
\NC dsize      \NC number \NC font size \NC \NR
\NC color      \NC table  \NC the object's color \NC \NR
\NC width      \NC number \NC \NC \NR
\NC height     \NC number \NC \NC \NR
\NC depth      \NC number \NC \NC \NR
\NC transform  \NC table  \NC a text transformation \NC \NR
\NC prescript  \NC string \NC the prescript text \NC \NR
\NC postscript \NC string \NC the postscript text \NC \NR
\stoptabulate

\subsubsection{special}

\starttabulate[|l|l|p|]
\NC prescript \NC string \NC special text \NC \NR
\stoptabulate

\subsubsection{start_bounds, start_clip}

\starttabulate[|l|l|p|]
\NC path \NC table \NC the list of knots \NC \NR
\stoptabulate

\subsubsection{stop_bounds, stop_clip}

Here are no fields available.

\subsection{Subsidiary table formats}

\subsubsection{Paths and pens}

Paths and pens (that are really just a special type of paths as far as \MPLIB\ is
concerned) are represented by an array where each entry is a table that
represents a knot.

\starttabulate[|lT|l|p|]
\NC left_type   \NC string \NC when present: endpoint, but usually absent \NC \NR
\NC right_type  \NC string \NC like \type {left_type} \NC \NR
\NC x_coord     \NC number \NC X coordinate of this knot \NC \NR
\NC y_coord     \NC number \NC Y coordinate of this knot \NC \NR
\NC left_x      \NC number \NC X coordinate of the precontrol point of this knot \NC \NR
\NC left_y      \NC number \NC Y coordinate of the precontrol point of this knot \NC \NR
\NC right_x     \NC number \NC X coordinate of the postcontrol point of this knot \NC \NR
\NC right_y     \NC number \NC Y coordinate of the postcontrol point of this knot \NC \NR
\stoptabulate

There is one special case: pens that are (possibly transformed) ellipses have an
extra string-valued key \type {type} with value \type {elliptical} besides the
array part containing the knot list.

\subsubsection{Colors}

A color is an integer array with 0, 1, 3 or 4 values:

\starttabulate[|l|l|p|]
\NC 0 \NC marking only \NC no values                                                     \NC \NR
\NC 1 \NC greyscale    \NC one value in the range $(0,1)$, \quote {black} is $0$         \NC \NR
\NC 3 \NC \RGB         \NC three values in the range $(0,1)$, \quote {black} is $0,0,0$  \NC \NR
\NC 4 \NC \CMYK        \NC four values in the range $(0,1)$, \quote {black} is $0,0,0,1$ \NC \NR
\stoptabulate

If the color model of the internal object was \type {uninitialized}, then it was
initialized to the values representing \quote {black} in the colorspace \type
{defaultcolormodel} that was in effect at the time of the \type {shipout}.

\subsubsection{Transforms}

Each transform is a six|-|item array.

\starttabulate[|l|l|p|]
\NC 1 \NC number \NC represents x  \NC \NR
\NC 2 \NC number \NC represents y  \NC \NR
\NC 3 \NC number \NC represents xx \NC \NR
\NC 4 \NC number \NC represents yx \NC \NR
\NC 5 \NC number \NC represents xy \NC \NR
\NC 6 \NC number \NC represents yy \NC \NR
\stoptabulate

Note that the translation (index 1 and 2) comes first. This differs from the
ordering in \POSTSCRIPT, where the translation comes last.

\subsubsection{Dashes}

Each \type {dash} is two-item hash, using the same model as \POSTSCRIPT\ for the
representation of the dashlist. \type {dashes} is an array of \quote {on} and
\quote {off}, values, and \type {offset} is the phase of the pattern.

\starttabulate[|l|l|p|]
\NC dashes \NC hash   \NC an array of on-off numbers \NC \NR
\NC offset \NC number \NC the starting offset value  \NC \NR
\stoptabulate

\subsection{Character size information}

These functions find the size of a glyph in a defined font. The \type {fontname}
is the same name as the argument to \type {infont}; the \type {char} is a glyph
id in the range 0 to 255; the returned \type {w} is in AFM units.

\subsubsection{\type {mp:char_width}}

\startfunctioncall
<number> w = mp:char_width(<string> fontname, <number> char)
\stopfunctioncall

\subsubsection{\type {mp:char_height}}

\startfunctioncall
<number> w = mp:char_height(<string> fontname, <number> char)
\stopfunctioncall

\subsubsection{\type {mp:char_depth}}

\startfunctioncall
<number> w = mp:char_depth(<string> fontname, <number> char)
\stopfunctioncall

\section{The \type {node} library}

The \type {node} library contains functions that facilitate dealing with (lists
of) nodes and their values. They allow you to create, alter, copy, delete, and
insert \LUATEX\ node objects, the core objects within the typesetter.

\LUATEX\ nodes are represented in \LUA\ as userdata with the metadata type
\type {luatex.node}. The various parts within a node can be accessed using
named fields.

Each node has at least the three fields \type {next}, \type {id}, and \type
{subtype}:

\startitemize[intro]

\startitem
    The \type {next} field returns the userdata object for the next node in a
    linked list of nodes, or \type {nil}, if there is no next node.
\stopitem

\startitem
    The \type {id} indicates \TEX's \quote{node type}. The field \type {id} has a
    numeric value for efficiency reasons, but some of the library functions also
    accept a string value instead of \type {id}.
\stopitem

\startitem
    The \type {subtype} is another number. It often gives further information
    about a node of a particular \type {id}, but it is most important when
    dealing with \quote {whatsits}, because they are differentiated solely based
    on their \type {subtype}.
\stopitem

\stopitemize

The other available fields depend on the \type {id} (and for \quote {whatsits},
the \type {subtype}) of the node. Further details on the various fields and their
meanings are given in~\in{chapter}[nodes].

Support for \type {unset} (alignment) nodes is partial: they can be queried and
modified from \LUA\ code, but not created.

Nodes can be compared to each other, but: you are actually comparing indices into
the node memory. This means that equality tests can only be trusted under very
limited conditions. It will not work correctly in any situation where one of the
two nodes has been freed and|/|or reallocated: in that case, there will be false
positives.

At the moment, memory management of nodes should still be done explicitly by the
user. Nodes are not \quote {seen} by the \LUA\ garbage collector, so you have to
call the node freeing functions yourself when you are no longer in need of a node
(list). Nodes form linked lists without reference counting, so you have to be
careful that when control returns back to \LUATEX\ itself, you have not deleted
nodes that are still referenced from a \type {next} pointer elsewhere, and that
you did not create nodes that are referenced more than once.

There are statistics available with regards to the allocated node memory, which
can be handy for tracing.

\subsection{Node handling functions}

\subsubsection{\type {node.is_node}}

\startfunctioncall
<boolean> t = node.is_node(<any> item)
\stopfunctioncall

This function returns true if the argument is a userdata object of
type \type {<node>}.

\subsubsection{\type {node.types}}

\startfunctioncall
<table> t = node.types()
\stopfunctioncall

This function returns an array that maps node id numbers to node type strings,
providing an overview of the possible top|-|level \type {id} types.

\subsubsection{\type {node.whatsits}}

\startfunctioncall
<table> t = node.whatsits()
\stopfunctioncall

\TEX's \quote{whatsits} all have the same \type {id}. The various subtypes are
defined by their \type {subtype} fields. The function is much like \type
{node.types}, except that it provides an array of \type {subtype} mappings.

\subsubsection{\type {node.id}}

\startfunctioncall
<number> id = node.id(<string> type)
\stopfunctioncall

This converts a single type name to its internal numeric representation.

\subsubsection{\type {node.subtype}}

\startfunctioncall
<number> subtype = node.subtype(<string> type)
\stopfunctioncall

This converts a single whatsit name to its internal numeric representation (\type
{subtype}).

\subsubsection{\type {node.type}}

\startfunctioncall
<string> type = node.type(<any> n)
\stopfunctioncall

In the argument is a number, then this function converts an internal numeric
representation to an external string representation. Otherwise, it will return
the string \type {node} if the object represents a node, and \type {nil}
otherwise.

\subsubsection{\type {node.fields}}

\startfunctioncall
<table> t = node.fields(<number> id)
<table> t = node.fields(<number> id, <number> subtype)
\stopfunctioncall

This function returns an array of valid field names for a particular type of
node. If you want to get the valid fields for a \quote {whatsit}, you have to
supply the second argument also. In other cases, any given second argument will
be silently ignored.

This function accepts string \type {id} and \type {subtype} values as well.

\subsubsection{\type {node.has_field}}

\startfunctioncall
<boolean> t = node.has_field(<node> n, <string> field)
\stopfunctioncall

This function returns a boolean that is only true if \type {n} is
actually a node, and it has the field.

\subsubsection{\type {node.new}}

\startfunctioncall
<node> n = node.new(<number> id)
<node> n = node.new(<number> id, <number> subtype)
\stopfunctioncall

Creates a new node. All of the new node's fields are initialized to either zero
or \type {nil} except for \type {id} and \type {subtype} (if supplied). If you
want to create a new whatsit, then the second argument is required, otherwise it
need not be present. As with all node functions, this function creates a node on
the \TEX\ level.

This function accepts string \type {id} and \type {subtype} values as well.

\subsubsection{\type {node.free}}

\startfunctioncall
node.free(<node> n)
\stopfunctioncall

Removes the node \type {n} from \TEX's memory. Be careful: no checks are done on
whether this node is still pointed to from a register or some \type {next} field:
it is up to you to make sure that the internal data structures remain correct.

\subsubsection{\type {node.flush_list}}

\startfunctioncall
node.flush_list(<node> n)
\stopfunctioncall

Removes the node list \type {n} and the complete node list following \type {n}
from \TEX's memory. Be careful: no checks are done on whether any of these nodes
is still pointed to from a register or some \type {next} field: it is up to you
to make sure that the internal data structures remain correct.

\subsubsection{\type {node.copy}}

\startfunctioncall
<node> m = node.copy(<node> n)
\stopfunctioncall

Creates a deep copy of node \type {n}, including all nested lists as in the case
of a hlist or vlist node. Only the \type {next} field is not copied.

\subsubsection{\type {node.copy_list}}

\startfunctioncall
<node> m = node.copy_list(<node> n)
<node> m = node.copy_list(<node> n, <node> m)
\stopfunctioncall

Creates a deep copy of the node list that starts at \type {n}. If \type {m} is
also given, the copy stops just before node \type {m}.

Note that you cannot copy attribute lists this way, specialized functions for
dealing with attribute lists will be provided later but are not there yet.
However, there is normally no need to copy attribute lists as when you do
assignments to the \type {attr} field or make changes to specific attributes, the
needed copying and freeing takes place automatically.

\subsubsection{\type {node.next}}

\startfunctioncall
<node> m = node.next(<node> n)
\stopfunctioncall

Returns the node following this node, or \type {nil} if there is no such node.

\subsubsection{\type {node.prev}}

\startfunctioncall
<node> m = node.prev(<node> n)
\stopfunctioncall

Returns the node preceding this node, or \type {nil} if there is no such node.

\subsubsection{\type {node.current_attr}}

\startfunctioncall
<node> m = node.current_attr()
\stopfunctioncall

Returns the currently active list of attributes, if there is one.

The intended usage of \type {current_attr} is as follows:

\starttyping
local x1 = node.new("glyph")
x1.attr = node.current_attr()
local x2 = node.new("glyph")
x2.attr = node.current_attr()
\stoptyping

or:

\starttyping
local x1 = node.new("glyph")
local x2 = node.new("glyph")
local ca = node.current_attr()
x1.attr = ca
x2.attr = ca
\stoptyping

The attribute lists are ref counted and the assignment takes care of incrementing
the refcount. You cannot expect the value \type {ca} to be valid any more when
you assign attributes (using \type {tex.setattribute}) or when control has been
passed back to \TEX.

Note: this function is somewhat experimental, and it returns the {\it actual}
attribute list, not a copy thereof. Therefore, changing any of the attributes in
the list will change these values for all nodes that have the current attribute
list assigned to them.

\subsubsection{\type {node.hpack}}

\startfunctioncall
<node> h, <number> b = node.hpack(<node> n)
<node> h, <number> b = node.hpack(<node> n, <number> w, <string> info)
<node> h, <number> b = node.hpack(<node> n, <number> w, <string> info, <string> dir)
\stopfunctioncall

This function creates a new hlist by packaging the list that begins at node \type
{n} into a horizontal box. With only a single argument, this box is created using
the natural width of its components. In the three argument form, \type {info}
must be either \type {additional} or \type {exactly}, and \type {w} is the
additional (\type {\hbox spread}) or exact (\type {\hbox to}) width to be used. The
second return value is the badness of the generated box.

Caveat: at this moment, there can be unexpected side|-|effects to this function,
like updating some of the \type {\marks} and \type {\inserts}. Also note that the
content of \type {h} is the original node list \type {n}: if you call \type
{node.free(h)} you will also free the node list itself, unless you explicitly set
the \type {list} field to \type {nil} beforehand. And in a similar way, calling
\type {node.free(n)} will invalidate \type {h} as well!

\subsubsection{\type {node.vpack}}

\startfunctioncall
<node> h, <number> b = node.vpack(<node> n)
<node> h, <number> b = node.vpack(<node> n, <number> w, <string> info)
<node> h, <number> b = node.vpack(<node> n, <number> w, <string> info, <string> dir)
\stopfunctioncall

This function creates a new vlist by packaging the list that begins at node \type
{n} into a vertical box. With only a single argument, this box is created using
the natural height of its components. In the three argument form, \type {info}
must be either \type {additional} or \type {exactly}, and \type {w} is the
additional (\type {\vbox spread}) or exact (\type {\vbox to}) height to be used.

The second return value is the badness of the generated box.

See the description of \type {node.hpack()} for a few memory allocation caveats.

\subsubsection{\type {node.dimensions}}

\startfunctioncall
<number> w, <number> h, <number> d  = node.dimensions(<node> n)
<number> w, <number> h, <number> d  = node.dimensions(<node> n, <string> dir)
<number> w, <number> h, <number> d  = node.dimensions(<node> n, <node> t)
<number> w, <number> h, <number> d  = node.dimensions(<node> n, <node> t, <string> dir)
\stopfunctioncall

This function calculates the natural in-line dimensions of the node list starting
at node \type {n} and terminating just before node \type {t} (or the end of the
list, if there is no second argument). The return values are scaled points. An
alternative format that starts with glue parameters as the first three arguments
is also possible:

\startfunctioncall
<number> w, <number> h, <number> d  =
  node.dimensions(<number> glue_set, <number> glue_sign,
                 <number> glue_order, <node> n)
<number> w, <number> h, <number> d  =
  node.dimensions(<number> glue_set, <number> glue_sign,
                 <number> glue_order, <node> n, <string> dir)
<number> w, <number> h, <number> d  =
  node.dimensions(<number> glue_set, <number> glue_sign,
                 <number> glue_order, <node> n, <node> t)
<number> w, <number> h, <number> d  =
  node.dimensions(<number> glue_set, <number> glue_sign,
                 <number> glue_order, <node> n, <node> t, <string> dir)
\stopfunctioncall

This calling method takes glue settings into account and is especially useful for
finding the actual width of a sublist of nodes that are already boxed, for
example in code like this, which prints the width of the space in between the
\type {a} and \type {b} as it would be if \type {\box0} was used as-is:

\starttyping
\setbox0 = \hbox to 20pt {a b}

\directlua{print (node.dimensions(
    tex.box[0].glue_set,
    tex.box[0].glue_sign,
    tex.box[0].glue_order,
    tex.box[0].head.next,
    node.tail(tex.box[0].head)
)) }
\stoptyping

\subsubsection{\type {node.mlist_to_hlist}}

\startfunctioncall
<node> h = node.mlist_to_hlist(<node> n,
             <string> display_type, <boolean> penalties)
\stopfunctioncall

This runs the internal mlist to hlist conversion, converting the math list in
\type {n} into the horizontal list \type {h}. The interface is exactly the same
as for the callback \type {mlist_to_hlist}.

\subsubsection{\type {node.slide}}

\startfunctioncall
<node> m = node.slide(<node> n)
\stopfunctioncall

Returns the last node of the node list that starts at \type {n}. As a
side|-|effect, it also creates a reverse chain of \type {prev} pointers between
nodes.

\subsubsection{\type {node.tail}}

\startfunctioncall
<node> m = node.tail(<node> n)
\stopfunctioncall

Returns the last node of the node list that starts at \type {n}.

\subsubsection{\type {node.length}}

\startfunctioncall
<number> i = node.length(<node> n)
<number> i = node.length(<node> n, <node> m)
\stopfunctioncall

Returns the number of nodes contained in the node list that starts at \type {n}.
If \type {m} is also supplied it stops at \type {m} instead of at the end of the
list. The node \type {m} is not counted.

\subsubsection{\type {node.count}}

\startfunctioncall
<number> i = node.count(<number> id, <node> n)
<number> i = node.count(<number> id, <node> n, <node> m)
\stopfunctioncall

Returns the number of nodes contained in the node list that starts at \type {n}
that have a matching \type {id} field. If \type {m} is also supplied, counting
stops at \type {m} instead of at the end of the list. The node \type {m} is not
counted.

This function also accept string \type {id}'s.

\subsubsection{\type {node.traverse}}

\startfunctioncall
<node> t = node.traverse(<node> n)
\stopfunctioncall

This is a \LUA\ iterator that loops over the node list that starts at \type {n}.
Typically code looks like this:

\starttyping
for n in node.traverse(head) do
   ...
end
\stoptyping

is functionally equivalent to:

\starttyping
do
  local n
  local function f (head,var)
    local t
    if var == nil then
       t = head
    else
       t = var.next
    end
    return t
  end
  while true do
    n = f (head, n)
    if n == nil then break end
    ...
  end
end
\stoptyping

It should be clear from the definition of the function \type {f} that even though
it is possible to add or remove nodes from the node list while traversing, you
have to take great care to make sure all the \type {next} (and \type {prev})
pointers remain valid.

If the above is unclear to you, see the section \quote {For Statement} in the
\LUA\ Reference Manual.

\subsubsection{\type {node.traverse_id}}

\startfunctioncall
<node> t = node.traverse_id(<number> id, <node> n)
\stopfunctioncall

This is an iterator that loops over all the nodes in the list that starts at
\type {n} that have a matching \type {id} field.

See the previous section for details. The change is in the local function \type
{f}, which now does an extra while loop checking against the upvalue \type {id}:

\starttyping
 local function f(head,var)
   local t
   if var == nil then
      t = head
   else
      t = var.next
   end
   while not t.id == id do
      t = t.next
   end
   return t
 end
\stoptyping

\subsubsection{\type {node.end_of_math}}

\startfunctioncall
<node> t = node.end_of_math(<node> start)
\stopfunctioncall

Looks for and returns the next \type {math_node} following the \type {start}. If
the given node is a math endnode this helper return that node, else it follows
the list and return the next math endnote. If no such node is found nil is
returned.

\subsubsection{\type {node.remove}}

\startfunctioncall
<node> head, current = node.remove(<node> head, <node> current)
\stopfunctioncall

This function removes the node \type {current} from the list following \type
{head}. It is your responsibility to make sure it is really part of that list.
The return values are the new \type {head} and \type {current} nodes. The
returned \type {current} is the node following the \type {current} in the calling
argument, and is only passed back as a convenience (or \type {nil}, if there is
no such node). The returned \type {head} is more important, because if the
function is called with \type {current} equal to \type {head}, it will be
changed.

\subsubsection{\type {node.insert_before}}

\startfunctioncall
<node> head, new = node.insert_before(<node> head, <node> current, <node> new)
\stopfunctioncall

This function inserts the node \type {new} before \type {current} into the list
following \type {head}. It is your responsibility to make sure that \type
{current} is really part of that list. The return values are the (potentially
mutated) \type {head} and the node \type {new}, set up to be part of the list
(with correct \type {next} field). If \type {head} is initially \type {nil}, it
will become \type {new}.

\subsubsection{\type {node.insert_after}}

\startfunctioncall
<node> head, new = node.insert_after(<node> head, <node> current, <node> new)
\stopfunctioncall

This function inserts the node \type {new} after \type {current} into the list
following \type {head}. It is your responsibility to make sure that \type
{current} is really part of that list. The return values are the \type {head} and
the node \type {new}, set up to be part of the list (with correct \type {next}
field). If \type {head} is initially \type {nil}, it will become \type {new}.

\subsubsection{\type {node.first_glyph}}

\startfunctioncall
<node> n = node.first_glyph(<node> n)
<node> n = node.first_glyph(<node> n, <node> m)
\stopfunctioncall

Returns the first node in the list starting at \type {n} that is a glyph node
with a subtype indicating it is a glyph, or \type {nil}. If \type {m} is given,
processing stops at (but including) that node, otherwise processing stops at the
end of the list.

\subsubsection{\type {node.ligaturing}}

\startfunctioncall
<node> h, <node> t, <boolean> success = node.ligaturing(<node> n)
<node> h, <node> t, <boolean> success = node.ligaturing(<node> n, <node> m)
\stopfunctioncall

Apply \TEX-style ligaturing to the specified nodelist. The tail node \type {m} is
optional. The two returned nodes \type {h} and \type {t} are the new head and
tail (both \type {n} and \type {m} can change into a new ligature).

\subsubsection{\type {node.kerning}}

\startfunctioncall
<node> h, <node> t, <boolean> success = node.kerning(<node> n)
<node> h, <node> t, <boolean> success = node.kerning(<node> n, <node> m)
\stopfunctioncall

Apply \TEX|-|style kerning to the specified node list. The tail node \type {m} is
optional. The two returned nodes \type {h} and \type {t} are the head and tail
(either one of these can be an inserted kern node, because special kernings with
word boundaries are possible).

\subsubsection{\type {node.unprotect_glyphs}}

\startfunctioncall
node.unprotect_glyphs(<node> n)
\stopfunctioncall

Subtracts 256 from all glyph node subtypes. This and the next function are
helpers to convert from \type {characters} to \type {glyphs} during node
processing.

\subsubsection{\type {node.protect_glyphs}}

\startfunctioncall
node.protect_glyphs(<node> n)
\stopfunctioncall

Adds 256 to all glyph node subtypes in the node list starting at \type {n},
except that if the value is 1, it adds only 255. The special handling of 1 means
that \type {characters} will become \type {glyphs} after subtraction of 256.

\subsubsection{\type {node.last_node}}

\startfunctioncall
<node> n = node.last_node()
\stopfunctioncall

This function pops the last node from \TEX's \quote{current list}. It returns
that node, or \type {nil} if the current list is empty.

\subsubsection{\type {node.write}}

\startfunctioncall
node.write(<node> n)
\stopfunctioncall

This is an experimental function that will append a node list to \TEX's \quote
{current list} The node list is not deep|-|copied! There is no error checking
either!

\subsubsection{\type {node.protrusion_skippable}}
\startfunctioncall
<boolean> skippable = node.protrusion_skippable(<node> n)
\stopfunctioncall

Returns \type {true} if, for the purpose of line boundary discovery when
character protrusion is active, this node can be skipped.

\subsection{Attribute handling}

Attributes appear as linked list of userdata objects in the \type {attr} field of
individual nodes. They can be handled individually, but it is much safer and more
efficient to use the dedicated functions associated with them.

\subsubsection{\type {node.has_attribute}}

\startfunctioncall
<number> v = node.has_attribute(<node> n, <number> id)
<number> v = node.has_attribute(<node> n, <number> id, <number> val)
\stopfunctioncall

Tests if a node has the attribute with number \type {id} set. If \type {val} is
also supplied, also tests if the value matches \type {val}. It returns the value,
or, if no match is found, \type {nil}.

\subsubsection{\type {node.set_attribute}}

\startfunctioncall
node.set_attribute(<node> n, <number> id, <number> val)
\stopfunctioncall

Sets the attribute with number \type {id} to the value \type {val}. Duplicate
assignments are ignored. {\em [needs explanation]}

\subsubsection{\type {node.unset_attribute}}

\startfunctioncall
<number> v = node.unset_attribute(<node> n, <number> id)
<number> v = node.unset_attribute(<node> n, <number> id, <number> val)
\stopfunctioncall

Unsets the attribute with number \type {id}. If \type {val} is also supplied, it
will only perform this operation if the value matches \type {val}. Missing
attributes or attribute|-|value pairs are ignored.

If the attribute was actually deleted, returns its old value. Otherwise, returns
\type {nil}.

\section{The \type {pdf} library}

This contains variables and functions that are related to the \PDF\ backend.

\subsection{\type {pdf.mapfile}, \type {pdf.mapline}}

\startfunctioncall
pdf.mapfile(<string> map file)
pdf.mapline(<string> map line)
\stopfunctioncall

These two functions can be used to replace primitives \type {\pdfmapfile} and
\type {\pdfmapline} from \PDFTEX. They expect a string as only parameter and have
no return value.

The also functions replace the former variables \type {pdf.pdfmapfile} and
\type {pdf.pdfmapline}.

\subsection{\type {pdf.catalog}, \type {pdf.info},\type {pdf.names},
    \type {pdf.trailer}}

These variables offer a read|-|write interface to the corresponding \PDFTEX\
token lists. The value types are strings and they are written out to the \PDF\
file directly after the \PDFTEX\ token registers.

The preferred interface is now \type {pdf.setcatalog}, \type {pdf.setinfo}
\type {pdf.setnames} and \type {pdf.settrailer} for setting these properties
and \type {pdf.getcatalog}, \type {pdf.getinfo} \type {pdf.getnames} and
\type {pdf.gettrailer} for querying them,

The corresponding \quote {\type {pdf}} parameter names \type {pdf.pdfcatalog},
\type {pdf.pdfinfo}, \type {pdf.pdfnames}, and \type {pdf.pdftrailer} are
not available.

\subsection{\type {pdf.<set/get>pageattributes}, \type {pdf.<set/get>pageresources},
    \type {pdf.<set/get>pagesattributes}}

These variables offer a read|-|write interface to related token lists. The value
types are strings. The variables have no interaction with the corresponding
\PDFTEX\ token registers \type {\pdfpageattr}, \type {\pdfpageresources}, and \type
{\pdfpagesattr}. They are written out to the \PDF\ file directly after the
\PDFTEX\ token registers.

The preferred interface is now \type {pdf.setpageattributes}, \type
{pdf.setpagesattributes} and \type {pdf.setpageresources} for setting these
properties and \type {pdf.getpageattributes}, \type {pdf.getpageattributes}
and \type {pdf.getpageresources} for querying them.

\subsection{\type {pdf.<set/get>xformattributes}, \type {pdf.<set/get>xformresources}}

These variables offer a read|-|write interface to related token lists. The value
types are strings. The variables have no interaction with the corresponding
\PDFTEX\ token registers \type {\pdfxformattr} and \type {\pdfxformresources}. They
are written out to the \PDF\ file directly after the \PDFTEX\ token registers.

The preferred interface is now \type {pdf.setxformattributes} and \type
{pdf.setxformattributes} for setting these properties and \type
{pdf.getxformattributes} and \type {pdf.getxformresources} for querying them.

\subsection{\type {pdf.setcompresslevel} and \type {pdf.setobjcompresslevel}}

These two functions set the level of compression. The minimum valu sis~0,
the maximum is~9.

\subsection{\type {pdf.setdecimaldigits} and \type {pdf.getdecimaldigits}}

These two functions set the accuracy of floats written to the \PDF file. You can
set any value but the backend will not go below 3 and above 6.

\subsection{\type {pdf.setpkresolution} and \type {pdf.getpkresolution}}

These setter takes two arguments: the resolution and an optional zero or one that
indicates if this is a fixed one. The getter returns these two values.

\subsection{\type {pdf.lastobj}, \type {pdf.lastlink}, \type {pdf.lastannot},
and \type {pdf.retval}}

These status variables are similar to the ones traditionally used at the \TEX\
end.

\subsection{\type {pdf.setorigin}, \type {pdf.getorigin}}

This one is used to set the horizonal and/or vertical offset (a traditional
backend property).

\starttyping
pdf.setorigin() -- sets both to 0pt
pdf.setorigin(tex.sp("1in")) -- sets both to 1in
pdf.setorigin(tex.sp("1in"),tex.sp("1in"))
\stoptyping

The counterpart of this function returns two values.

\subsection{\type {pdf.setlinkmargin}, \type {pdf.getlinkmargin} \type
{pdf.setdestmargin}, \type {pdf.getdestmargin} \type {pdf.setthreadmargin},
\type {pdf.getthreadmargin} \type {pdf.setxformmargin}, \type
{pdf.getxformmargin}}

These function can be used to set and retrieve the margins that are added to the
natural bounding boxes of the respective objects.

\subsection{\type {pdf.h}, \type {pdf.v}}

These are the \type {h} and \type {v} values that define the current location on
the output page, measured from its lower left corner. The values can be queried
using scaled points as units.

\starttyping
local h = pdf.h
local v = pdf.v
\stoptyping

\subsection{\type {pdf.getpos}, \type {pdf.gethpos}, \type {pdf.getvpos}}

These are the function variants of \type {pdf.h} and \type {pdf.v}. Sometimes
using a function is preferred over a key so this saves wrapping. Also, these
functions are faster then the key based access, as \type {h} and \type {v} keys
are not real variables but looked up using a metatable call. The \type {getpos}
function returns two values, the other return one.

\starttyping
local h, v = pdf.getpos()
\stoptyping

\subsection{\type {pdf.hasmatrix}, \type {pdf.getmatrix}}

The current matrix transformation is available via the \type {getmatrix} command,
which returns 6 values: \type {sx}, \type {rx}, \type {ry}, \type {sy}, \type
{tx}, and \type {ty}. The \type {hasmatrix} function returns \type {true} when a
matrix is applied.

\starttyping
if pdf.hasmatrix() then
    local sx, rx, ry, sy, tx, ty = pdf.getmatrix()
    -- do something useful or not
end
\stoptyping

\subsection{\type {pdf.print}}

A print function to write stuff to the \PDF\ document that can be used from
within a \type {\latelua} argument. This function is not to be used inside
\type {\directlua} unless you know {\it exactly} what you are doing.

\startfunctioncall
pdf.print(<string> s)
pdf.print(<string> type, <string> s)
\stopfunctioncall

The optional parameter can be used to mimic the behavior of \type {\pdfliteral}:
the \type {type} is \type {direct} or \type {page}.

\subsection{\type {pdf.immediateobj}}

This function creates a \PDF\ object and immediately writes it to the \PDF\ file.
It is modelled after \PDFTEX's \type {\immediate} \type {\pdfobj} primitives. All
function variants return the object number of the newly generated object.

\startfunctioncall
<number> n = pdf.immediateobj(<string> objtext)
<number> n = pdf.immediateobj("file", <string> filename)
<number> n = pdf.immediateobj("stream", <string> streamtext, <string> attrtext)
<number> n = pdf.immediateobj("streamfile", <string> filename, <string> attrtext)
\stopfunctioncall

The first version puts the \type {objtext} raw into an object. Only the object
wrapper is automatically generated, but any internal structure (like \type {<<
>>} dictionary markers) needs to provided by the user. The second version with
keyword \type {"file"} as 1st argument puts the contents of the file with name
\type {filename} raw into the object. The third version with keyword \type
{"stream"} creates a stream object and puts the \type {streamtext} raw into the
stream. The stream length is automatically calculated. The optional \type
{attrtext} goes into the dictionary of that object. The fourth version with
keyword \type {"streamfile"} does the same as the 3rd one, it just reads the
stream data raw from a file.

An optional first argument can be given to make the function use a previously
reserved \PDF\ object.

\startfunctioncall
<number> n = pdf.immediateobj(<integer> n, <string> objtext)
<number> n = pdf.immediateobj(<integer> n, "file", <string> filename)
<number> n = pdf.immediateobj(<integer> n, "stream", <string> streamtext, <string> attrtext)
<number> n = pdf.immediateobj(<integer> n, "streamfile", <string> filename, <string> attrtext)
\stopfunctioncall

\subsection{\type {pdf.obj}}

This function creates a \PDF\ object, which is written to the \PDF\ file only
when referenced, e.g., by \type {pdf.refobj()}.

All function variants return the object number of the newly generated object, and
there are two separate calling modes.

The first mode is modelled after \PDFTEX's \type {\pdfobj} primitive.

\startfunctioncall
<number> n = pdf.obj(<string> objtext)
<number> n = pdf.obj("file", <string> filename)
<number> n = pdf.obj("stream", <string> streamtext, <string> attrtext)
<number> n = pdf.obj("streamfile", <string> filename, <string> attrtext)
\stopfunctioncall

An optional first argument can be given to make the function use a previously
reserved \PDF\ object.

\startfunctioncall
<number> n = pdf.obj(<integer> n, <string> objtext)
<number> n = pdf.obj(<integer> n, "file", <string> filename)
<number> n = pdf.obj(<integer> n, "stream", <string> streamtext, <string> attrtext)
<number> n = pdf.obj(<integer> n, "streamfile", <string> filename, <string> attrtext)
\stopfunctioncall

The second mode accepts a single argument table with key--value pairs.

\startfunctioncall
<number> n = pdf.obj {
    type           = <string>,
    immmediate     = <boolean>,
    objnum         = <number>,
    attr           = <string>,
    compresslevel  = <number>,
    objcompression = <boolean>,
    file           = <string>,
    string         = <string>
}
\stopfunctioncall

The \type {type} field can have the values \type {raw} and \type {stream}, this
field is required, the others are optional (within constraints).

Note: this mode makes \type {pdf.obj} look more flexible than it actually is: the
constraints from the separate parameter version still apply, so for example you
can't have both \type {string} and \type {file} at the same time.

\subsection{\type {pdf.refobj}}

This function, the \LUA\ version of the \type {\pdfrefobj} primitive, references an
object by its object number, so that the object will be written out.

\startfunctioncall
pdf.refobj(<integer> n)
\stopfunctioncall

This function works in both the \type {\directlua} and \type {\latelua} environment.
Inside \type {\directlua} a new whatsit node \quote {pdf_refobj} is created, which
will be marked for flushing during page output and the object is then written
directly after the page, when also the resources objects are written out. Inside
\type {\latelua} the object will be marked for flushing.

This function has no return values.

\subsection{\type {pdf.reserveobj}}

This function creates an empty \PDF\ object and returns its number.

\startfunctioncall
<number> n = pdf.reserveobj()
<number> n = pdf.reserveobj("annot")
\stopfunctioncall

\subsection{\type {pdf.registerannot}}

This function adds an object number to the \type {/Annots} array for the current
page without doing anything else. This function can only be used from within
\type {\latelua}.

\startfunctioncall
pdf.registerannot (<number> objnum)
\stopfunctioncall

\subsection{\type {pdf.newcolorstack}}

This function allocates a new color stack and returns it's id. The arguments
are the same as for the similar backend extension primitive.

\startfunctioncall
pdf.newcolorstack("0 g","page",true) -- page|direct|origin
\stopfunctioncall

\section{The \type {pdfscanner} library}

The \type {pdfscanner} library allows interpretation of PDF content streams and
\type {/ToUnicode} (cmap) streams. You can get those streams from the \type
{epdf} library, as explained in an earlier section. There is only a single
top|-|level function in this library:

\startfunctioncall
pdfscanner.scan (<Object> stream, <table> operatortable, <table> info)
\stopfunctioncall

The first argument, \type {stream}, should be either a PDF stream object, or a
PDF array of PDF stream objects (those options comprise the possible return
values of \type {<Page>:getContents()} and \type {<Object>:getStream()} in the
\type {epdf} library).

The second argument, \type {operatortable}, should be a Lua table where the keys
are PDF operator name strings and the values are Lua functions (defined by you)
that are used to process those operators. The functions are called whenever the
scanner finds one of these PDF operators in the content stream(s). The functions
are called with two arguments: the \type {scanner} object itself, and the \type
{info} table that was passed are the third argument to \type {pdfscanner.scan}.

Internally, \type {pdfscanner.scan} loops over the PDF operators in the
stream(s), collecting operands on an internal stack until it finds a PDF
operator. If that PDF operator's name exists in \type {operatortable}, then the
associated function is executed. After the function has run (or when there is no
function to execute) the internal operand stack is cleared in preparation for the
next operator, and processing continues.

The \type {scanner} argument to the processing functions is needed because it
offers various methods to get the actual operands from the internal operand
stack.

A simple example of processing a PDF's document stream could look like this:

\starttyping
function Do (scanner, info)
   local val       = scanner:pop()
   local name      = val[2] -- val[1] == 'name'
   local resources = info.resources
   local xobject   = resources:lookup("XObject"):getDict():lookup(name)
   print (info.space ..'Use XObject '.. name)
   if xobject and xobject:isStream() then
      local dict = xobject:getStream():getDict()
      if dict then
        local name = dict:lookup("Subtype")
        if name:getName() == "Form" then
          local newinfo =  {
            space = info.space .. "  " ,
            resources = dict:lookup("Resources"):getDict()
          }
          pdfscanner.scan(xobject, operatortable, newinfo)
        end
      end
   end
end

operatortable = { Do = Do }

doc      = epdf.open(arg[1])
pagenum  = 1

while pagenum <= doc:getNumPages() do
   local page = doc:getCatalog():getPage(pagenum)
   local info = {
     space     = "  " ,
     resources = page:getResourceDict()
   }
   print('Page ' .. pagenum)
   pdfscanner.scan(page:getContents(), operatortable, info)
   pagenum = pagenum + 1
end
\stoptyping

This example iterates over all the actual content in the PDF, and prints out the
found XObject names. While the code demonstrates quite some of the \type {epdf}
functions, let's focus on the type \type {pdfscanner} specific code instead.

From the bottom up, the line

\starttyping
   pdfscanner.scan(page:getContents(), operatortable, info)
\stoptyping

runs the scanner with the PDF page's top-level content.

The third argument, \type {info}, contains two entries: \type {space} is used to
indent the printed output, and \type {resources} is needed so that embedded \type
{XForms} can find their own content.

The second argument, \type {operatortable} defines a processing function for a
single PDF operator, \type {Do}.

The function \type {Do} prints the name of the current XObject, and then starts a
new scanner for that object's content stream, under the condition that the
XObject is in fact a \type {/Form}. That nested scanner is called with new \type
{info} argument with an updated \type {space} value so that the indentation of
the output nicely nests, and with an new \type {resources} field to help the next
iteration down to properly process any other, embedded XObjects.

Of course, this is not a very useful example in practise, but for the purpose of
demonstrating \type {pdfscanner}, it is just long enough. It makes use of only
one \type {scanner} method: \type {scanner:pop()}. That function pops the top
operand of the internal stack, and returns a \LUA\ table where the object at index
one is a string representing the type of the operand, and object two is its
value.

The list of possible operand types and associated \LUA\ value types is:

\starttabulate[|lT|p|]
\NC integer  \NC <number>  \NC \NR
\NC real     \NC <number>  \NC \NR
\NC boolean  \NC <boolean> \NC \NR
\NC name     \NC <string>  \NC \NR
\NC operator \NC <string>  \NC \NR
\NC string   \NC <string>  \NC \NR
\NC array    \NC <table>   \NC \NR
\NC dict     \NC <table>   \NC \NR
\stoptabulate

In case of \type {integer} or \type {real}, the value is always a \LUA\ (floating
point) number.

In case of \type {name}, the leading slash is always stripped.

In case of \type {string}, please bear in mind that PDF actually supports
different types of strings (with different encodings) in different parts of the
PDF document, so may need to reencode some of the results; \type {pdfscanner}
always outputs the byte stream without reencoding anything. \type {pdfscanner}
does not differentiate between literal strings and hexadecimal strings (the
hexadecimal values are decoded), and it treats the stream data for inline images
as a string that is the single operand for \type {EI}.

In case of \type {array}, the table content is a list of \type {pop} return
values.

In case of \type {dict}, the table keys are PDF name strings and the values are
\type {pop} return values.

\blank

There are few more methods defined that you can ask \type {scanner}:

\starttabulate[|lT|p|]
\NC pop       \NC as explained above \NC \NR
\NC popNumber \NC return only the value of a \type {real} or \type {integer} \NC \NR
\NC popName   \NC return only the value of a \type {name} \NC \NR
\NC popString \NC return only the value of a \type {string} \NC \NR
\NC popArray  \NC return only the value of a \type {array} \NC \NR
\NC popDict   \NC return only the value of a \type {dict} \NC \NR
\NC popBool   \NC return only the value of a \type {boolean} \NC \NR
\NC done      \NC abort further processing of this \type {scan()} call \NC \NR
\stoptabulate

The \type {popXXX} are convenience functions, and come in handy when you know the
type of the operands beforehand (which you usually do, in PDF). For example, the
\type {Do} function could have used \type {local name = scanner:popName()}
instead, because the single operand to the \type {Do} operator is always a PDF
name object.

The \type {done} function allows you to abort processing of a stream once you
have learned everything you want to learn. This comes in handy while parsing
\type {/ToUnicode}, because there usually is trailing garbage that you are not
interested in. Without \type {done}, processing only end at the end of the
stream, possibly wasting CPU cycles.

\section{The \type {status} library}

This contains a number of run|-|time configuration items that you may find useful
in message reporting, as well as an iterator function that gets all of the names
and values as a table.

\startfunctioncall
<table> info = status.list()
\stopfunctioncall

The keys in the table are the known items, the value is the current value. Almost
all of the values in \type {status} are fetched through a metatable at run|-|time
whenever they are accessed, so you cannot use \type {pairs} on \type {status},
but you {\it can\/} use \type {pairs} on \type {info}, of course. If you do not
need the full list, you can also ask for a single item by using its name as an
index into \type {status}.

The current list is:

\starttabulate[|lT|p|]
\NC \ssbf key          \NC \bf explanation \NC \NR
\NC pdf_gone           \NC written \PDF\ bytes \NC \NR
\NC pdf_ptr            \NC not yet written \PDF\ bytes \NC \NR
\NC dvi_gone           \NC written \DVI\ bytes \NC \NR
\NC dvi_ptr            \NC not yet written \DVI\ bytes \NC \NR
\NC total_pages        \NC number of written pages \NC \NR
\NC output_file_name   \NC name of the \PDF\ or \DVI\ file \NC \NR
\NC log_name           \NC name of the log file \NC \NR
\NC banner             \NC terminal display banner \NC \NR
\NC var_used           \NC variable (one|-|word) memory in use \NC \NR
\NC dyn_used           \NC token (multi|-|word) memory in use  \NC \NR
\NC str_ptr            \NC number of strings \NC \NR
\NC init_str_ptr       \NC number of \INITEX\ strings \NC \NR
\NC max_strings        \NC maximum allowed strings \NC \NR
\NC pool_ptr           \NC string pool index \NC \NR
\NC init_pool_ptr      \NC \INITEX\ string pool index \NC \NR
\NC pool_size          \NC current size allocated for string characters \NC \NR
\NC node_mem_usage     \NC a string giving insight into currently used nodes \NC \NR
\NC var_mem_max        \NC number of allocated words for nodes \NC \NR
\NC fix_mem_max        \NC number of allocated words for tokens \NC \NR
\NC fix_mem_end        \NC maximum number of used tokens \NC \NR
\NC cs_count           \NC number of control sequences \NC \NR
\NC hash_size          \NC size of hash \NC \NR
\NC hash_extra         \NC extra allowed hash \NC \NR
\NC font_ptr           \NC number of active fonts \NC \NR
\NC input_ptr          \NC th elevel of input we're at \NC \NR
\NC max_in_stack       \NC max used input stack entries \NC \NR
\NC max_nest_stack     \NC max used nesting stack entries \NC \NR
\NC max_param_stack    \NC max used parameter stack entries \NC \NR
\NC max_buf_stack      \NC max used buffer position \NC \NR
\NC max_save_stack     \NC max used save stack entries \NC \NR
\NC stack_size         \NC input stack size \NC \NR
\NC nest_size          \NC nesting stack size \NC \NR
\NC param_size         \NC parameter stack size \NC \NR
\NC buf_size           \NC current allocated size of the line buffer \NC \NR
\NC save_size          \NC save stack size \NC \NR
\NC obj_ptr            \NC max \PDF\ object pointer \NC \NR
\NC obj_tab_size       \NC \PDF\ object table size \NC \NR
\NC pdf_os_cntr        \NC max \PDF\ object stream pointer \NC \NR
\NC pdf_os_objidx      \NC \PDF\ object stream index \NC \NR
\NC pdf_dest_names_ptr \NC max \PDF\ destination pointer \NC \NR
\NC dest_names_size    \NC \PDF\ destination table size \NC \NR
\NC pdf_mem_ptr        \NC max \PDF\ memory used \NC \NR
\NC pdf_mem_size       \NC \PDF\ memory size \NC \NR
\NC largest_used_mark  \NC max referenced marks class \NC \NR
\NC filename           \NC name of the current input file \NC \NR
\NC inputid            \NC numeric id of the current input \NC \NR
\NC linenumber         \NC location in the current input file \NC \NR
\NC lasterrorstring    \NC last \TEX\ error string \NC \NR
\NC lastluaerrorstring \NC last \LUA\ error string \NC \NR
\NC lastwarningtag     \NC last warning string\NC \NR
\NC lastwarningstring  \NC last warning tag, normally an indication of in what part\NC \NR
\NC lasterrorcontext   \NC last error context string (with newlines) \NC \NR
\NC luabytecodes       \NC number of active \LUA\ bytecode registers \NC \NR
\NC luabytecode_bytes  \NC number of bytes in \LUA\ bytecode registers \NC \NR
\NC luastate_bytes     \NC number of bytes in use by \LUA\ interpreters \NC \NR
\NC output_active      \NC \type {true} if the \type {\output} routine is active \NC \NR
\NC callbacks          \NC total number of executed callbacks so far \NC \NR
\NC indirect_callbacks \NC number of those that were themselves
                           a result of other callbacks (e.g. file readers) \NC \NR
\NC luatex_version     \NC the \LUATEX\ version number \NC \NR
\NC luatex_revision    \NC the \LUATEX\ revision string \NC \NR
\NC ini_version        \NC \type {true} if this is an \INITEX\ run \NC \NR
\NC shell_escape       \NC \type {0} means disabled, \type {1} is restricted and
                           \type {2} means anything is permitted \NC \NR
\stoptabulate

The error and warning messages can be wiped with the \type {resetmessages}
function.

\section{The \type {tex} library}

The \type {tex} table contains a large list of virtual internal \TEX\
parameters that are partially writable.

The designation \quote {virtual} means that these items are not properly defined
in \LUA, but are only front\-ends that are handled by a metatable that operates
on the actual \TEX\ values. As a result, most of the \LUA\ table operators (like
\type {pairs} and \type {#}) do not work on such items.

At the moment, it is possible to access almost every parameter that has these
characteristics:

\startitemize[packed]
\item You can use it after \type {\the}
\item It is a single token.
\item Some special others, see the list below
\stopitemize

This excludes parameters that need extra arguments, like \type {\the\scriptfont}.

The subset comprising simple integer and dimension registers are
writable as well as readable (stuff like \type {\tracingcommands} and
\type {\parindent}).

\subsection{Internal parameter values}

For all the parameters in this section, it is possible to access them directly
using their names as index in the \type {tex} table, or by using one of the
functions \type {tex.get} and \type {tex.set}. If you created aliasses,
you can use accessors like \type {tex.getdimen} as these also understand
names of built|-|in variables.

The exact parameters and return values differ depending on the actual parameter,
and so does whether \type {tex.set} has any effect. For the parameters that {\it
can\/} be set, it is possible to use \type {global} as the first argument to
\type {tex.set}; this makes the assignment global instead of local.

\startfunctioncall
tex.set (<string> n, ...)
tex.set ("global", <string> n, ...)
... = tex.get (<string> n)
\stopfunctioncall

There are also dedicated setters, getters and checkers:

\startfunctioncall
local d = tex.getdimen("foo")
if tex.isdimen("bar") then
    tex.setdimen("bar",d)
end
\stopfunctioncall

There are such helpers for \type {dimen}, \type {count}, \type {skip}, \type
{box} and \type {attribute} registers.

\subsubsection{Integer parameters}

The integer parameters accept and return \LUA\ numbers.

Read|-|write:

\starttwocolumns
\starttyping
tex.adjdemerits
tex.binoppenalty
tex.brokenpenalty
tex.catcodetable
tex.clubpenalty
tex.day
tex.defaulthyphenchar
tex.defaultskewchar
tex.delimiterfactor
tex.displaywidowpenalty
tex.doublehyphendemerits
tex.endlinechar
tex.errorcontextlines
tex.escapechar
tex.exhyphenpenalty
tex.fam
tex.finalhyphendemerits
tex.floatingpenalty
tex.globaldefs
tex.hangafter
tex.hbadness
tex.holdinginserts
tex.hyphenpenalty
tex.interlinepenalty
tex.language
tex.lastlinefit
tex.lefthyphenmin
tex.linepenalty
tex.localbrokenpenalty
tex.localinterlinepenalty
tex.looseness
tex.mag
tex.maxdeadcycles
tex.month
tex.newlinechar
tex.outputpenalty
tex.pausing
tex.postdisplaypenalty
tex.predisplaydirection
tex.predisplaypenalty
tex.pretolerance
tex.relpenalty
tex.righthyphenmin
tex.savinghyphcodes
tex.savingvdiscards
tex.showboxbreadth
tex.showboxdepth
tex.time
tex.tolerance
tex.tracingassigns
tex.tracingcommands
tex.tracinggroups
tex.tracingifs
tex.tracinglostchars
tex.tracingmacros
tex.tracingnesting
tex.tracingonline
tex.tracingoutput
tex.tracingpages
tex.tracingparagraphs
tex.tracingrestores
tex.tracingscantokens
tex.tracingstats
tex.uchyph
tex.vbadness
tex.widowpenalty
tex.year
\stoptyping
\stoptwocolumns

Read|-|only:

\startthreecolumns
\starttyping
tex.deadcycles
tex.insertpenalties
tex.parshape
tex.prevgraf
tex.spacefactor
\stoptyping
\stopthreecolumns

\subsubsection{Dimension parameters}

The dimension parameters accept \LUA\ numbers (signifying scaled points) or
strings (with included dimension). The result is always a number in scaled
points.

Read|-|write:

\startthreecolumns
\starttyping
tex.boxmaxdepth
tex.delimitershortfall
tex.displayindent
tex.displaywidth
tex.emergencystretch
tex.hangindent
tex.hfuzz
tex.hoffset
tex.hsize
tex.lineskiplimit
tex.mathsurround
tex.maxdepth
tex.nulldelimiterspace
tex.overfullrule
tex.pagebottomoffset
tex.pageheight
tex.pageleftoffset
tex.pagerightoffset
tex.pagetopoffset
tex.pagewidth
tex.parindent
tex.predisplaysize
tex.scriptspace
tex.splitmaxdepth
tex.vfuzz
tex.voffset
tex.vsize
tex.prevdepth
tex.prevgraf
tex.spacefactor
\stoptyping
\stopthreecolumns

Read|-|only:

\startthreecolumns
\starttyping
tex.pagedepth
tex.pagefilllstretch
tex.pagefillstretch
tex.pagefilstretch
tex.pagegoal
tex.pageshrink
tex.pagestretch
tex.pagetotal
\stoptyping
\stopthreecolumns

Beware: as with all \LUA\ tables you can add values to them. So, the following is valid:

\starttyping
tex.foo = 123
\stoptyping

When you access a \TEX\ parameter a look up takes place. For read||only variables
that means that you will get something back, but when you set them you create a
new entry in the table thereby making the original invisible.

There are a few special cases that we make an exception for: \type {prevdepth},
\type {prevgraf} and \type {spacefactor}. These normally are accessed via the
\type {tex.nest} table:

\starttyping
tex.nest[tex.nest.ptr].prevdepth   = p
tex.nest[tex.nest.ptr].spacefactor = s
\stoptyping

However, the following also works:

\starttyping
tex.prevdepth   = p
tex.spacefactor = s
\stoptyping

Keep in mind that when you mess with node lists directly at the \LUA\ end you
might need to update the top of the nesting stack's \type {prevdepth} explicitly
as there is no way \LUATEX\ can guess your intentions. By using the accessor in
the \type {tex} tables, you get and set the values atthe top of the nest stack.

\subsubsection{Direction parameters}

The direction parameters are read|-|only and return a \LUA\ string.

\startthreecolumns
\starttyping
tex.bodydir
tex.mathdir
tex.pagedir
tex.pardir
tex.textdir
\stoptyping
\stopthreecolumns

\subsubsection{Glue parameters}

The glue parameters accept and return a userdata object that represents a \type
{glue_spec} node.

\startthreecolumns
\starttyping
tex.abovedisplayshortskip
tex.abovedisplayskip
tex.baselineskip
tex.belowdisplayshortskip
tex.belowdisplayskip
tex.leftskip
tex.lineskip
tex.parfillskip
tex.parskip
tex.rightskip
tex.spaceskip
tex.splittopskip
tex.tabskip
tex.topskip
tex.xspaceskip
\stoptyping
\stopthreecolumns

\subsubsection{Muglue parameters}

All muglue parameters are to be used read|-|only and return a \LUA\ string.

\startthreecolumns
\starttyping
tex.medmuskip
tex.thickmuskip
tex.thinmuskip
\stoptyping
\stopthreecolumns

\subsubsection{Tokenlist parameters}

The tokenlist parameters accept and return \LUA\ strings. \LUA\ strings are
converted to and from token lists using \type {\the} \type {\toks} style expansion:
all category codes are either space (10) or other (12). It follows that assigning
to some of these, like \quote {tex.output}, is actually useless, but it feels bad
to make exceptions in view of a coming extension that will accept full|-|blown
token strings.

\startthreecolumns
\starttyping
tex.errhelp
tex.everycr
tex.everydisplay
tex.everyeof
tex.everyhbox
tex.everyjob
tex.everymath
tex.everypar
tex.everyvbox
tex.output
tex.pdfpageattr
tex.pdfpageresources
tex.pdfpagesattr
tex.pdfpkmode
\stoptyping
\stopthreecolumns

\subsection{Convert commands}

All \quote {convert} commands are read|-|only and return a \LUA\ string. The
supported commands at this moment are:

\starttwocolumns
\starttyping
tex.eTeXVersion
tex.eTeXrevision
tex.formatname
tex.jobname
tex.luatexbanner
tex.luatexrevision
tex.pdfnormaldeviate
tex.fontname(number)
tex.pdffontname(number)
tex.pdffontobjnum(number)
tex.pdffontsize(number)
tex.uniformdeviate(number)
tex.number(number)
tex.romannumeral(number)
tex.pdfpageref(number)
tex.pdfxformname(number)
tex.fontidentifier(number)
\stoptyping
\stoptwocolumns

If you are wondering why this list looks haphazard; these are all the cases of
the \quote {convert} internal command that do not require an argument, as well as
the ones that require only a simple numeric value.

The special (lua-only) case of \type {tex.fontidentifier} returns the \type
{csname} string that matches a font id number (if there is one).

if these are really needed in a macro package.

\subsection{Last item commands}

All \quote {last item} commands are read|-|only and return a number.

The supported commands at this moment are:

\startthreecolumns
\starttyping
tex.lastpenalty
tex.lastkern
tex.lastskip
tex.lastnodetype
tex.inputlineno
tex.pdflastobj
tex.pdflastxform
tex.pdflastximage
tex.pdflastximagepages
tex.pdflastannot
tex.pdflastxpos
tex.pdflastypos
tex.pdfrandomseed
tex.pdflastlink
tex.luatexversion
tex.eTeXminorversion
tex.eTeXversion
tex.currentgrouplevel
tex.currentgrouptype
tex.currentiflevel
tex.currentiftype
tex.currentifbranch
tex.pdflastximagecolordepth
\stoptyping
\stopthreecolumns

\subsection{Attribute, count, dimension, skip and token registers}

\TEX's attributes (\type {\attribute}), counters (\type {\count}), dimensions (\type
{\dimen}), skips (\type {\skip}) and token (\type {\toks}) registers can be accessed
and written to using two times five virtual sub|-|tables of the \type {tex}
table:

\startthreecolumns
\starttyping
tex.attribute
tex.count
tex.dimen
tex.skip
tex.toks
\stoptyping
\stopthreecolumns

It is possible to use the names of relevant \type {\attributedef}, \type {\countdef},
\type {\dimendef}, \type {\skipdef}, or \type {\toksdef} control sequences as indices
to these tables:

\starttyping
tex.count.scratchcounter = 0
enormous = tex.dimen['maxdimen']
\stoptyping

In this case, \LUATEX\ looks up the value for you on the fly. You have to use a
valid \type {\countdef} (or \type {\attributedef}, or \type {\dimendef}, or \type
{\skipdef}, or \type {\toksdef}), anything else will generate an error (the intent
is to eventually also allow \type {<chardef tokens>} and even macros that expand
into a number).

The attribute and count registers accept and return \LUA\ numbers.

The dimension registers accept \LUA\ numbers (in scaled points) or strings (with
an included absolute dimension; \type {em} and \type {ex} and \type {px} are
forbidden). The result is always a number in scaled points.

The token registers accept and return \LUA\ strings. \LUA\ strings are converted
to and from token lists using \type {\the} \type {\toks} style expansion: all
category codes are either space (10) or other (12).

The skip registers accept and return \type {glue_spec} userdata node objects (see
the description of the node interface elsewhere in this manual).

As an alternative to array addressing, there are also accessor functions defined
for all cases, for example, here is the set of possibilities for \type {\skip}
registers:

\startfunctioncall
tex.setskip (<number> n, <node> s)
tex.setskip (<string> s, <node> s)
tex.setskip ('global',<number> n, <node> s)
tex.setskip ('global',<string> s, <node> s)
<node> s = tex.getskip (<number> n)
<node> s = tex.getskip (<string> s)
\stopfunctioncall

We have similar setters for \type {count}, \type {dimen}, \type {muskip}, and
\type {toks}. Counters and dimen are represented by numbers, skips and muskips by
nodes, and toks by strings. For tokens registers we have an alternative where a
catcode table is specified:

\startfunctioncall
tex.scantoks(0,3,"$e=mc^2$")
tex.scantoks("global",0,"$\int\limits^1_2$")
\stopfunctioncall

In the function-based interface, it is possible to define values globally by
using the string \type {global} as the first function argument.

\subsection{Character code registers}

\TEX's character code tables (\type {\lccode}, \type {\uccode}, \type {\sfcode}, \type
{\catcode}, \type {\mathcode}, \type {\delcode}) can be accessed and written to using
six virtual subtables of the \type {tex} table

\startthreecolumns
\starttyping
tex.lccode
tex.uccode
tex.sfcode
tex.catcode
tex.mathcode
tex.delcode
\stoptyping
\stopthreecolumns

The function call interfaces are roughly as above, but there are a few twists.
\type {sfcode}s are the simple ones:

\startfunctioncall
tex.setsfcode (<number> n, <number> s)
tex.setsfcode ('global', <number> n, <number> s)
<number> s = tex.getsfcode (<number> n)
\stopfunctioncall

The function call interface for \type {lccode} and \type {uccode} additionally
allows you to set the associated sibling at the same time:

\startfunctioncall
tex.setlccode (['global'], <number> n, <number> lc)
tex.setlccode (['global'], <number> n, <number> lc, <number> uc)
<number> lc = tex.getlccode (<number> n)
tex.setuccode (['global'], <number> n, <number> uc)
tex.setuccode (['global'], <number> n, <number> uc, <number> lc)
<number> uc = tex.getuccode (<number> n)
\stopfunctioncall

The function call interface for \type {catcode} also allows you to specify a
category table to use on assignment or on query (default in both cases is the
current one):

\startfunctioncall
tex.setcatcode (['global'], <number> n, <number> c)
tex.setcatcode (['global'], <number> cattable, <number> n, <number> c)
<number> lc = tex.getcatcode (<number> n)
<number> lc = tex.getcatcode (<number> cattable, <number> n)
\stopfunctioncall

The interfaces for \type {delcode} and \type {mathcode} use small array tables to
set and retrieve values:

\startfunctioncall
tex.setmathcode (['global'], <number> n, <table> mval )
<table> mval = tex.getmathcode (<number> n)
tex.setdelcode (['global'], <number> n, <table> dval )
<table> dval = tex.getdelcode (<number> n)
\stopfunctioncall

Where the table for \type {mathcode} is an array of 3 numbers, like this:

\starttyping
{<number> mathclass, <number> family, <number> character}
\stoptyping

And the table for \type {delcode} is an array with 4 numbers, like this:

\starttyping
{<number> small_fam, <number> small_char, <number> large_fam, <number> large_char}
\stoptyping

You can also avoid the table:

\startfunctioncall
class, family, char = tex.getmathcodes (<number> n)
smallfam, smallchar, largefam, largechar = tex.getdelcodes (<number> n)
\stopfunctioncall

Normally, the third and fourth values in a delimiter code assignment will be zero
according to \type {\Udelcode} usage, but the returned table can have values there
(if the delimiter code was set using \type {\delcode}, for example). Unset \type
{delcode}'s can be recognized because \type {dval[1]} is $-1$.

\subsection{Box registers}

It is possible to set and query actual boxes, using the node interface as defined
in the \type {node} library:

\starttyping
tex.box
\stoptyping

for array access, or

\starttyping
tex.setbox(<number> n, <node> s)
tex.setbox(<string> cs, <node> s)
tex.setbox('global', <number> n, <node> s)
tex.setbox('global', <string> cs, <node> s)
<node> n = tex.getbox(<number> n)
<node> n = tex.getbox(<string> cs)
\stoptyping

for function|-|based access. In the function-based interface, it is possible to
define values globally by using the string \type {global} as the first function
argument.

Be warned that an assignment like

\starttyping
tex.box[0] = tex.box[2]
\stoptyping

does not copy the node list, it just duplicates a node pointer. If \type {\box2}
will be cleared by \TEX\ commands later on, the contents of \type {\box0} becomes
invalid as well. To prevent this from happening, always use \type
{node.copy_list()} unless you are assigning to a temporary variable:

\starttyping
tex.box[0] = node.copy_list(tex.box[2])
\stoptyping

The following function will register a box for reuse (this is modelled after so
called xforms in \PDF). You can (re)use the box with \type {\useboxresource} or
by creating a rule node with subtype~2.

\starttyping
local index = tex.saveboxresource(n,attributes,resources,immediate)
\stoptyping

The optional second and third arguments are strings, the fourth is a boolean.

You can generate the reference (a rule type) with:

\starttyping
local reused = tex.useboxresource(n,wd,ht,dp)
\stoptyping

The dimensions are optional and the final ones are returned as extra values. The
following is just a bonus (no dimensions returned means that the resource is
unknown):

\starttyping
local w, h, d = tex.getboxresourcedimensions(n)
\stoptyping

You can split a box:

\starttyping
local vlist = tex.splitbox(n,height,mode)
\stoptyping

The remainder is kept in the original box and a packaged vlist is returned. This
operation is comparable to the \type {\vsplit} operation. The mode can be \type
{additional} or \type {exactly} and concerns the split off box.

\subsection{Math parameters}

It is possible to set and query the internal math parameters using:

\startfunctioncall
tex.setmath(<string> n, <string> t, <number> n)
tex.setmath('global', <string> n, <string> t, <number> n)
<number> n = tex.getmath(<string> n, <string> t)
\stopfunctioncall

As before an optional first parameter \type {global} indicates a global
assignment.

The first string is the parameter name minus the leading \quote {Umath}, and the
second string is the style name minus the trailing \quote {style}.

Just to be complete, the values for the math parameter name are:

\starttyping
quad                axis                operatorsize
overbarkern         overbarrule         overbarvgap
underbarkern        underbarrule        underbarvgap
radicalkern         radicalrule         radicalvgap
radicaldegreebefore radicaldegreeafter  radicaldegreeraise
stackvgap           stacknumup          stackdenomdown
fractionrule        fractionnumvgap     fractionnumup
fractiondenomvgap   fractiondenomdown   fractiondelsize
limitabovevgap      limitabovebgap      limitabovekern
limitbelowvgap      limitbelowbgap      limitbelowkern
underdelimitervgap  underdelimiterbgap
overdelimitervgap   overdelimiterbgap
subshiftdrop        supshiftdrop        subshiftdown
subsupshiftdown     subtopmax           supshiftup
supbottommin        supsubbottommax     subsupvgap
spaceafterscript    connectoroverlapmin
ordordspacing       ordopspacing        ordbinspacing     ordrelspacing
ordopenspacing      ordclosespacing     ordpunctspacing   ordinnerspacing
opordspacing        opopspacing         opbinspacing      oprelspacing
opopenspacing       opclosespacing      oppunctspacing    opinnerspacing
binordspacing       binopspacing        binbinspacing     binrelspacing
binopenspacing      binclosespacing     binpunctspacing   bininnerspacing
relordspacing       relopspacing        relbinspacing     relrelspacing
relopenspacing      relclosespacing     relpunctspacing   relinnerspacing
openordspacing      openopspacing       openbinspacing    openrelspacing
openopenspacing     openclosespacing    openpunctspacing  openinnerspacing
closeordspacing     closeopspacing      closebinspacing   closerelspacing
closeopenspacing    closeclosespacing   closepunctspacing closeinnerspacing
punctordspacing     punctopspacing      punctbinspacing   punctrelspacing
punctopenspacing    punctclosespacing   punctpunctspacing punctinnerspacing
innerordspacing     inneropspacing      innerbinspacing   innerrelspacing
inneropenspacing    innerclosespacing   innerpunctspacing innerinnerspacing
\stoptyping

The values for the style parameter name are:

\starttyping
display       crampeddisplay
text          crampedtext
script        crampedscript
scriptscript  crampedscriptscript
\stoptyping

The value is either a number (representing a dimension or number) or a glue spec
node representing a muskip for \type {ordordspacing} and similar spacing
parameters.

\subsection{Special list heads}

The virtual table \type {tex.lists} contains the set of internal registers that
keep track of building page lists.

\starttabulate[|lT|p|]
\NC \bf field           \NC \bf description \NC \NR
\NC page_ins_head       \NC circular list of pending insertions \NC \NR
\NC contrib_head        \NC the recent contributions \NC \NR
\NC page_head           \NC the current page content \NC \NR
%NC temp_head           \NC \NC \NR
\NC hold_head           \NC used for held-over items for next page \NC \NR
\NC adjust_head         \NC head of the current \type {\vadjust} list \NC \NR
\NC pre_adjust_head     \NC head of the current \type {\vadjust pre} list \NC \NR
%NC align_head          \NC \NC \NR
\NC page_discards_head  \NC head of the discarded items of a page break \NC \NR
\NC split_discards_head \NC head of the discarded items in a vsplit \NC \NR
\stoptabulate

\subsection{Semantic nest levels}

The virtual table \type {tex.nest} contains the currently active
semantic nesting state. It has two main parts: a zero-based array of userdata for
the semantic nest itself, and the numerical value \type {tex.nest.ptr}, which
gives the highest available index. Neither the array items in \type {tex.nest[]}
nor \type {tex.nest.ptr} can be assigned to (as this would confuse the
typesetting engine beyond repair), but you can assign to the individual values
inside the array items, e.g.\ \type {tex.nest[tex.nest.ptr].prevdepth}.

\type {tex.nest[tex.nest.ptr]} is the current nest state, \type {tex.nest[0]} the
outermost (main vertical list) level.

The known fields are:

\starttabulate[|lT|l|l|p|]
\NC \ssbf key   \NC \bf type \NC \bf modes \NC \bf explanation \NC \NR
\NC mode        \NC number   \NC all       \NC The current mode. This is a number representing the
                                               main mode at this level:\crlf
                                               \type {0} == no mode (this happens during \type {\write})\crlf
                                               \type {1} == vertical,\crlf
                                               \type {127} = horizontal,\crlf
                                               \type {253} = display math.\crlf
                                               \type {-1} == internal vertical,\crlf
                                               \type {-127} = restricted horizontal,\crlf
                                               \type {-253} = inline math. \NC \NR
\NC modeline    \NC number   \NC all       \NC source input line where this mode was entered in,
                                               negative inside the output routine \NC \NR
\NC head        \NC node     \NC all       \NC the head of the current list \NC \NR
\NC tail        \NC node     \NC all       \NC the tail of the current list \NC \NR
\NC prevgraf    \NC number   \NC vmode     \NC number of lines in the previous paragraph \NC \NR
\NC prevdepth   \NC number   \NC vmode     \NC depth of the previous paragraph (equal to \type {\pdfignoreddimen}
                                               when it is to be ignored) \NC \NR
\NC spacefactor \NC number   \NC hmode     \NC the current space factor \NC \NR
\NC dirs        \NC node     \NC hmode     \NC used for temporary storage by the line break algorithm\NC \NR
\NC noad        \NC node     \NC mmode     \NC used for temporary storage of a pending fraction numerator,
                                               for \type {\over} etc. \NC \NR
\NC delimptr    \NC node     \NC mmode     \NC used for temporary storage of the previous math delimiter,
                                               for \type {\middle} \NC \NR
\NC mathdir     \NC boolean  \NC mmode     \NC true when during math processing the \type {\mathdir} is not
                                               the same as the surrounding \type {\textdir} \NC \NR
\NC mathstyle   \NC number   \NC mmode     \NC the current \type {\mathstyle} \NC \NR
\stoptabulate

\subsection[sec:luaprint]{Print functions}

The \type {tex} table also contains the three print functions that are the
major interface from \LUA\ scripting to \TEX.

The arguments to these three functions are all stored in an in|-|memory virtual
file that is fed to the \TEX\ scanner as the result of the expansion of
\type {\directlua}.

The total amount of returnable text from a \type {\directlua} command is only
limited by available system \RAM. However, each separate printed string has to
fit completely in \TEX's input buffer.

The result of using these functions from inside callbacks is undefined
at the moment.

\subsubsection{\type {tex.print}}

\startfunctioncall
tex.print(<string> s, ...)
tex.print(<number> n, <string> s, ...)
tex.print(<table> t)
tex.print(<number> n, <table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a separate input line. If there is a
table argument instead of a list of strings, this has to be a consecutive array
of strings to print (the first non-string value will stop the printing process).

The optional parameter can be used to print the strings using the catcode regime
defined by \type {\catcodetable}~\type {n}. If \type {n} is $-1$, the currently
active catcode regime is used. If \type {n} is $-2$, the resulting catcodes are
the result of \type {\the} \type {\toks}: all category codes are 12 (other) except for
the space character, that has category code 10 (space). Otherwise, if \type {n}
is not a valid catcode table, then it is ignored, and the currently active
catcode regime is used instead.

The very last string of the very last \type {tex.print()} command in a \type
{\directlua} will not have the \type {\endlinechar} appended, all others do.

\subsubsection{\type {tex.sprint}}

\startfunctioncall
tex.sprint(<string> s, ...)
tex.sprint(<number> n, <string> s, ...)
tex.sprint(<table> t)
tex.sprint(<number> n, <table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a special kind of input line that
makes it suitable for use as a partial line input mechanism:

\startitemize[packed]
\startitem
    \TEX\ does not switch to the \quote {new line} state, so that leading spaces
    are not ignored.
\stopitem
\startitem
    No \type {\endlinechar} is inserted.
\stopitem
\startitem
    Trailing spaces are not removed.

    Note that this does not prevent \TEX\ itself from eating spaces as result of
    interpreting the line. For example, in

\starttyping
before\directlua{tex.sprint("\\relax")tex.sprint(" inbetween")}after
\stoptyping
    the space before \type {in between} will be gobbled as a result of the \quote
    {normal} scanning of \type {\relax}.
\stopitem
\stopitemize

If there is a table argument instead of a list of strings, this has to
be a consecutive array of strings to print (the first non-string value
will stop the printing process).

The optional argument sets the catcode regime, as with \type {tex.print()}.

\subsubsection{\type {tex.tprint}}

\startfunctioncall
tex.tprint({<number> n, <string> s, ...}, {...})
\stopfunctioncall

This function is basically a shortcut for repeated calls to \type
{tex.sprint(<number> n, <string> s, ...)}, once for each of the supplied argument
tables.

\subsubsection{\type {tex.cprint}}

This function takes a number indicating the to be used catcode, plus either a
table of strings or an argument list of strings that will be pushed into the
input stream.

\startfunctioncall
tex.cprint( 1," 1: $&{\\foo}") tex.print("\\par") -- a lot of \bgroup s
tex.cprint( 2," 2: $&{\\foo}") tex.print("\\par") -- matching \egroup s
tex.cprint( 9," 9: $&{\\foo}") tex.print("\\par") -- all get ignored
tex.cprint(10,"10: $&{\\foo}") tex.print("\\par") -- all become spaces
tex.cprint(11,"11: $&{\\foo}") tex.print("\\par") -- letters
tex.cprint(12,"12: $&{\\foo}") tex.print("\\par") -- other characters
tex.cprint(14,"12: $&{\\foo}") tex.print("\\par") -- comment triggers
\stopfunctioncall

\subsubsection{\type {tex.write}}

\startfunctioncall
tex.write(<string> s, ...)
tex.write(<table> t)
\stopfunctioncall

Each string argument is treated by \TEX\ as a special kind of input line that
makes it suitable for use as a quick way to dump information:

\startitemize
\item All catcodes on that line are either \quote{space} (for '~') or
     \quote{character} (for all others).
\item There is no \type {\endlinechar} appended.
\stopitemize

If there is a table argument instead of a list of strings, this has to be a
consecutive array of strings to print (the first non-string value will stop the
printing process).

\subsection{Helper functions}

\subsubsection{\type {tex.round}}

\startfunctioncall
<number> n = tex.round(<number> o)
\stopfunctioncall

Rounds \LUA\ number \type {o}, and returns a number that is in the range of a
valid \TEX\ register value. If the number starts out of range, it generates a
\quote {number to big} error as well.

\subsubsection{\type {tex.scale}}

\startfunctioncall
<number> n = tex.scale(<number> o, <number> delta)
<table> n = tex.scale(table o, <number> delta)
\stopfunctioncall

Multiplies the \LUA\ numbers \type {o} and \type {delta}, and returns a rounded
number that is in the range of a valid \TEX\ register value. In the table
version, it creates a copy of the table with all numeric top||level values scaled
in that manner. If the multiplied number(s) are of range, it generates
\quote{number to big} error(s) as well.

Note: the precision of the output of this function will depend on your computer's
architecture and operating system, so use with care! An interface to \LUATEX's
internal, 100\% portable scale function will be added at a later date.

\subsubsection{\type {tex.sp}}

\startfunctioncall
<number> n = tex.sp(<number> o)
<number> n = tex.sp(<string> s)
\stopfunctioncall

Converts the number \type {o} or a string \type {s} that represents an explicit
dimension into an integer number of scaled points.

For parsing the string, the same scanning and conversion rules are used that
\LUATEX\ would use if it was scanning a dimension specifier in its \TEX|-|like
input language (this includes generating errors for bad values), expect for the
following:

\startitemize[n]
\startitem
    only explicit values are allowed, control sequences are not handled
\stopitem
\startitem
    infinite dimension units (\type {fil...}) are forbidden
\stopitem
\startitem
    \type {mu} units do not generate an error (but may not be useful either)
\stopitem
\stopitemize

\subsubsection{\type {tex.definefont}}

\startfunctioncall
tex.definefont(<string> csname, <number> fontid)
tex.definefont(<boolean> global, <string> csname, <number> fontid)
\stopfunctioncall

Associates \type {csname} with the internal font number \type {fontid}. The
definition is global if (and only if) \type {global} is specified and true (the
setting of \type {globaldefs} is not taken into account).

\subsubsection{\type {tex.getlinenumber} and \type {tex.setlinenumber}}

You can mess with the current line number:

\startfunctioncall
local n = tex.getlinenumber()
tex.setlinenumber(n+10)
\stopfunctioncall

which can be shortcut to:

\startfunctioncall
tex.setlinenumber(10,true)
\stopfunctioncall

This might be handy when you have a callback that read numbers from a file and
combines them in one line (in which case an error message probably has to refer
to the original line). Interference with \TEX's internal handling of numbers is
of course possible.

\subsubsection{\type {tex.error}}

\startfunctioncall
tex.error(<string> s)
tex.error(<string> s, <table> help)
\stopfunctioncall

This creates an error somewhat like the combination of \type {\errhelp} and \type
{\errmessage} would. During this error, deletions are disabled.

The array part of the \type {help} table has to contain strings, one for each
line of error help.

\subsubsection{\type {tex.hashtokens}}

\startfunctioncall
for i,v in pairs (tex.hashtokens()) do ... end
\stopfunctioncall

Returns a name and token table pair (see~\in {section} [luatokens] about token
tables) iterator for every non-zero entry in the hash table. This can be useful
for debugging, but note that this also reports control sequences that may be
unreachable at this moment due to local redefinitions: it is strictly a dump of
the hash table.

\subsection[luaprimitives]{Functions for dealing with primitives }

\subsubsection{\type {tex.enableprimitives}}

\startfunctioncall
tex.enableprimitives(<string> prefix, <table> primitive names)
\stopfunctioncall

This function accepts a prefix string and an array of primitive names.

For each combination of \quote {prefix} and \quote {name}, the \type
{tex.enableprimitives} first verifies that \quote {name} is an actual primitive
(it must be returned by one of the \type {tex.extraprimitives()} calls explained
below, or part of \TEX82, or \type {\directlua}). If it is not, \type
{tex.enableprimitives} does nothing and skips to the next pair.

But if it is, then it will construct a csname variable by concatenating the
\quote {prefix} and \quote {name}, unless the \quote {prefix} is already the
actual prefix of \quote {name}. In the latter case, it will discard the \quote
{prefix}, and just use \quote {name}.

Then it will check for the existence of the constructed csname. If the csname is
currently undefined (note: that is not the same as \type {\relax}), it will
globally define the csname to have the meaning: run code belonging to the
primitive \quote {name}. If for some reason the csname is already defined, it
does nothing and tries the next pair.

An example:

\starttyping
  tex.enableprimitives('LuaTeX', {'formatname'})
\stoptyping

will define \type {\LuaTeXformatname} with the same intrinsic meaning as the
documented primitive \type {\formatname}, provided that the control sequences \type
{\LuaTeXformatname} is currently undefined.

When \LUATEX\ is run with \type {--ini} only the \TEX82 primitives and \type
{\directlua} are available, so no extra primitives {\bf at all}.

If you want to have all the new functionality available using their default
names, as it is now, you will have to add

\starttyping
  \ifx\directlua\undefined \else
     \directlua {tex.enableprimitives('',tex.extraprimitives ())}
  \fi
\stoptyping

near the beginning of your format generation file. Or you can choose different
prefixes for different subsets, as you see fit.

Calling some form of \type {tex.enableprimitives()} is highly important though,
because if you do not, you will end up with a \TEX82-lookalike that can run \LUA\
code but not do much else. The defined csnames are (of course) saved in the
format and will be available at runtime.

\subsubsection{\type {tex.extraprimitives}}

\startfunctioncall
<table> t = tex.extraprimitives(<string> s, ...)
\stopfunctioncall

This function returns a list of the primitives that originate from the engine(s)
given by the requested string value(s). The possible values and their (current)
return values are:

\startluacode
function document.showprimitives(tag)
    for k, v in table.sortedpairs(tex.extraprimitives(tag)) do
        if v == ' ' then
            v = '\\normalcontrolspace'
        end
        context.type(v)
        context.space()
    end
end
\stopluacode

\starttabulate[|l|pl|]
\NC \bf name\NC \bf values \NC \NR
\NC tex     \NC \ctxlua{document.showprimitives('tex')    } \NC \NR
\NC core    \NC \ctxlua{document.showprimitives('core')   } \NC \NR
\NC etex    \NC \ctxlua{document.showprimitives('etex')   } \NC \NR
\NC luatex  \NC \ctxlua{document.showprimitives('luatex') } \NC \NR
\stoptabulate

Note that \type {'luatex'} does not contain \type {directlua}, as that is
considered to be a core primitive, along with all the \TEX82 primitives, so it is
part of the list that is returned from \type {'core'}.

% \type {'umath'} is a subset of \type {'luatex'} that covers the Unicode math
% primitives as it might be desired to handle the prefixing of that subset
% differently.

Running \type {tex.extraprimitives()} will give you the complete list of
primitives \type {-ini} startup. It is exactly equivalent to \type
{tex.extraprimitives('etex' and 'luatex')}.

\subsubsection{\type {tex.primitives}}

\startfunctioncall
<table> t = tex.primitives()
\stopfunctioncall

This function returns a hash table listing all primitives that \LUATEX\ knows
about. The keys in the hash are primitives names, the values are tables
representing tokens (see~\in{section }[luatokens]). The third value is always
zero.

{\em In the beginning we had \type {omega} and \type {pdftex} subsets but in the
meantime relevant primitives ave been promoted (either or not adapted) to the
\type {luatex} set when found useful, or removed when considered to be of no use.
Originally we had two sets of math definition primitives but the \OMEGA\ ones
have been removed, so we no longer have a subset for math either.}

\subsection{Core functionality interfaces}

\subsubsection{\type {tex.badness}}

\startfunctioncall
<number> b = tex.badness(<number> t, <number> s)
\stopfunctioncall

This helper function is useful during linebreak calculations. \type {t} and \type
{s} are scaled values; the function returns the badness for when total \type {t}
is supposed to be made from amounts that sum to \type {s}. The returned number is
a reasonable approximation of $100(t/s)^3$;

\subsubsection{\type {tex.resetparagraph}}

This function resets the parameters that \TEX\ normally resets when a new paragraph
is seen.

\subsubsection{\type {tex.linebreak}}

\startfunctioncall
local <node> nodelist, <table> info =
       tex.linebreak(<node> listhead, <table> parameters)
\stopfunctioncall

The understood parameters are as follows:

\starttabulate[|l|l|p|]
\NC \bf name                 \NC \bf type        \NC \bf description \NC \NR
\NC pardir                   \NC string          \NC \NC \NR
\NC pretolerance             \NC number          \NC \NC \NR
\NC tracingparagraphs        \NC number          \NC \NC \NR
\NC tolerance                \NC number          \NC \NC \NR
\NC looseness                \NC number          \NC \NC \NR
\NC hyphenpenalty            \NC number          \NC \NC \NR
\NC exhyphenpenalty          \NC number          \NC \NC \NR
\NC pdfadjustspacing         \NC number          \NC \NC \NR
\NC adjdemerits              \NC number          \NC \NC \NR
\NC pdfprotrudechars         \NC number          \NC \NC \NR
\NC linepenalty              \NC number          \NC \NC \NR
\NC lastlinefit              \NC number          \NC \NC \NR
\NC doublehyphendemerits     \NC number          \NC \NC \NR
\NC finalhyphendemerits      \NC number          \NC \NC \NR
\NC hangafter                \NC number          \NC \NC \NR
\NC interlinepenalty         \NC number or table \NC if a table, then it is an array like \type {\interlinepenalties} \NC \NR
\NC clubpenalty              \NC number or table \NC if a table, then it is an array like \type {\clubpenalties} \NC \NR
\NC widowpenalty             \NC number or table \NC if a table, then it is an array like \type {\widowpenalties} \NC \NR
\NC brokenpenalty            \NC number          \NC \NC \NR
\NC emergencystretch         \NC number          \NC in scaled points \NC \NR
\NC hangindent               \NC number          \NC in scaled points \NC \NR
\NC hsize                    \NC number          \NC in scaled points \NC \NR
\NC leftskip                 \NC glue_spec node  \NC \NC \NR
\NC rightskip                \NC glue_spec node  \NC \NC \NR
\NC pdfignoreddimen          \NC number          \NC in scaled points \NC \NR
\NC parshape                 \NC table           \NC \NC \NR
\stoptabulate

Note that there is no interface for \type {\displaywidowpenalties}, you have to
pass the right choice for \type {widowpenalties} yourself.

The meaning of the various keys should be fairly obvious from the table (the
names match the \TEX\ and \PDFTEX\ primitives) except for the last 5 entries. The
four \type {pdf...line...} keys are ignored if their value equals \type
{pdfignoreddimen}.

It is your own job to make sure that \type {listhead} is a proper paragraph list:
this function does not add any nodes to it. To be exact, if you want to replace
the core line breaking, you may have to do the following (when you are not
actually working in the \type {pre_linebreak_filter} or \type {linebreak_filter}
callbacks, or when the original list starting at listhead was generated in
horizontal mode):

\startitemize
\startitem
    add an \quote {indent box} and perhaps a \type {local_par} node at the start
    (only if you need them)
\stopitem
\startitem
    replace any found final glue by an infinite penalty (or add such a penalty,
    if the last node is not a glue)
\stopitem
\startitem
    add a glue node for the \type {\parfillskip} after that penalty node
\stopitem
\startitem
    make sure all the \type {prev} pointers are OK
\stopitem
\stopitemize

The result is a node list, it still needs to be vpacked if you want to assign it
to a \type {\vbox}.

The returned \type {info} table contains four values that are all numbers:

\starttabulate[|l|p|]
\NC prevdepth \NC depth of the last line in the broken paragraph \NC \NR
\NC prevgraf  \NC number of lines in the broken paragraph \NC \NR
\NC looseness \NC the actual looseness value in the broken paragraph \NC \NR
\NC demerits  \NC the total demerits of the chosen solution  \NC \NR
\stoptabulate

Note there are a few things you cannot interface using this function: You cannot
influence font expansion other than via \type {pdfadjustspacing}, because the
settings for that take place elsewhere. The same is true for hbadness and hfuzz
etc. All these are in the \type {hpack()} routine, and that fetches its own
variables via globals.

\subsubsection{\type {tex.shipout}}

\startfunctioncall
tex.shipout(<number> n)
\stopfunctioncall

Ships out box number \type {n} to the output file, and clears the box register.

\section[texconfig]{The \type {texconfig} table}

This is a table that is created empty. A startup \LUA\ script could
fill this table with a number of settings that are read out by
the executable after loading and executing the startup file.

\starttabulate[|lT|l|l|p|]
\NC \ssbf key             \NC \bf type \NC \bf default \NC \bf explanation \NC \NR
\NC kpse_init             \NC boolean  \NC true
\NC
    \type {false} totally disables \KPATHSEA\ initialisation, and enables
    interpretation of the following numeric key--value pairs. (only ever unset
    this if you implement {\it all\/} file find callbacks!)
\NC \NR
\NC
    shell_escape          \NC string   \NC \type {'f'} \NC
    Use \type {'y'} or \type {'t'} or \type {'1'} to enable \type {\write18}
    unconditionally, \type {'p'} to enable the commands that are listed in \type
    {shell_escape_commands}
\NC \NR
\NC
    shell_escape_commands \NC string \NC \NC Comma-separated list of command
    names that may be executed by \type {\write18} even if \type {shell_escape}
    is set to \type {'p'}. Do {\it not\/} use spaces around commas, separate any
    required command arguments by using a space, and use the \ASCII\ double quote
    (\type {"}) for any needed argument or path quoting
\NC \NR

\NC string_vacancies      \NC number   \NC  75000  \NC cf.\ web2c docs \NC \NR
\NC pool_free             \NC number   \NC   5000  \NC cf.\ web2c docs \NC \NR
\NC max_strings           \NC number   \NC  15000  \NC cf.\ web2c docs \NC \NR
\NC strings_free          \NC number   \NC    100  \NC cf.\ web2c docs \NC \NR
\NC nest_size             \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
\NC max_in_open           \NC number   \NC     15  \NC cf.\ web2c docs \NC \NR
\NC param_size            \NC number   \NC     60  \NC cf.\ web2c docs \NC \NR
\NC save_size             \NC number   \NC   4000  \NC cf.\ web2c docs \NC \NR
\NC stack_size            \NC number   \NC    300  \NC cf.\ web2c docs \NC \NR
\NC dvi_buf_size          \NC number   \NC  16384  \NC cf.\ web2c docs \NC \NR
\NC error_line            \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
\NC half_error_line       \NC number   \NC     50  \NC cf.\ web2c docs \NC \NR
\NC max_print_line        \NC number   \NC     79  \NC cf.\ web2c docs \NC \NR
\NC hash_extra            \NC number   \NC      0  \NC cf.\ web2c docs \NC \NR
\NC pk_dpi                \NC number   \NC     72  \NC cf.\ web2c docs \NC \NR
\NC trace_file_names      \NC boolean  \NC true
\NC
    \type {false} disables \TEX's normal file open|-|close feedback (the
    assumption is that callbacks will take care of that)
\NC \NR
\NC file_line_error       \NC boolean  \NC false
\NC
    do \type {file:line} style error messages
\NC \NR
\NC halt_on_error         \NC boolean  \NC false
\NC
    abort run on the first encountered error
\NC \NR
\NC formatname            \NC string   \NC
\NC
    if no format name was given on the command line, this key will be tested first
    instead of simply quitting
\NC \NR
\NC jobname               \NC string   \NC
\NC
    if no input file name was given on the command line, this key will be tested
    first instead of simply giving up
\NC \NR
\stoptabulate

Note: the numeric values that match web2c parameters are only used if \type
{kpse_init} is explicitly set to \type {false}. In all other cases, the normal
values from \type {texmf.cnf} are used.

\section{The \type {texio} library}

This library takes care of the low|-|level I/O interface.

\subsection{Printing functions}

\subsubsection{\type {texio.write}}

\startfunctioncall
texio.write(<string> target, <string> s, ...)
texio.write(<string> s, ...)
\stopfunctioncall

Without the \type {target} argument, writes all given strings to the same
location(s) \TEX\ writes messages to at this moment. If \type {\batchmode} is in
effect, it writes only to the log, otherwise it writes to the log and the
terminal. The optional \type {target} can be one of three possibilities: \type
{term}, \type {log} or \type {term and log}.

Note: If several strings are given, and if the first of these strings is or might
be one of the targets above, the \type {target} must be specified explicitly to
prevent \LUA\ from interpreting the first string as the target.

\subsubsection{\type {texio.write_nl}}

\startfunctioncall
texio.write_nl(<string> target, <string> s, ...)
texio.write_nl(<string> s, ...)
\stopfunctioncall

This function behaves like \type {texio.write}, but make sure that the given
strings will appear at the beginning of a new line. You can pass a single empty
string if you only want to move to the next line.

\subsubsection{\type {texio.setescape}}

You can disable \type {^^} escaping of control characters by passing a value of
zero.

\subsection{The \type {token} libray}

The current \type {token} library will be replaced by a new one that is more
flexible and powerful. The transition takes place in steps. In version 0.80 we
have \type {token} and in version 0.85 the old lib will be replaced
completely. So if you use this new mechanism in production code you need to be
aware of incompatible updates between 0.80 and 0.90. Because the related in- and
output code will also be cleaned up and rewritten you should be aware of
incompatible logging and error reporting too.

The old library presents tokens as triplets or numbers, the new library presents
a userdata object. The old library used a callback to intercept tokens in the
input but the new library provides a basic scanner infrastructure that can be
used to write macros that accept a wide range of arguments. This interface is on
purpose kept general and as performance is quite ok one can build additional
parsers without too much overhead. It's up to macro package writers to see how
they can benefit from this as the main principle behind \LUATEX\ is to provide a
minimal set of tools and no solutions.

The current functions in the \type {token} namespace are given in the next
table:

\starttabulate[|lT|lT|p|]
\NC \bf function \NC \bf argument       \NC \bf result \NC \NR
\HL
\NC is_token     \NC token              \NC checks if the given argument is a token userdatum \NC \NR
\NC get_next     \NC                    \NC returns the next token in the input \NC \NR
\NC scan_keyword \NC string             \NC returns true if the given keyword is gobbled \NC \NR
\NC scan_int     \NC                    \NC returns a number \NC \NR
\NC scan_dimen   \NC infinity, mu-units \NC returns a number representing a dimension and or two numbers being the filler and order \NC \NR
\NC scan_glue    \NC mu-units           \NC returns a glue spec node \NC \NR
\NC scan_toks    \NC definer, expand    \NC returns a table of tokens token list (this can become a linked list in later releases) \NC \NR
\NC scan_code    \NC bitset             \NC returns a character if its category is in the given bitset (representing catcodes) \NC \NR
\NC scan_string  \NC                    \NC returns a string given between \type {{}}, as \type {\macro} or as sequence of characters with catcode 11 or 12 \NC \NR
\NC scan_word    \NC                    \NC returns a sequence of characters with catcode 11 or 12 as string \NC \NR
\NC scan_csname  \NC                    \NC returns \type {foo} after scanning \type {\foo} \NC \NR
\NC set_macro    \NC see below          \NC assign a macro \NC \NR
\NC create       \NC                    \NC returns a userdata token object of the given control sequence name (or character); this interface can change  \NC \NR
\stoptabulate

The scanners can be considered stable apart from the one scanning for a token.
This is because futures releases can return a linked list instead of a table (as
with nodes). The \type {scan_code} function takes an optional number, the \type
{keyword} function a normal \LUA\ string. The \type {infinity} boolean signals
that we also permit \type {fill} as dimension and the \type {mu-units} flags the
scanner that we expect math units. When scanning tokens we can indicate that we
are defining a macro, in which case the result will also provide information
about what arguments are expected and in the result this is separated from the
meaning by a separator token. The \type {expand} flag determines if the list will
be expanded.

The string scanner scans for something between curly braces and expands on the
way, or when it sees a control sequence it will return its meaning. Otherwise it
will scan characters with catcode \type {letter} or \type {other}. So, given the
following definition:

\startbuffer
\def\bar{bar}
\def\foo{foo-\bar}
\stopbuffer

\typebuffer \getbuffer

we get:

\starttabulate[|l|Tl|l|]
\NC \type {\directlua{token.scan_string()}{foo}} \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} {foo} \NC full expansion \NR
\NC \type {\directlua{token.scan_string()}foo}   \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")} foo   \NC letters and others \NR
\NC \type {\directlua{token.scan_string()}\foo}  \NC \directlua{context("{\\red\\type {"..token.scan_string().."}}")}\foo   \NC meaning \NR
\stoptabulate

The \type {\foo} case only gives the meaning, but one can pass an already
expanded definition (\type {\edef}'d). In the case of the braced variant one can of
course use the \type {\detokenize} and \type {\unexpanded} primitives as there we
do expand.

The \type {scan_word} scanner can be used to implement for instance a number scanner:

\starttyping
function token.scan_number(base)
    return tonumber(token.scan_word(),base)
end
\stoptyping

This scanner accepts any valid \LUA\ number so it is a way to pick up floats
in the input.

The creator function can be used as follows:

\starttyping
local t = token.create("relax")
\stoptyping

This gives back a token object that has the properties of the \type {\relax}
primitive. The possible properties of tokens are:

\starttabulate[|lT|p|]
\NC command    \NC a number representing the internal command number \NC \NR
\NC cmdname    \NC the type of the command (for instance the catcode in case of a
                   character or the classifier that determines the internal
                   treatment \NC \NR
\NC csname     \NC the associated control sequence (if applicable) \NC \NR
\NC id         \NC the unique id of the token \NC \NR
%NC tok        \NC \NC \NR % might change
\NC active     \NC a boolean indicating the active state of the token \NC \NR
\NC expandable \NC a boolean indicating if the token (macro) is expandable \NC \NR
\NC protected  \NC a boolean indicating if the token (macro) is protected \NC \NR
\stoptabulate

The numbers that represent a catcode are the same as in \TEX\ itself, so using
this information assumes that you know a bit about \TEX's internals. The other
numbers and names are used consistently but are not frozen. So, when you use them
for comparing you can best query a known primitive or character first to see the
values.

More interesting are the scanners. You can use the \LUA\ interface as follows:

\starttyping
\directlua {
    function mymacro(n)
        ...
    end
}

\def\mymacro#1{%
    \directlua {
        mymacro(\number\dimexpr#1)
    }%
}

\mymacro{12pt}
\mymacro{\dimen0}
\stoptyping

You can also do this:

\starttyping
\directlua {
    function mymacro()
        local d = token.scan_dimen()
        ...
    end
}

\def\mymacro{%
    \directlua {
        mymacro()
    }%
}

\mymacro 12pt
\mymacro \dimen0
\stoptyping

It is quite clear from looking at the code what the first method needs as
argument(s). For the second method you need to look at the \LUA\ code to see what
gets picked up. Instead of passing from \TEX\ to \LUA\ we let \LUA\ fetch from
the input stream.

In the first case the input is tokenized and then turned into a string when it's
passed to \LUA\ where it gets interpreted. In the second case only a function
call gets interpreted but then the input is picked up by explicitly calling the
scanner functions. These return proper \LUA\ variables so no further conversion
has to be done. This is more efficient but in practice (given what \TEX\ has to
do) this effect should not be overestimated. For numbers and dimensions it saves a
bit but for passing strings conversion to and from tokens has to be done anyway
(although we can probably speed up the process in later versions if needed).

When the interface is stable and has replaced the old one completely we will add
some more information here. By that time the internals have been cleaned up a bit
more so we know then what will stay and go. A positive side effect of this
transition is that we can simplify the input part because we no longer need to
intercept using callbacks.

The \type {set_macro} function can get upto 4 arguments:

\starttyping
setmacro("csname","content")
setmacro("csname","content","global")
setmacro("csname")
\stoptyping

You can pass a catcodetable identifier as first argument:

\starttyping
setmacro(catcodetable,"csname","content")
setmacro(catcodetable,"csname","content","global")
setmacro(catcodetable,"csname")
\stoptyping

The results are like:

\starttyping
 \def\csname{content}
\gdef\csname{content}
 \def\csname{}
\stoptyping

There is a (for now) experimental putter:

\starttyping
local t1 = token.get_next()
local t2 = token.get_next()
local t3 = token.get_next()
local t4 = token.get_next()
-- watch out, we flush in sequence
token.put_next { t1, t2 }
-- but this one gets pushed in front
token.put_next ( t3, t4 )
\stoptyping

When we scan \type {wxyz!} we get \type {yzwx!} back. The argument is either a table
with tokens or a list of tokens.

\stopchapter

\stopcomponent