1 \section{Splicing and quoting
}
2 As the previous section shows, AST are not extremely readable, and as
3 promized, Metalua offer a way to avoid dealing with them
4 directly. Well, rarely dealing with them anyway.
6 In this section, we will deal a lot with
\verb|+
{...
}| and
7 \verb|-
{...
}|; the only (but real) difficulty is not to get lost
8 between meta-levels, i.e. not getting confused between a piece of
9 code, the AST representing that piece of code, some code returning an
10 AST that shall be executed during compilation, etc.
12 \subsection{Quasi-quoting
}
13 Quoting an expression is extremely easy: just put it between
14 quasi-quotes. For instance, to get the AST representing
\verb|
2+
2|,
15 just type
\verb|+
{expr:
2+
2}|. Actually, since most of quotes are
16 actually expression quotes, you are even allowed to skip the ``expr:''
17 part:
\verb|+
{2+
2}| works just as well.
19 If you want to quote a statement, just substitute ``expr:'' with
20 ``stat:'':
{\tt+\
{stat: if x>
3 then foo(bar) end\
}}.
22 Finally, you might wish to quote a block of code. As you can guess,
25 \verb|+
{block: y =
7; x = y+
1; if x>
3 then foo(bar) end
}|.
27 A block is just a list of statements. That means that
28 \verb|+
{block: x=
1}| is the same as
\verb|
{ +
{stat: x=
1} }| (a
29 single-element list of statements).
31 However, quoting alone is not really useful: if it's just about
32 pasting pieces of code verbatim, there is little point in
33 meta-programming. We want to be able to poke ``holes'' in quasi-quotes
34 (hence the ``quasi''), and fill them with bits of AST comming from
35 outside. Such holes are marked with a
\verb|-
{...
}| construct, called
36 a splice, inside the quote. For instance, the following piece of
37 Metalua will put the AST of
\verb|
2+
2| in variable X, then insert it
38 in the AST an assignement in Y:
42 Y = +
{ four = -
{ X
} }
45 After this, Y will contain the AST representing
\verb|four =
2+
2|.
46 Because of this, a splice inside a quasi-quote is often called an
47 anti-quote (as we shall see, splices also make sense, although a
48 different one, outside quotes).
50 Of course, quotes and antiquotes can be mixed with explicit AST. The
51 following lines all put the same value in Y, although often in a
54 \begin{Verbatim
}[fontsize=
\scriptsize]
56 Y = +
{stat: four =
2+
2 }
57 -- Without any quote, directly as an AST:
58 Y = `Let
{ { `Id "four"
},
{ `Op
{ `Add, `Number
2, `Number
2 } } }
59 -- Various mixes of direct AST and quotes:
60 X = +
{ 2+
2 }; Y = +
{stat: four = -
{ X
} }
61 X = `Op
{ `Add, +
{2}, +
{2} }; Y = +
{stat: four = -
{ X
} }
62 X = `Op
{ `Add, `Number
2, `Number
2 }; Y = +
{stat: four = -
{ X
} }
63 Y = +
{stat: four = -
{ `Op
{ `Add, `Number
2, `Number
2 } } }
64 Y = +
{stat: four = -
{ +
{ 2+
2 } } }
65 Y = `Let
{ { `Id "four"
},
{ +
{ 2+
2 } } }
66 -- Nested quotes and splices cancel each other:
67 Y = +
{stat: four = -
{ +
{ -
{ +
{ -
{ +
{ -
{ +
{ 2+
2 } } } } } } } } }
70 The content of an anti-quote is expected to be an expression by
71 default. However, it is legal to put a statement or a block of
72 statements in it, provided that it returns an AST through a
73 \verb+return+ statement. To do this, just add a ``block:''
74 (or ``stat:'') markup at the beginning of the antiquote. The
75 following line is (also) equivalent to the previous ones:
78 Y = +
{stat: four = -
{ block:
80 return `Op
{ 'add', two, two
} } }
83 Notice that in a block, where a statement is expected, a sub-block is
84 also be accepted, and is simply combined with the upper-level
85 one. Unlike
{\tt`Do\
{ \
}} statements, it doesn't create its own scope.
86 For instance, you can write
\verb|-
{block: f(); g()
}| instead of
87 \verb|-
{stat:f()
}; -
{stat:g()
}|.
90 Splicing is used in two, rather different contexts. First, as seen
91 above, it's used to poke holes into quotations. But it is also used to
92 execute code at compile time.
94 As can be expected from their syntaxes,
\verb|-
{...
}| undoes what
95 \verb|+
{...
}| does: quotes change a piece of code into the AST
96 representing it, and splices cancel the quotation of a piece of code,
97 including it directly in the AST (that piece of code therefore has to
98 either be an AST, or evaluate to an AST. If not, the result of the
99 surrounding quote won't be an AST).
101 But what happens when a splice is put outside of any quote? There is
102 no explicit quotation to cancel, but actually, there is an hidden AST
103 generation. The process of compiling a Metalua source file consists in
106 \begin{Verbatim
}[fontsize=
\scriptsize]
108 +-----------+ / \ +---+ / \ +--------+
109 |SOURCE FILE|-->< Parser >-->|AST|-->< Compiler >-->|BYTECODE|
110 +-----------+
\______/ +---+
\________/ +--------+
114 So in reality, the source file is translated into an AST; when a
115 splice is found, instead of just turning that AST into bytecode, we
116 will execute the corresponding program, and put the AST it must return
117 in the source code. This computed AST is the one which will be turned
118 into bytecode in the resulting program. Of course, that means locally
119 compiling the piece of code in the splice, in order to execute it:
121 \begin{Verbatim
}[fontsize=
\scriptsize]
123 ______ ________ +->|BYTECODE|
124 +-----------+ / \ +---+ / \ | +--------+
125 |SOURCE FILE|-->< Parser >-->|AST|-->< Compiler >-+
126 +-----------+
\______/ +-^-+
\________/ | +--------+
127 /|\ ________ +->|BYTECODE|
129 +-----< Eval ><-------+
133 As an example, consider the following source code, its compilation and
136 \def\braces#1{\
{#1\
}}
139 {\bf{}fabien@macfabien\$
} cat sample.mlua
140 -
\braces{block: print "META HELLO"
141 return +
\braces{ print "GENERATED HELLO"
} }
144 {\bf{}fabien@macfabien\$
} metalua -v sample.mlua -o sample.luac
145 [ Param "sample.mlua" considered as a source file
]
146 [ Compiling `File "sample.mlua"
]
148 [ Saving to file "sample.luac"
]
150 {\bf{}fabien@macfabien\$
} lua sample.luac
153 {\bf{}fabien@macfabien\$
} _
157 Thanks to the print statement in the splice, we see that the code
158 it contains is actually executed during evaluation. More in details,
159 what happens is that:
161 \item The code inside the splice is parsed and compiled separately;
162 \item it is executed: the call to
\verb|print "META HELLO"| is
163 performed, and the AST representing \\
\verb|print "GENERATED HELLO"| is
164 generated and returned;
165 \item in the AST generated from the source code, the splice is
166 replaced by the AST representing \\
\verb|print "GENERATED HELLO"|.
167 Therefore, what is passed to the compiler is the AST representing\\
168 \verb|print "GENERATED HELLO";|
\verb|print "NORMAL HELLO"|.
171 Take time to read, re-read, play and re-play with the manipulation
172 described above: understanding the transitions between meta-levels is
173 the essence of meta-programming, and you must be comfortable with such
174 transitions in order to make the best use of Metalua.
176 Notice that it is admissible, for a splice outside a quote, not to
177 return anything. This allows to execute code at compile time without
178 adding anything in the AST, typically to load syntax extensions. For
179 instance, this source will just print "META HELLO" at compile time,
180 and "NORMAL HELLO" at runtime:
181 \verb|-
{print "META HELLO"
}; print "NORMAL HELLO"|
183 \subsection{A couple of simple concrete examples
}
185 \paragraph{ternary choice operator
}
186 Let's build something more useful. As an example, we will build here a
187 ternary choice operator, equivalent to the
\verb|_ ? _ : _| from
188 C. Here, we will not deal yet with syntax sugar: our operator will
189 have to be put inside splices. Extending the syntax will be dealt with
190 in the next section, and then, we will coat it with a sweet syntax.
192 Here is the problem: in Lua, choices are made by using
193 \verb|if _ then _ else _ end| statements. It is a statement, not an
194 expression, which means that we can't use it in, for instance:
197 local hi = if lang=="fr" then "Bonjour"
198 else "hello" end -- illegal!
201 This won't compile. So, how to turn the ``if'' statement into an
202 expression? The simplest solution is to put it inside a function
203 definition. Then, to actually execute it, we need to evaluate that
204 function. Which means that our pseudo-code
205 \verb|local hi = (lang == "fr" ? "Bonjour" : "Hello")| will
206 actually be compiled into:
211 if lang == "fr" then return "Bonjour"
212 else return "Hello" end end) ()
215 We are going to define a function building the AST above, filling
216 holes with parameters. Then we are going to use it in the actual code,
221 {\bf{}fabien@macfabien\$
} cat sample.lua
223 -- Declaring the
[ternary
] metafunction. As a
224 -- metafunction, it only exists within -\
{...\
},
225 -- i.e. not in the program itself.
226 function ternary (cond, b1, b2)
227 return +\
{ (function()
237 hi = -
\braces{ ternary (+
\braces{lang=="fr"
}, +
\braces{"Bonjour"
}, +
\braces{"Hello"
})
}
241 hi = -
\braces{ ternary (+
\braces{lang=="fr"
}, +
\braces{"Bonjour"
}, +
\braces{"Hello"
})
}
244 {\bf{}fabien@macfabien\$
} mlc sample.lua
245 Compiling sample.lua...
247 {\bf{}fabien@macfabien\$
} lua sample.luac
250 {\bf{}fabien@macfabien\$
} _
254 \paragraph{Incrementation operator
}
255 Now, we will write another simple example, which doesn't use
256 quasi-quotes, just to show that we can. Another operator that C
257 developpers might be missing with Lua is the
\verb|++| operator. As
258 with the ternary operator, we won't show yet how to put the syntax
259 sugar coating around it, just how to build the backend functionnality.
261 Here, the transformation is really trivial: we want to encode
262 \verb|x++| as
\verb|x=x+
1|. We will only deal with
\verb|++| as
263 statement, not as an expression. However,
\verb|++| as an expression is not
264 much more complicated to do. Hint: use the turn-statement-into-expr
265 trick shown in the previous example. The AST corresponding to
267 \verb|`Let
{ { `Id x
},
{ `Op
{ `Add, `Id x, `Number
1 } } }|. From
268 here, the code is straightforward:
272 {\bf{}fabien@macfabien\$
} cat sample.lua
274 function plusplus (var)
275 assert (var.tag == "Id")
276 return `Let
\braces{ \braces{ var
},
\braces{ `Op
\braces{ `Add, var, `Number
1 } } }
280 print ("x = " .. tostring (x))
281 -
\braces{ plusplus ( +
\braces{x
} )
};
282 print ("Incremented x: x = " .. tostring (x))
284 {\bf{}fabien@macfabien\$
} mlc sample.lua
285 Compiling sample.lua...
287 {\bf{}fabien@macfabien\$
} lua sample.luac
290 {\bf{}fabien@macfabien\$
} _
294 Now, we just miss a decent syntax around this, and we are set! This is
295 the subject of the next sections:
\verb|gg| is the generic grammar
296 generator, which allows to build and grow parsers. It's used to
297 implement
\verb|mlp|, the Metalua parser, which turns Metalua sources
300 Therefore, the informations useful to extend Metalua syntax are:
303 \item What are the relevant entry points in mlp, the methods which
304 allow syntax extension.
305 \item How to use these methods: this consists into knowing the classes
306 defined into gg, which offer dynamic extension possibilities.