BR3288901: Relax concat rules in preprocessor code
We simply allow the following terminals to be concat'ed
if they are written without space or any other separator
inbetween.
a := id | preproc-id | number | float | other
b := id | preproc-id | number | float | other
if match(a,b):
s := concat(a,b)
re-tokenize(s)
Basically it means it's up to code author to write
preproc code a way the sane production appears.
Some notes.
1) We don't concat strings.
2) The 'weirdpaste' test fails now because with relaxed
rules it works as needed and was borken before.
The lacmus snippet is
%define N 1e%++%+ 5
dd N, 1e+5
Previously the output was
dd 1e+%+ 5, 1e+5
which is wrong since we have explicit concat here
with %+ operator. The new code production is correct
and looks like
dd 1e+5, 1e+5
as expected.
Signed-off-by: Cyrill Gorcunov <gorcunov@gmail.com>