[STD] work around viv bug on "%02x"

[pugs.git] / misc / pm.txt

This file contains miscellaneous questions about design+spec
that Pm has come up with that are awaiting answers from TimToady
and/or others.  We'll likely record the answers here as well.

Unanswered questions:

Pm-20:  What's the result of something like...?
            my $x = [<a b c d>];  'a b c d' ~~ / $x /;

Pm-23:  Should TOP (or any other grammar regex) be considered
    to have implied $-anchors when invoked from .parse?
    (See RT #77022.) Example:

         grammar B { regex TOP { b } };
         say B.parse('bcd');                # failed match?

Pm-24:  Is               token { <abc>* }  
        the same as      regex { [ <abc>* ]: } 
        or               regex { [ <abc>: ]*: }  ?
    
===========================================================

Answered questions:


Pm-1:  In STD.pm, what is the semantic or key difference between <noun>
    and <term>?

None, they are now unified under <term>

----------------------

Pm-2:  Are calls to subrules in other grammars still valid as
    C<<  / abc <OtherGrammar::xyz> def / >> ?  If so, then 
    for the invocation of the subrule, do we construct a new
    cursor of type OtherGrammar and invoke the 'xyz' method on it?
    (Pm's preference is "yes" and "yes", but want confirmation.)

Yes, and yes.  That's essentially what STD is already doing for
Regexen and such.

----------------------

Pm-3:  When we generate a metaop, where does it live?  Lexical?
    Package?  If package, then what package?

Good question, if CORE is immutable, we can't add to it.  Probably
UNIT, though that might prevent sharing of common definitions
among different compilation units.  On the other hand, being sure
you're based on the same underlying semantics is difficult anyway
without a lot of lifting, so maybe UNIT is good enough for now.

----------------------

Pm-4:  The C<.ast> method on a Match object returns the matched
    text if no abstract object has been set.  Is there (or should
    there be) a method to determine if an abstract object has been
    set?  (Currently I'm using C<.peek_ast> in nqp-rx for this.)

Uh, surely .ast is false if there isn't one.

[S05:2434 r28936 currently says that .ast returns the matched text,
I expect this to change shortly.  --Pm]

----------------------

Pm-5:  How "read-only" are subroutine parameters?  For example, given
    a subroutine like  C<<  sub abc($x) { ... }  >>, I know we can't
    assign to C<$x>, but can it be rebound using &infix:<:=> in the 
    body of the sub?

That seems okay to me, since the intent is to not modify the
passed in object, and the rebinding effectively anonymizes the
argument that was bound.

----------------------

Pm-6:  Is there a syntax that allows a (trusted) routine to access
    the private attributes of another object without going through
    an accessor?  For example, if object $b has a private attribute 
    of "$!xyz", is there a syntax for me to get to that attribute?

See r28932.

[Answer:  as of r28932, C<< $b!SomeClass::xyz >>.  --Pm]

----------------------

Pm-7:  S05 says that a match's reduction object is given by the
    C<:action> parameter, STD.pm's initparse has C<:actions> and
    $*ACTIONS.  Should we pick one and stick with it?  PGE and 
    Rakudo have traditionally used C<:action> -- if there's to 
    be a change, now would be a good time for it.  :-)
    (After looking at the way I typically use it, I'm leaning
    towards the plural form. --Pm)

Now pluralized in specland.

----------------------

Pm-8: Are closures embedded in regexes creating a new lexical scope,
    or do they share the same scope as the regex block itself?
    (Currently I'm assuming they create a new scope, to be consistent
    with other uses of curlies. --Pm

Yes.

----------------------

Pm-9: Inside of a regex, what happens with C<< <[z..a]> >> ?  Is it
    a compile-time error, an empty range, or ... ?

Let's make it a compile-time error for direct code, and a failure
with warning for code compiled indirectly via <$code>.

----------------------

Pm-10:  Subs are canonically considered to be stored in symbol
    tables (lexpads, namespaces) with the & sigil.  Is the same
    true for methods?  If we ask a method for its name or otherwise
    obtain a list of method names from an object, would we expect
    those names to have a & sigil as well?  (For HLL interop reasons
    Pm tends to want methods to not include a & sigil, but it's not
    a strong tendency.)

Whether there's the & sigil or not depends on where you're storing
the name.  The MOP doesn't keep the sigil, but if a method is declared
"my" or "our", the alias in the symbol table does have the &, since as
far as the symbol table is concerned, the method is just a subroutine.
(Also note that, while the MOP doesn't track the sigil on methods,
it probably does track the sigil on attributes, such as $!foo, &!foo.)
One could view exportation of methods as multi as a two-step process; first,
declare my or our to make the & alias in the current lexpad, then
"is export" does no more magic than it ordinarily does.  I suppose
method aliases are always considered multi when they show up in a
symbol table.

----------------------

Kh-1: Is it true that Bool::True.name should work?  Should it return
    the string 'True'?
    I ask because of http://rt.perl.org/rt3/Public/Bug/Display.html?id=66576

Yes, because Bool is an object type, so it knows its .name.  (A native
bool would not know its name, according to S12:1649.)

----------------------

Jw-1: What is the difference between :(\$x) and :($x is ref)? Both
would need to leave the original argument untouched. Since C<is ref>
is meant to be used for e.g. map, which needs to work on lists like
(1,2,3), it can't cut constants out, so it seems no more constraining
than :(\$x) too. Should one of them go away?

No difference that I can see offhand.  I'm inclined to make 'is ref' go away
and keep the backslashed form.

----------------------

Pm-11: S11:257 says "Without an import list, C<import> imports
    the C<:DEFAULT> imports."  How does one import one of the
    other tagsets?  (I think I'm missing something obvious here.)

I believe the idea is simply to use the tagset pair directly as an argument.
And since lexical importation is assumed, there's an implicit :MY() around it.
So 'use Foo :tag' is probably short for 'use Foo :MY(:tag)'.

----------------------

Ml-1: In my $x; say($x~'a'), what's the output? "a\n" or "Object()\n"?
      If it's "a\n", what kind of undef is stored in $x?

Here is perhaps a decent compromise, based on the recent Num vs
Numeric distinction.  If we assume a similar Str vs Stringy difference
and give the ~ operators Stringy semantics, we could distinguish basic
stringification from the more abstract ~ like so:

    my $a;              # always stores Object()

    say $a.perl         # produces 'Object'

    say $a.Str          # stringifies to 'Object()'
    say $a              # uses .Str internaly, so also stringifies to 'Object()'

    say $a.Stringy      # says '' with warning
    say ~$a             # says '' with warning
    say $a ~ 'tion'     # says 'tion' with warning

As with type objects, junctions also need different abstraction levels,
so we could say that

    any(1,2,3).Str      # 'any(1,2,3)'
    ~any(1,2,3)         # any('1','2','3')

Note that most methods would still autothread on junctions by virtue of
not being defined in the junction class.

----------------------

Pm-12: S05:2121 says that smartmatching against regex/token/rule
    automatically anchors the match at both ends.  What construct is
    actually responsible for performing the anchor checks?  Is it the
    smart match operator (infix:<~~>), the .ACCEPTS method on the
    regex/token/rule, an option/flag passed to the regex/token/rule,
    or something else?

S05:2114 says that these are of type Method, not Regex, and therefore
it's probable that Method.ACCEPTS sets up the .parse with :p semantics
and also checks that the returned Cursor ends at the end of the string.

Or maybe we add a flag to .parse that checks anchoring on both ends,
if it seems generally useful.  That'd help with the / ^ a | b | c $/,
if people instead got into the habit of saying mt/ a | b | c / or
some such, where :t would mean "totally", or "token", or some such.
(Unfortunately, most of the other letters are taken, like :a (for
:all, but already means :accent), or would at least be confusing
if overloaded.  For instance, we could use :x without an arg to mean
match exactly, but :x(1) would mean something rather different.  mx//
reads well though.  Could also make a case for overloading :p without
an argument, but mp doesn't do much for me.)

Anyway, it's Method.ACCEPTS that makes the decision, I suspect,
regardless of how it's implemented.  We can always switch to
a .parse option later.

----------------------

Pm-13:  What's the parent type of C<Regex>?  (I hope it's C<Method>.)

Don't see any reason why not, offhand, unless Regex is really a role
that gets mixed into ordinary methods (hence we could mix it into
such methods as the opp as well).

----------------------

Pm-14:  Is the :c modifier allowed on token/regex/rule?  I.e., could
    someone do...?
        'abcdef' ~~ token :c(1) { cdef }

:c and :p are intended primarily as mods to the Cursor constructor, not to
the matcher method, so I don't think so, unless we decide to support them
anywhere within a regex, which seems relatively useless and confusing.  (Well,
a better case can be made for internal :p(1) than for :c(1), in any case.)

----------------------

Pm-15:  S11:300 gives the following as an example of importing
    a tagset into package scope:

        require Sense :OUR<ALL> # but this works

    Should this be :OUR(:ALL) instead?  It seems to me that
    :OUR<ALL> would attempt to import a &ALL symbol (since
    :MY<common> imports the &common symbol).

Yes, just a typo.

----------------------

Pm-16:  S03:1996 looks like a fossil, or at least inconsistent 
    with S07:72.  Any clarifications?

See r29571.

    [Pm]  Okay, that helps, but there still seems to be an
    inconsistency.  r29571 says that list assignment is "mostly
    eager", but then says it evaluates leading iterators as long as 
    they're known to be finite.  In other words, it suspends on the
    first iterator that is not provably finite.  But S07 says that 
    "mostly eager" obtains all items until reaching a value that is 
    known to be infinite.  "Not provably finite" is not the same as 
    "known to be infinite"; the S03 interpretation would stop obtaining 
    values on anything that _might_ be infinite, the S07 interpretation 
    would eagerly obtain values until it reaches something that is 
    _known_ to be (or declares itself to be) infinite.

"Mostly Eager" is now allowed to slack off after chewing on (or refusing
to chew on) something that is 'not provably infinite'.

----------------------

Pm-17:  Are the builtin types such as C<Num>, C<Int>, C<Rat>, 
    C<Str>, C<List>, etc.  subclasses of C<Cool> or do they
    rely on C<Cool>'s method fallback mechanisms for the
    common methods?  (Or, another way of asking:  is
    C<List ~~ Cool> true?)

As it currently stands the fallback is only in the multi dispatcher.
For single dispatch BuiltinType isa Cool.

----------------------

Pm-18:  (Confirmation request)   S03:2111 indicates that the C< @(...) > 
    sigil contextualizer is the same as the C<list> listop.  Is this correct?
    In particular, given C< my $a = [1,2,3]; >, then C< +@($a) >  is 1 and  
    C< +@$a > is 3 ?   (For some reason I had been thinking that @($a) would
    act more like $a.list  than list($a).)

This part of the design seems rather confused in several dimensions.
We will need to detangle some things.  First off, we will probably
need to decouple the sigils from specific flattening/non-flattening
behavior.  The @ sigil is probably going to mean some combination of
Positional/Iterable, and not imply anything further such as .list.
(Further context is supplied by how you use it.)  The @@ sigil as
slice context is almost certainly going away just for being too ugly,
and slices are also fundamentally ordered just like flat lists.

I think that @($a) and @$a must be made to mean the same thing.
The form is fundamentally macroish, not function-callish.
so @($a,$b) is not making a Capture out of $a and $b.  It does
get a parcel, and whatever processes it treats that as an item,
not as a list.  Parcel is iterable, so @ is basically a no-op.
For @($a) or @$a, we have a degenerate parcel which would unwrap
to $a in either case, so in both cases the make-iterable
code sees $a direction, and not the parcel around it.

Similarly for any other coercion, we want to force macroization to
treat both Foo(...) and (...).Foo the same, so ... should be
treated as bare Parcel without any additional assumptions.

Therefore list($a) and $a.list should do the same thing, if indeed
it's a coercion.  But again, that thing it's doing is something
different from @.  We won't use the sigils to indicate the
difference between flat/slice, just as we don't with lazy/eager/hyper.
These contexts are looking more like dynamic variables or parameters
internally, but attaching dynamic context to sigils is just as
bad as trying to make other data structures intrinsically carry
context they shouldn't know.

I'm trying to come up with a new slice notation that isn't sigil
based.

----------------------

Pm-19:  In each statement below, how many times is the block argument
    to .map() executed?  (Assume the block has arity/count of 1.)

        my @b  = (1,2,3 Z 4,5,6).map({ ... });
        my @@c = (1,2,3 Z 4,5,6).map({ ... });

        my ($x, $y, @@z) = (1,2,3 Z 4,5,6).map({ ... });

Iterating subsignatures need a rethink, as discussed on IRC.  If people
expect a map block of -> $x { $x } to flatten, then $x is being
bound in the variadic/flattening part of a bind, not in the positional,
which defaults to getobj.  So either we force people to write

    -> *$x, *$y {...}

(ick, doesn't extend to $^x or $_ easily) or we find some way
of flipping the default binding of some block sigs to being slurpy
scalars that use .get by default, and then use whatever slice notation
we come up with to change the default back to .getobj.  So there's
some hidden transmogrifier that map/grep use on their block parameter
to make its sig behave, or they don't call the block directly, but
rely on something else to do the transformation to a subpattern match.

The former approach:

    sub map (&block, \$orig) {
        my &flatblock := &block.default_to_slurpy;
        my $cursor = $orig.get_iter_cursor;
        gather loop {
            take &flatblock($cursor, $newcursor) E last;
            $cursor = $newcursor; # or some such
        }
    }

where .default_to_slurpy takes the sig of -> $x and turns it into -> *$x,
and get_iter_cursor gets a thread of pattern-matching iteration from the original
capture.  (Which we don't strictly have to keep around here, since there's
no nextsame, but still.)

The latter approach would be more like:

    sub map (&block, \$orig) {
        my $cursor = $orig.get_iter_cursor;
        gather loop {
            take $cursor.apply(&block) E last;
        }
    }

where .apply would both mutate the block's sig semantics somehow and also
mutate the cursor so we don't have to track it.

Update: more recent discussion on IRC suggests that the instruction for whether
to anchor the sig match at the end should not come in either of those ways,
but as a flag within the cursor itself: whereas a normal capture always
anchors, a CaptureCursor has a flag that can go either way, but is set by
map/grep to not anchor, but just advance the pointer.  (Or possibly they never
anchor, and you have to convert back to a Capture, if we want it type based.)
I will assume there's a Capture method, .get_unanchored_cursor for the nonce,
which does the right thing.

With a mutable cursor, the notation is rather simple, (but the semantics
are more problematic):

    sub map (&block, \$cap) {
        my CaptureCursor $cursor = $cap.get_unanchored_cursor;
        gather loop {
            take block(|$cursor) E last;
        }
    }

(That's assuming that |$cursor does the right thing there, and doesn't
convert back to Capture.)

Alternately, we could use immutable cursors, but then we need to tap in
at a lower level for the invocation, in order to return both a new capture
and a result parcel, something like:

    sub map (&block, \$cap) {
        my CaptureCursor $cursor = $cap.get_unanchored_cursor;
        gather loop {
            my ($newcursor, $parcel) := INVOKE_WITH_CURSOR(&block, $cursor);
            take |$parcel E last;
        }
    }

Or maybe the last goes on the invoke:

    sub map (&block, \$cap) {
        my CaptureCursor $cursor = $cap.get_unanchored_cursor;
        gather loop {
            my ($newcursor, $parcel) := INVOKE_WITH_CURSOR(&block, $cursor) // last;
            take |$parcel;
        }
    }

The immutable form is more amenable to invocation that has to track
how many elements were consumed (like 'reduce' maybe?), since you
could just compare the position of old cursor with the new, without
having to remember the old position specially.  (I'm imagining cursors
as lightweight objects pointing into the original capture here, not
as heavy capture copies, much as a grammar parser doesn't pass the
original text around, but just a match position into the original.)

Multiple return values make it harder to do fancy junctional tricks
on Z without temporaries, though.  Self-mutating cursors jangle my
FP nerves, but maybe I can argue myself into liking them from an
OO perspective.  Maybe a mutable cursor is just a container of
the current immutable cursor, and we could perhaps have it both ways,
modulo yet another level of indirection.  Performance suffers, maybe.

----------------------

Pm-21:  Does a regex in a grammar definition "hides" an outer-scoped 
    lexical regex of the same name?  (I'm guessing "yes" but want
    to confirm.)

        my regex abc { ... };

        grammar XYZ {
            regex TOP { <abc> };
            regex abc { ... };
        }

        # XYZ.TOP ends up calling the XYZ.abc, and not &abc, yes?

Pm-21 and Pm-22 have been abandoned; regexes outside of grammars
are expected to make use of an anonymous Grammar slang that is
updated with the new regexes.

Pm-22:  What about something like ... ?

        my regex alpha { <[A..Z]> }

        grammar XYZ {
            regex TOP { <alpha> };   # uses the lexical &alpha?
        }

Pm-21 and Pm-22 have been abandoned; regexes outside of grammars
are expected to make use of an anonymous Grammar slang that is
updated with the new regexes.

----------