Revisiting Twigils

[Ovid]

In my work to translate existing code to use Corinna, I did find it a bit frustrating to not have a clear indication of what's a field variable:

    method set_attrs (@pairs) {
        my $sequence   = [ $attrseq->@* ];
        my $attributes = { $attr->%* };

        # note: this is not a hash due to some code I deleted
        # to make this example easier to follow
        while ( my ( $attribute, $value ) = splice @pairs, 0, 2 ) {
            $attribute = lc $attribute;
            unless ( exists $attributes->{$attribute} ) {
                push @$sequence => $attribute;
            }
            $attributes->{$attribute} = $value;
        }
        return $self->_rewrite_tag( $sequence, $attributes );
    }

Twigils would instantly solve that, but I objected at first because I was concerned that even more punctuation added to the language would not help. I think I may have been wrong. However, what twigil would we use? Many of them are already used for special variables. This makes parsing much harder. But imagine if we didn't allow formats to be declared in classes (they're almost dead anyway, right?). We could use field bar :param and have a $:bar variable. Now it's easier to distinguish:

    method set_attrs (@pairs) {
        my $sequence   = [ $:attrseq->@* ];
        my $attributes = { $:attr->%* };
        while ( my ( $attribute, $value ) = splice @pairs, 0, 2 ) {
            $attribute = lc $attribute;
            unless ( exists $attributes->{$attribute} ) {
                push @$sequence => $attribute;
            }
            $attributes->{$attribute} = $value;
        }
        return $self->_rewrite_tag( $sequence, $attributes );
    }

However, for the colon, people could type $::attr and not realize they're getting $main::attr instead. Oops.

The subscript operator is a possibility: $;var, but it's ugly (yes, that's subjective).

$*var is also a possibility as it was removed in v5.10.0.

$=var. $= is for formats and we'd have to ban them in classes.

[leonerd]

The Object::Pad solution is just to prefix the field variable name with a _, which is automatically stripped off when making accessor methods. This is in fact why. It means you can recognise the fields because of their naming pattern.

use Object::Pad;
class C {
   field $_fieldvar :reader :writer;
   method m { ... $_fieldvar is visually distinct here... }
}

I'm in two minds about whether we should add them to feature-class. On the pro side:

• They look a bit more distinct
• They'd be easier for a syntax highlighter to draw attention to

and honestly that's about all. I can't really think of any other advantage of actually having them. There is a possible idea that the notation of them does lend itself a way to get access to field variables of other instances than $self.

field $:x;
field $:y;
method cmp($other) {
   return $:x <=> $other:x || $:y <=> $other:y;
}

but that's fairly outlandish at the moment.

Whereas, on the con side:

• They're impossible to create in Object::Pad - meaning that Feature::Compat::Class would no longer be possible. At all.
• The $:foo notation might be too easy to confuse with $::foo. Doubly so if we consider allowing $other:foo.

I was initially in favour of having them, only I couldn't implement it in Object::Pad. This is why we went for the leading underscore prefix $_foo convention instead; and honestly after two years of using it on CPAN I find I quite like it. It's entirely optional as well - tiny "dumb struct" classes don't need to bother.

At the moment I am leaning towards saying we shouldn't do these in core perl - or, at least, if we do they'd be optional. Making them optional means it is possible at all to write code using Feature::Compat::Class. I feel this is important - the other Feature::Compat:: modules (Try and Defer) have themselves been very useful in writing code that can take advantage of newer perl features, while at the same time still working on older perls by way of plugin syntax modules. I feel that feature-class is going to be in development for long enough that it will be very useful indeed to keep targeting both that and Object::Pad, which is what Feature::Compat::Class is for. If twigils were to become mandatory that would no longer be possible, and a major area for experimentation would be lost.

[aquanight]

And if we make field twigils optional, then it's just a different underscore, and underscore is already legal so we may as well use it.

Also maybe it's just me but I think _name tends to be common in other OO practices so it won't even be surprising to others.

[leonerd]

@aquanight It was a convention at Google when writing C++ code, that all object member variables all began with m_..., and all parameters to methods/functions began p_..., so when reading code you can easily read what is what. Also it means names never clash, so you can have mutator methods nicely easily. Written in Perl it would look:

class C {
   field $m_foo;
   method set_foo ($p_foo) { $m_foo = $p_foo; }
}

And of course it's purely a naming convention for and enforced by humans; none of the code had to know or care anything about it. Which was useful in itself.

[Ovid]

Summarizing the above ...

Do nothing.

This is the easiest. bless systems didn't have the attendant problems because accessing instance data was typically via methods or "reaching inside." So Corinna, in enforcing encapsulation, has introduced something different.

Pros and cons.

• Easy to do nothing
• Makes shadowing easier
• Harder to maintain (I already find it a frustrating on brand-new code I'm writing)
• Feature::Compat::Class is possible.

field $_name :reader;

This generates a reader without a leading underscore. It does this by special-casing an allowed identifier character in the first position of the identifier string.

This seems to imply a conflict:

field $_name :reader;
field $name  :reader;

Both of those would generate the same reader, yes? Should this be fatal?

If, for some strange reason, you wanted a reader with a single leading underscore, do you need two leading underscores? field $__two_underscores :reader

Is field $name even allowed now? I assume it would be, but now we'll have codebases which will not use leading underscores and others which will. I'm unsure how I feel about this inconsistency.

I'm unsure that I object to this approach, but it feels awkward. One one hand, it sort of gets us our twigils, but by kind of hacking it in.

Pros and cons.

• Has to be implemented (doesn't sound like a big deal)
• Makes shadowing less common
• Maybe causes a new variant of the "tabs" versus "spaces" debate
• Having a special case for a leading identifier character feels wrong
• Feature::Compat::Class is possible.

Twigils: field $name is $*name in methods

I agree that this also seems rather problematic. A clean twigil syntax (I can't believe I'm writing that) solves a lot of problems, but the Feature::Compat::Class issue is important. Also, while I like * being the twigil, I think $=name is also an option and on English-language keyboards, usually doesn't need a shift key to type.

Pros and cons:

• Prevent shadowing
• Be more maintainable
• More annoying to type
• No special casing leading identifier character
• Feature::Compat::Class is no longer possible

---

My objection to the twigil has always been contingent upon the long-standing "punctuation" objections to Perl. However, it seems it's a small price to pay for maintainability. Take care of existing programmers first, I think.

Given the pros and cons, having a proper twigil seems like the best solution if we ignore the Feature::Compat::Class issue. But we can't ignore that issue.

This is a big decision. If we don't allow twigils, we'll perpetually have this albatross around our neck, unless we go with the leading underscore compromise. Special-casing an allowed identifier character, depending on its position in the identifier, feels wrong, but this could be an aesthetic preference on my part. It also means we'll have to handle the field $name :reader; field $_name :reader; issue (may not be that big of an issue).

@leonerd argues:

I feel this is important - the other Feature::Compat:: modules (Try and Defer) have themselves been very useful in writing code that can take advantage of newer perl features, while at the same time still working on older perls by way of plugin syntax modules. I feel that feature-class is going to be in development for long enough that it will be very useful indeed to keep targeting both that and Object::Pad, which is what Feature::Compat::Class is for. If twigils were to become mandatory that would no longer be possible, and a major area for experimentation would be lost.

I think this is a solid argument. If we accept it, this decision could be irrevocable. We'd be accepting a temporary (albeit long-term) Feature::Compat::Class and giving up the permanent benefit of the twigil (unless we use the leading underscore hack).

I honestly am unsure of the best approach here, yet perhaps there is a third alternative. Make the twigil optional. In Java, shadowing can be avoided in a method by prefixing the static or instance variable with this.:

public void setName(String name) {
    this.name = name;
}

What if we have this in Corinna?

class Foo {
    field $name :param;

    # yes, we could use :writer, but this is just an example
    method set_name ($name) {
        $*name = $name;
    }

    method reverse_name () {
        return scalar reverse $name; # refers to instance variable
    }
}

In Feature::Compat::Class, that twigil isn't available, so set_name would have to be written as:

    method set_name ($new_name) {
        $name = $new_name;
    }

With that, all Feature::Compat::Class code is forwards-compatible to Corinna, but we lose backwards-compatibility.

Note that I'm not advocating for this idea, but I want to explore alternatives.

[aquanight]

A couple of thoughts to consider, based on the recommendations in perlstyle:

• Methods intended to be used publicly shouldn't be named with leading underscores. As such, class interface (methods and NCAs) autogenerated by :reader et al should not have them unless expressly asked for.
• Additionally: multiple leading underscores are typically reserved for perl special barewords (like __PACKAGE__ and __SUB__). This is probably not an issue with method names but it can come up if we start talking about people who want - for whatever reason - private but exposed members, thus trying to do field $__name :reader; in order to get _name -- they shouldn't do that.

field $_name :reader; seems like a good solution. If they want the _ intact we can give them :reader(_name)

Keeping field $name allowed is good because small trivial classes (eg "dumb struct" classes) have little need for the code-clarity benefits of twigils so it might be more convenient to dispense with them for such cases.

And if for some reason someone tries to :reader a $name and a $_name? That's https://perldoc.perl.org/perldiag#Subroutine-%25s-redefined -- at minimum.

[thoughtstream]

I still don't like the idea of twigils for Perl fields (ETOOMUCHPUNCTUATION).
And I can't say I'm keen on Object::Pad's magical underscored fields either (ETOOMUCHMAGIC).

The problem to be solved is: it's tough to distinguish field variables from other "internal" variables and parameters
within a method.

I would have thought the easiest solution was to reverse the Object::Pad approach and to suggest a convention
wherein fields are named "normally" (so the names of their readers and writers are easily predicted),
while variables and parameters that are internal to methods are named with leading underscores.

Taking @Ovid's original example, that would mean:

    field $attrseq;
    field $attr;

    method set_attrs (@_pairs) {
        my $_sequence   = [ $attrseq->@* ];
        my $_attributes = { $attr->%* };
        while ( my ( $_attribute, $_value ) = splice @_pairs, 0, 2 ) {
            $_attribute = lc $_attribute;
            unless ( exists $_attributes->{$_attribute} ) {
                push @$_sequence => $_attribute;
            }
            $_attributes->{$_attribute} = $_value;
        }
        return $self->_rewrite_tag( $_sequence, $_attributes );

Or taking a subsequent example of field/parameter name clashes:

class Foo {
    field $name :param;

    method set_name ($_name) {
        $name = $_name;
    }

    method reverse_name () {
        return scalar reverse $name;
    }
}

If we have to have something more than a convention, then I'm very much opposed to "optional" field twigils.
If we have to have twigils, they should be required on the field declaration as well, and on every use of
the field variable.

That is, you would have to write:

class Foo {
    field $*name :param;

    method set_name ($name) {
        $*name = $name;
    }

    method reverse_name () {
        return scalar reverse $*name;
    }
}

I'd still (marginally) prefer magical underscores to twigils. But in as unmagical a form as possible.
Namely: a field whose name is declared with one or more leading underscores always has all those underscores
stripped from its name when forming the names for any :reader, :writer, or :param. If you want a reader
or writer or initialization parameter name with leading underscores, use :name(_whatever) – which will not
have any leading underscores stripped.

But again, I'm still not convinced we really need either mechanism.