Larraysaywhut?

---

Larry Wall, (in)famous creator of that (in)famous Perl language, has contributed
a few cents' worth to the debate over "scripting" languages:

I think, to most people, scripting is a lot like obscenity. I can't define it, but
I'll know it when I see it.

Aside from the fact that the original quote reads "pornography" instead of "obscenity",
I get what he's talking about. Finding a good definition for scripting is like trying
to find a good definition for "object-oriented" or "service-oriented" or... come to
think of it, like a lot of the terms that we tend to use on a daily basis. So I'm
right there along with him, assuming that his goal here is to call out a workable
definition for "scripting" languages.

Here are some common memes floating around:

    Simple language
    "Everything is a string"
    Rapid prototyping
    Glue language
    Process control
    Compact/concise
    Worse-is-better
    Domain specific
    "Batteries included"

...I don't see any real center here, at least in terms of technology. If I had to
pick one metaphor, it'd be easy onramps. And a slow lane. Maybe even with some optional
fast lanes.

I'm not sure where some of these memes come from, but some of them I recognize (Simple
language, Rapid prototyping, glue language, compact/concise), some of them are new
to me ("Everything is a string", process control), and some of them I seriously question
the sanity of anybody suggesting them (worse-is-better, domain specific, "batteries
included"). Fortunately he didn't include the "dynamically typed" or "loosely coupled"
memes, which I hear tagged on scripting languages all the time.

But basically, scripting is not a technical term. When we call something a scripting
language, we're primarily making a linguistic and cultural judgment, not a technical
judgment. I see scripting as one of the humanities. It's our linguistic roots showing
through.

I can definitely see the use of the term "scripting" as a term of value judgement,
but I'm not sure I see the idea that scripting languages somehow demonstrate our linguistic
roots.

We then are treated to one-sentence reviews of every language Larry ever programmed
in, starting from his earliest days in BASIC, with some interesting one-liners scattered
in there every so often:

On Ruby: "... a great deal of Ruby's syntax is borrowed from Perl, layered
over Smalltalk semantics."

On Lisp: "Is LISP a candidate for a scripting language? While you can certainly
write things rapidly in it, I cannot in good conscience call LISP a scripting language.
By policy, LISP has never really catered to mere mortals. And, of course, mere mortals
have never really forgiven LISP for not catering to them."

On JavaScript: "Then there's JavaScript, a nice clean design. It has some
issues, but in the long run JavaScript might actually turn out to be a decent platform
for running Perl 6 on. Pugs already has part of a backend for JavaScript, though sadly
that has suffered some bitrot in the last year. I think when the new JavaScript engines
come out we'll probably see renewed interest in a JavaScript backend." Presumably
he means a new JavaScript backend for Perl 6. Or maybe a new Perl 6 backend for JavaScript.

On scripting langauges as a whole: "When I look at the present situation,
what I see is the various scripting communities behaving a lot like neighboring tribes
in the jungle, sometimes trading, sometimes warring, but by and large just keeping
out of each other's way in complacent isolation."

Like the prize at the bottom of the cereal box, if you can labor through all of this,
though, you get treated to one of the most amazing succinct discussions/point-lists
of language design and implementation I've seen in a long while; I've copied that
section over verbatim, though I annotate with my own comments in italics:

early binding / late binding

Binding in this context is about exactly when you decide which routine you're going
to call for a given routine name. In the early days of computing, most binding was
done fairly early for efficiency reasons, either at compile time, or at the latest,
at link time. You still tend to see this approach in statically typed languages. With
languages like Smalltalk, however, we began to see a different trend, and these days
most scripting languages are trending towards later binding. That's because scripting
languages are trying to be dwimmy (Do What I Mean), and the dwimmiest decision is
usually a late decision because you then have more available semantic and even pragmatic
context to work with. Otherwise you have to predict the future, which is hard.

So scripting languages naturally tend to move toward an object-oriented point of view,
where the binding doesn't happen 'til method dispatch time. You can still see the
scars of conflict in languages like C++ and Java though. C++ makes the default method
type non-virtual, so you have to say virtual explicitly to get late binding. Java
has the notion of final classes, which force calls to the class to be bound at compile
time, essentially. I think both of those approaches are big mistakes. Perl 6 will
make different mistakes. In Perl 6 all methods are virtual by default, and only the
application as a whole can tell the optimizer to finalize classes, presumably only
after you know how all the classes are going to be used by all the other modules in
the program.

[Frankly, I think he leaves out a whole class of binding ideas here, that being
the "VM-bound" notion that both the JVM and the CLR make use of. In other words, the
Java language is early-bound, but the actual linking doesn't take place until runtime
(or link time, as it were). The CLR takes this one step further with its delegates
design, essentially allowing developrs to load a metadata token describing a function
and construct a delegate object--a functor, as it were--around that. This is, in some
ways, a highly useful marriage of both early and late binding.

[I'm also a little disturbed by his comments "only the application as a whole
can tell the optimizer to finalize classes, presumably only after you know how all
that classes are going to be used by all the other modules in the program. Since when
can programmers reasonably state that they know how classes are going to be used by
all the other modules in the program? This seems like a horrible set-you-up-for-failure
point to me.]

single dispatch / multiple dispatch

In a sense, multiple dispatch is a way to delay binding even longer. You not only
have to delay binding 'til you know the type of the object, but you also have to know
the types of all rest of the arguments before you can pick a routine to call. Python
and Ruby always do single dispatch, while Dylan does multiple dispatch. Here is one
dimension in which Perl 6 forces the caller to be explicit for clarity. I
think it's an important distinction for the programmer to bear in mind, because single
dispatch and multiple dispatch are philosophically very different ideas, based on
different metaphors.

With single-dispatch languages, you are basically sending a message to an object,
and the object decides what to do with that message. With multiple dispatch languages,
however, there is no privileged object. All the objects involved in the call have
equal weight. So one way to look at multiple dispatch is that the objects are completely
passive. But if the objects aren't deciding how to bind, who is?

Well, it's sort of a democratic thing. All the routines of a given name get together
and hold a political conference. (Well, not really, but this is how the metaphor works.)
Each of the routines is a delegate to the convention. All the potential candidates
put their names in the hat. Then all the routines vote on who the best candidate is,
and the next best, and the next best after that. And eventually the routines themselves
decide what the best routine to call is.

So basically, multiple dispatch is like democracy. It's the worst way to do late binding,
except for all the others.

But I really do think that's true, and likely to become truer as time goes on. I'm
spending a lot of time on this multiple dispatch issue because I think programming
in the large is mutating away from the command-and-control model implicit in single
dispatch. I think the field of computation as a whole is moving more toward the kinds
of decisions that are better made by swarms of insects or schools of fish, where no
single individual is in control, but the swarm as a whole has emergent behaviors that
are somehow much smarter than any of the individual components.

[I think it's a pretty long stretch to go from "multiple dispatch", where the
call is dispatched based not just on the actual type of the recipient but the caller
as well, to suggesting that whole "swarms" of objects are going to influence where
the call comes out. People criticized AOP for creating systems where developers couldn't
predict, a priori, where a call would end up, how will they react to systems where
nondeterminism--having no real idea at source level which objects are "voting", to
use his metaphor--is the norm, not the exception?]

eager evaluation / lazy evaluation

Most languages evaluate eagerly, including Perl 5. Some languages evaluate all expressions
as lazily as possible. Haskell is a good example of that. It doesn't compute anything
until it is forced to. This has the advantage that you can do lots of cool things
with infinite lists without running out of memory. Well, at least until someone asks
the program to calculate the whole list. Then you're pretty much hosed in any language,
unless you have a real Turing machine.

So anyway, in Perl 6 we're experimenting with a mixture of eager and lazy. Interestingly,
the distinction maps very nicely onto Perl 5's concept of scalar context vs. list
context. So in Perl 6, scalar context is eager and list context is lazy. By default,
of course. You can always force a scalar to be lazy or a list to be eager if you like.
But you can say things like for 1..Inf as long as your loop exits some
other way a little bit before you run into infinity.

[This distinction is, I think, becoming one of continuum rather than a binary
choice; LINQ, for example, makes use of deferred execution, which is fundamentally
a lazy operation, yet C# itself as a whole generally prefers eager evaluation where
and when it can... except in certain decisions where the CLR will make the call, such
as with the aforementioned delegates scenario. See what I mean?]

eager typology / lazy typology

Usually known as static vs. dynamic, but again there are various positions for the
adjustment knob. I rather like the gradual typing approach for a number of reasons.
Efficiency is one reason. People usually think of strong typing as a reason, but the
main reason to put types into Perl 6 turns out not to be strong typing, but rather
multiple dispatch. Remember our political convention metaphor? When the various candidates
put their names in the hat, what distinguishes them? Well, each candidate has a political
platform. The planks in those political platforms are the types of arguments they
want to respond to. We all know politicians are only good at responding to the types
of arguments they want to have...

[OK, Larry, enough with the delegates and the voting thing. It just doesn't work.
I know it's an election year, and everybody wants to get in on the whole "I picked
the right candidate" thing, but seriously, this metaphor is getting pretty tortured
by this point.]

There's another way in which Perl 6 is slightly more lazy than Perl 5. We still have
the notion of contexts, but exactly when the contexts are decided has changed. In
Perl 5, the compiler usually knows at compile time which arguments will be in scalar
context, and which arguments will be in list context. But Perl 6 delays that decision
until method binding time, which is conceptually at run time, not at compile time.
This might seem like an odd thing to you, but it actually fixes a great number of
things that are suboptimal in the design of Perl 5. Prototypes, for instance. And
the need for explicit references. And other annoying little things like that, many
of which end up as frequently asked questions.

[Again, this is a scenario where smarter virtual machines and execution engines
can help with this--in Java, for example, the JVM can make some amazing optimizations
in its runtime compiler (a.k.a. JIT compiler) that a normal ahead-of-time compiler
simply can't make, such as monomorphic interface calls. One area that I think he's
hinting at here, though, which I think is an interesting area of research and extension,
is that of being able to access the context in which a call is being made, a la the
.NET context architecture, which had some limited functionality in the EJB space,
as well. This would also be a good "middle-ground" for multi-dispatch, since now the
actual dispatch could be done on the basis of the context itself, which could be known,
rather than on random groups of objects that Larry's gathered together for an open
conference on dispatching the method call.... I kid, I kid.]

limited structures / rich structures

Awk, Lua, and PHP all limit their composite structures to associative arrays. That
has both pluses and minuses, but the fact that awk did it that way is one of the reasons
that Perl does it differently, and differentiates ordered arrays from unordered hashes.
I just think about them differently, and I think a lot of other people do too.

[Frankly, none of the "popular" languages really has a good set-based first-class
concept, whereas many of the functional languages do, and thanks to things like LINQ,
I think the larger programming world is beginning to see the power in sets and set
projections. So let's not limit the discussion to associative arrays; yes, they're
useful, but in five years they'll be useful in the same way that line-numbered BASIC
and use of the goto keyword can still be useful.]

symbolic / wordy

Arguably APL is also a kind of scripting language, largely symbolic. At the other
extreme we have languages that eschew punctuation in favor of words, such as AppleScript
and COBOL, and to a lesser extent all the Algolish languages that use words to indicate
blocks where the C-derived languages use curlies. I prefer a balanced approach here,
where symbols and identifiers are each doing what they're best at. I like it when
most of the actual words are those chosen by the programmer to represent the problem
at hand. I don't like to see words used for mere syntax. Such syntactic functors merely
obscure the real words. That's one thing I learned when I switched from Pascal to
C. Braces for blocks. It's just right visually.

[Sez you, though I have to admit my own biases agree. As with all things, though,
this can get out of hand pretty quickly if you're not careful. The prosecution presents
People's 1, Your Honor: the Perl programming langauge.]

Actually, there are languages that do it even worse than COBOL. I remember one Pascal
variant that required your keywords to be capitalized so that they would stand out.
No, no, no, no, no! You don't want your functors to stand out. It's shouting the wrong
words: IF! foo THEN! bar ELSE! baz END! END! END! END!

[Oh, now, that's just silly.]

Anyway, in Perl 6 we're raising the standard for where we use punctuation, and where
we don't. We're getting rid of some of our punctuation that isn't really pulling its
weight, such as parentheses around conditional expressions, and most of the punctuational
variables. And we're making all the remaining punctuation work harder. Each symbol
has to justify its existence according to Huffman coding.

Oddly, there's one spot where we're introducing new punctuation. After your sigil
you can add a twigil, or secondary sigil. Just as a sigil tells you the basic structure
of an object, a twigil tells you that a particular variable has a weird scope. This
is basically an idea stolen from Ruby, which uses sigils to indicate weird scoping.
But by hiding our twigils after our sigils, we get the best of both worlds, plus an
extensible twigil system for weird scopes we haven't thought of yet.

[Did he just say "twigil"? As in, this is intended to be a serious term? As in,
Perl wasn't symbol-heavy enough, so now they're adding twigils that will hide after
sigils, with maybe forgils and fivegils to come in Perl 7 and 8, respectively?]

We think about extensibility a lot. We think about languages we don't know how to
think about yet. But leaving spaces in the grammar for new languages is kind of like
reserving some of our land for national parks and national forests. Or like an archaeologist
not digging up half the archaeological site because we know our descendants will have
even better analytical tools than we have.

[Or it's just YAGNI, Larry. Look, if your language wants to have syntactic macros--which
is really the only way to have langauge extensibility without having to rewrite your
parser and lexer and AST code every n number of years, then build in syntactic macros,
but really, now you're just emulating LISP, that same language you said wasn't for
mere mortals, waaaay back there up at the top.]

Really designing a language for the future involves a great deal of humility. As with
science, you have to assume that, over the long term, a great deal of what you think
is true will turn out not to be quite the case. On the other hand, if you don't make
your best guess now, you're not really doing science either. In retrospect, we know
APL had too many strange symbols. But we wouldn't be as sure about that if APL hadn't
tried it first.

[So go experiment with something that doesn't have billions of lines of code scattered
all across the planet. That's what everybody else does. Witness Gregor Kiczales' efforts
with AspectJ: he didn't go and modify Java proper, he experimented with a new language
to see what AOP constructs would fit. And he never proposed AspectJ as a JSR to modify
core Java. Not because he didn't want to, mind you, I know that this was actively
discussed. But I also know that he was waiting to see what a large-scale AOP system
looked like, so we could find the warts and fix them. The fact that he never opened
an AspectJ JSR suggests to me that said large-scale AOP system never materialized.]

compile time / run time

Many dynamic languages can eval code at run time. Perl also takes it the other direction
and runs a lot of code at compile time. This can get messy with operational definitions.
You don't want to be doing much file I/O in your BEGIN blocks, for instance.
But that leads us to another distinction:

declarational / operational

Most scripting languages are way over there on the operational side. I thought Perl
5 had an oversimplified object system till I saw Lua. In Lua, an object is just a
hash, and there's a bit of syntactic sugar to call a hash element if it happens to
contain code. Thats all there is. [Dude, it's the same with JavaScript/ECMAScript.
And a few other langauges, besides.] They don't even have classes. Anything resembling
inheritance has to be handled by explicit delegation. That's a choice the designers
of Lua made to keep the language very small and embeddable. For them, maybe it's the
right choice.

Perl 5 has always been a bit more declarational than either Python or Ruby. I've always
felt strongly that implicit scoping was just asking for trouble, and that scoped variable
declarations should be very easy to recognize visually. Thats why we have my.
It's short because I knew we'd use it frequently. Huffman coding. Keep common things
short, but not too short. In this case, 0 is too short.

Perl 6 has more different kinds of scopes, so we'll have more declarators like my and our.
But appearances can be deceiving. While the language looks more declarative on the
surface, we make most of the declarations operationally hookable underneath to retain
flexibility. When you declare the type of a variable, for instance, you're really
just doing a kind of tie, in Perl 5 terms. The main difference is that you're tying
the implementation to the variable at compile time rather than run time, which makes
things more efficient, or at least potentially optimizable.

[The whole declarational vs operational point here seems more about type systems
than the style of code; in a classless system, a la JavaScript/ECMAScript, objects
are just objects, and you can mess with them at runtime as much as you wish. How you
define the statements that use them, on the other hand, is another axis of interest
entirely. For example, SQL is a declarational language, really more functional in
nature (since functional languages tend to be declarational as well), since the interpreter
is free to tackle the statement in any sub-clause it wishes, rather than having to
start from the beginning and parse right. There's definitely greater distinctions
waiting to be made here, IMHO, since there's still a lot of fuzziness in the taxonomy.]

immutable classes / mutable classes

Classes in Java are closed, which is one of the reasons Java can run pretty fast.
In contrast, Ruby's classes are open, which means you can add new things to them at
any time. Keeping that option open is perhaps one of the reasons Ruby runs so slow.
But that flexibility is also why Ruby has Rails. [Except that Ruby now compiles
to the JVM, and fully supports open classes there, and runs a lot faster than the
traditional Ruby interpreter, which means that either the mutability of classes has
nothing to do with the performance of a virtual machine, or else the guys working
on the traditional Ruby interpreter are just morons compared to the guys working on
Java. Since I don't believe the latter, I believe that the JVM has some intrinsic
engineering in it that the Ruby interpreter could have--given enough time and effort--but
simply doesn't have yet. Frankly, from having spelunked the CLR, there's really nothing
structurally restricting the CLR from having open classes, either, so long as the
semantics of modifying a class structure in memory were well understood: concurrency
issues, outstanding objects, changes in method execution semantics, and so on.]

Perl 6 will have an interesting mix of immutable generics and mutable classes here,
and interesting policies on who is allowed to close classes when. Classes are never
allowed to close or finalize themselves, for instance. Sorry, for some reason I keep
talking about Perl 6. It could have something to do with the fact that we've had to
think about all of these dimensions in designing Perl 6.

class-based / prototype-based

Here's another dimension that can open up to allow both approaches. Some of you may
be familiar with classless languages like Self or JavaScript. Instead of classes,
objects just clone from their ancestors or delegate to other objects. For many kinds
of modeling, it's actually closer to the way the real world works. Real organisms
just copy their DNA when they reproduce. They don't have some DNA of their own, and
an @ISA array telling you which parent objects contain the rest of their
DNA.

[I get nervous whenever people start drawing analogies and start pursuing them
too strongly. Yes, this is how living organisms replicate... but we're not designing
living organisms. A model is just supposed to represent a part of reality, not try
to recreate reality itself. Having said that, though, there's definitely a lot to
be said for classless languages (which don't necessarily have to be prototype-based,
by the way, though it makes sense for them to be). Again, what I think makes the most
sense here is a middle-of-the-road scenario combined with open classes. Objects belong
to classes, but fully support runtime reification of types.]

The meta-object protocol for Perl 6 defaults to class-based, but is flexible enough
to set up prototype-based objects as well. Some of you have played around with Moose in
Perl 5. Moose is essentially a prototype of Perl 6's object model. On a semantic level,
anyway. The syntax is a little different. Hopefully a little more natural in Perl
6.

passive data, global consistency / active data, local consistency

Your view of data and control will vary with how functional or object-oriented your
brain is. People just think differently. Some people think mathematically, in terms
of provable universal truths. Functional programmers don't much care if they strew
implicit computation state throughout the stack and heap, as long as everything looks pure
and free from side-effects.

Other people think socially, in terms of cooperating entities that each have their
own free will. And it's pretty important to them that the state of the computation
be stored with each individual object, not off in some heap of continuations somewhere.

Of course, some of us can't make up our minds whether we'd rather emulate the logical
Sherlock Holmes or sociable Dr. Watson. Fortunately, scripting is not incompatible
with either of these approaches, because both approaches can be made more approachable
to normal folk.

[Or, don't choose at all, but combine as you need to, a la Scala or F#. By the
way, objects are not "free willed" entities--they are intrinsically passive entities,
waiting to be called, unless you bind a thread into their execution model, which then
makes them "active objects" or sometimes called "actors" (not to be confused with
the concurrency model Actors, such as Scala uses). So let's not get too hog-wild with
that "individual object/live free or die" meme, not unless you're going to differentiate
between active objects and passive objects. Which, I think, is a valuable thing to
differentiate on, FWIW.]

info hiding / scoping / attachment

And finally, if you're designing a computer language, there are a couple bazillion
ways to encapsulate data. You have to decide which ones are important. What's the
best way to let the programmer achieve separation of concerns?

object / class / aspect / closure / module / template / trait

You can use any of these various traditional encapsulation mechanisms.

transaction / reaction / dynamic scope

Or you can isolate information to various time-based domains.

process / thread / device / environment

You can attach info to various OS concepts.

screen / window / panel / menu / icon

You can hide info various places in your GUI. Yeah, yeah, I know, everything is an
object. But some objects are more equal than others. [NO. Down this road lies
madness, at least at the language level. A given application might choose to, for
reasons of efficiency... but doing so is a local optimization, not something to consider
at the language level itself.]

syntactic scope / semantic scope / pragmatic scope

Information can attach to various abstractions of your program, including, bizarrely,
lexical scopes. Though if you think about it hard enough, you realize lexical scopes
are also a funny kind of dynamic scope, or recursion wouldn't work right. A state variable
is actually more purely lexical than a my variable, because it's shared
by all calls to that lexical scope. But even state variables get cloned with closures.
Only global variables can be truly lexical, as long as you refer to them only in a
given lexical scope. Go figure.

So really, most of our scopes are semantic scopes that happen to be attached to a
particular syntactic scope.

[Or maybe scope is just scope.]

You may be wondering what I mean by a pragmatic scope. That's the scope of
what the user of the program is storing in their brain, or in some surrogate for their
brain, such as a game cartridge. In a sense, most of the web pages out there on the
Internet are part of the pragmatic scope. As is most of the data in databases. The
hallmark of the pragmatic scope is that you really don't know the lifetime of the
container. It's just out there somewhere, and will eventually be collected by that
Great Garbage Collector that collects all information that anyone forgets to remember.
The Google cache can only last so long. Eventually we will forget the meaning of every
URL. But we must not forget the principle of the URL. [This is weirdly
Zen, and either makes no sense at all, or has a scope (pardon the pun) far outside
of that of programming languages and is therefore rendered meaningless for this discussion,
or he means something entirely different from what I'm reading.] That leads us
to our next degree of freedom.

use Lingua::Perligata;

If you allow a language to mutate its own grammar within a lexical scope, how do you
keep track of that cleanly? Perl 5 discovered one really bad way to do it, namely
source filters, but even so we ended up with Perl dialects such as Perligata and Klingon.
What would it be like if we actually did it right?

[Can it even be done right? Lisp had a lot of success here with syntactic macros,
but I don't think they had scope attached to them the way Larry is looking at trying
to apply here. Frankly, what comes to mind most of all here is the C/C++ preprocessor,
and multiple nested definitions of macros. Yes, it can be done. It is incredibly ugly.
Do not ask me to remember it again.]

Doing it right involves treating the evolution of the language as a pragmatic scope,
or as a set of pragmatic scopes. You have to be able to name your dialect, kind of
like a URL, so there needs to be a universal root language, and ways of warping that
universal root language into whatever dialect you like. This is actually near the
heart of the vision for Perl 6. We don't see Perl 6 as a single language, but as the
root for a family of related languages. As a family, there are shared cultural values
that can be passed back and forth among sibling languages as well as to the descendants.

I hope you're all scared stiff by all these degrees of freedom. I'm sure there are
other dimensions that are even scarier.

But... I think its a manageable problem. I think its possible to still think of Perl
6 as a scripting language, with easy onramps.

And the reason I think its manageable is because, for each of these dimensions, it's
not just a binary decision, but a knob that can be positioned at design time, compile
time, or even run time. For a given dimension X, different scripting languages make
different choices, set the knob at different locations.

Somewhere in the universe, a budding programming language designer reads that last
paragraph, thinks to himself, I know! I'll create a language where the programmer
can set that knob wherever they want, even at runtime! Sort of like a "Option open_classes
on; Option dispatch single; Option meta-object-programming off;" thing....

And with any luck, somebody will kill him before he unleashes it on us all.

Meanwhile, I just sit back and wonder, All this from the guy who proudly claimed
that Perl never had a formal design to it whatsoever?

---

Enterprise consulting, mentoring or instruction. Java, C++, .NET or XML services.
1-day or multi-day workshops available. Contact
me for details.