Closures are back again!

Those of you who've seen me speak on Java 7 at various conferences have heard me lament
(in a small way) the fact that Sun decided last year (Dec 2008) to forgo the idea
of including closures in the Java language. Imagine my surprise, then, to check my
Twitter feed and discover that, to everyone's surprise, closures are
back in as a consideration for the Java7 release.

Several thoughts come to mind:

• "WTF?!?!? This is a community effort?" Originally,
  when Sun created the Java Community Process, the tradeoff for a committee-based development
  process was against the open and fair inclusion of ideas from outside of Sun. But
  with the Java7 release still lacking a JSR (as of a few weeks ago, anyway; I haven't
  checked today to see if it was opened), and both the Modules facility and language
  extensions deferred to "Projects" (not JSRs), it seems Sun is now abandoning
  the JCP in favor of a Sun-dominant process that is certainly solicitous of the community
  at large, but not constrained or defined by it. And for the life of me, I can't tell
  if this is a good thing or a bad thing. It's good in that now we don't have to garner
  a critical mass of community momentum to get something included into the platform
  or language, but it's bad in that Sun has historically been the bigger drag on innovation
  there, not the community.
• "Can we please stop calling them closures?" This
  is a nit, but technically what we're talking about adding here are either lambda expressions
  or anonymous methods, depending on whose glossary you're using when you're talking.
  A true closure is one that will compute all referenced variables from the enclosing
  scope and automatically include them in the generated code, which (so far as I can
  tell) none of the Java anonymous method or lambda expression proposals currently include.
  But it's a nit, so I'll say it this once and then drop it.
• "Will Groovy, Scala, Clojure and all other JVM languages please report
  to the refactoring room?" People look at me quizzically when I
  say I'd like to see Java have closures in the language, because in general my take
  on language features in Java is that the Java language is more or less dead, and I
  could care less what happens to it; I'd vastly prefer to code in Groovy or Scala or
  Clojure or JRuby before writing something in Java. My rationale for wanting closures
  in Java, however, is this: by defining a common implementation for closures
  in Java, all of the above languages can refactor their implementations of anonymous
  methods/lambda expressions/etc into something that uses Java's closure implementation,
  and that'll make calling Groovy anonymous methods from Scala much much easier.
• "Why there, now?" Devoxx is apparently turning
  into JavaOne Winter, because Sun's been making a lot of pretty big announcements at
  that show, including last year's "no closures, no built-in XML support, ..."
  announcement about Java7, and now this year's "well, we lied, we're thinking
  about closures again". Fortunately I think the Devoxx folks have much better
  skills at keeping their conference relevant to the Java community than JavaOne's organizers
  did. And I say that despite the fact (or perhaps because of the fact) that
  I didn't speak there this year. ;-)
• "When is this all supposed to ship again?" Originally,
  my understanding was that JDK7 was slated to ship in the early part of 2010, but now
  rumor has it slipping to this time next year (2010). That is a huge postponement,
  and gives Microsoft a bit of an edge, since Visual Studio 2010 and .NET 4.0 are (again,
  according to rumor) supposed to ship somewhere around the end of 1Q2010. If Sun/Oracle
  keeps this up, we could very well be seeing a 2-.NET-releases-to-1-Java-release pattern,
  and that's disturbing in its own right. (Anybody else remember the days when Sun withdrew
  Java from ECMA, ISO and ANSI standardization consideration because they wanted to
  "innovate on the platform faster"?)
• "We really have no clue what we're talking about." Aside
  from rumors and hearsay (including the one that says that Mark Reinhold, who made
  the announcement, made up the syntax on the flight from the US to Belgium), we really
  don't have much by way of Sun-blessed official discussions of what this will
  look like or act like, at least none so far as I've been able to find, so any sort
  of supposition on whether it will be good or suck like an inverted hurricane is a
  tad premature. Trust me, I want to see where this goes, too, so I'll be keeping an
  eye out.

In the meantime, if you want to keep on top of the Java space, maybe it's time to consider
a trip to Antwerp this time next year, since, if the new ship date rumors are
to be believed,  it looks like Sun (once again) is planning to use Devoxx as
the platform from which to make a large announcement, this time the release Java7
itself.

Update: Ola Bini noted
that...

Two things:

• They are definitely closures. Calling them anonymous functions are incorrect,
  since they aren't really functions. Lambda expression is an OK name, but it has connotations
  that aren't really correct for a language like Java. A closure is defined as an anonymous
  piece of code that closes over at least one free variable, which in the case of this
  proposal will definitely happen. In fact, all of these will be closures, since they
  will be closing over the this at least.
• This is mostly on the level of compiler, syntax and type checking, and will NOT have
  any real implications for runtime. This means there will be no real sharing of implementation
  - at most JRuby, Groovy and Scala blocks will implement another interface (but all
  of them already implement Runnable and Callable so it's a limited win).

which prompted me to respond thusly:

First off, I actually never used the term "anonymous function"; instead,
I said "anonymous method", which, as I understand it, is how the underlying
implementation of these proposals will work: the syntax "#() return 42"
will create an anonymous inner class instance of an interface defined by the library
(in its "SimpleClosure" example, the BGGA compiler uses the interface "javax.lang.function.I",
which has one method on it, "invoke()"), which, thus, makes this an anonymous
method. We can't call them "anonymous functions" because Java has no function
type, and probably never will. (And yes, it may seem like we're splitting hairs somewhat
to differentiate between functions and methods,but once you've explored ML, Haskell,
Scala, or F#, you really begin to see a huge difference in those terms, so it's important
to be precise with our terminology, or else the conversation becomes almost entirely
meaningless.)

Neal Gafter uses the definition "A closure is a function that captures the bindings
of free variables in its lexical context." (http://gafter.blogspot.com/2007/01/definition-of-closures.html) Given
that said same post also claims that Java has no function type (and therefore, by
his definition, can't really have a closure), I suppose we could split the hairs even
further and suggest that Java will never have closures until it has true function
types. Personally, I'm happy to say that we can swap in "methods" for "functions"
in this particular discussion, but my understanding is that capturing free variables
also implies capturing variables referenced in the enclosing lexical context, which
the current "closures" proposal (as reported by Alex Miller's closures page)
will not do. (Non-final enclosing parameters will not be accessible, only those passed
in formally as parameters. Stephen
Colebourne reports as much: "[Mark Reinhold] also indicated that access to
non-final variables was unlikely.")

Given that the current proposal suggests the new #() syntax will essentially generate
an anonymous inner class with a method of the appropriate signature (though I do believe
that method handles are targeted for use at some point, based on what I've been hearing
through the rumor mill), to me it feels like the "closures" implementation
is generating an anonymous method of an anonymous class with a few other restrictions
included--hence my commentary above.

(Having said all that, the FCM
proposal does provide complete capture of all referenced variables in enclosing
scope, but Mark's keynote hasn't officially endorsed either the BGGA proposal or the
FCM proposal, and if Sun keeps to their habits, they won't. They'll build something
that's an amalgamation of all of them. Right now the current consensus seems to be
to adopt the BGGA implementation behind the FCM syntax, which jives with Neal's 0.6a
specification proposal.)

On top of that, the comment "all of these will be closures, since they will be
closing over the this at least" is not, I don't think, entirely true. The details
of the closures proposal aren't clear, but the "outer this" (which I believe
is the "this" Ola refers to above) hasn't been explicitly mentioned in any
of the closures proposals I've seen, nor have I seen any text suggesting that they
will honor it, so I don't know that this is true. Of course, in absence of a specification
or real working bits, all we can do is just speculate. However, having said that,
playing around a bit with the BGGA prototype compiler (which, admittedly, is still
one minor rev back from Neal's revised proposal), I saw no generated "outer this"
in the generated code for the generated inner class implementation of the closure.
If the comment above is meant to refer to the "this" of the inner class
instance, then that would make all methods of an object-oriented language that provided
an implicit "this" a closure, and somehow I doubt that's what Ola means,
though I could, as always, be wrong.

As for the runtime implementation, as I said earlier I believe the plan is to use
method handles (already on the table for JDK 7), which do have some runtime implications
(generally good ones, from what I can tell so far), but not beyond what was already
on the table for 7.

Enterprise consulting, mentoring or instruction. Java, C++, .NET or XML services.
1-day or multi-day workshops available. Contact
me for details.