Scala: Post-Functional, Post-Modern, or Just Perl++? March 6, 2010 | 10:30 pm

Let’s start with some background.

I complained that Scala did not seem to be very functional to me, but I didn’t really know how best to express what was fundamentally wrong with it. I did know that if “functional languages have a fixed set of features” like Scala’s creator, Odersky, claims, then it wasn’t simply “first-class functions in there, function literals, closures”, “types, generics, [and] pattern matching”. Scala has missed the functional boat in some basic way.

After a kerfuffle in the comments, Brian enlightened us all by telling us what is a functional programming language. His explanation (while being a self-admitted generalization) is summarized as follows:

So, what is it that differentiates the functional programming languages from all the other programming languages? It is simply this: the functional programming languages use, as their fundamental model of computation, the lambda calculus, while all the other programming languages use the Turing machine as their fundamental model of computation.

Six months later, Odersky responds with a very interesting post, which actually agrees that Scala is not a functional language in Brian’s sense, but instead argues that any language is functional if it “makes programming centered around functions easy and natural”. He then runs through a list of features which is in common with functional languages, noting that Scala has them within handwave enough (more on that later). He ends wishing that people would “stop thinking of functional programming as a different, novel, or exotic way to code”. Even more, though, Scala is apparently “an early example of a new breed of postfunctional languages”.

And that gets us to this blog post.

First of all, Odersky is still missing the point. It’s not about whether you use fold, map, and iter, or whether you can write closures easily. It’s not even really about pure functions vs. side-effects. To code in a functional style is a fundamentally different way of thinking about problems: instead of thinking about problems as nouns that are doing things, functional programming views a problem as a series of transformations to the world which results in an answer. This is why functional programming is considered “a different, novel, or exotic way to code”: it is a different, novel, and (as of yet) exotic way to code. It’s as different, novel, and exotic from OO as OO was from procedural. It’s a different way of thinking about the entire issue. You can check out this snippet of an IRC conversation from #ocaml for more on that.

The paragon of this way of programming is point-free programming, where you are quite literally building up a mega-function that describes how your program works, and then executing that one, single function when you run that program. If your language doesn’t lead people to re-discover point free programming at least in the small, then the language really isn’t taking function manipulation and functional language type conceptions seriously. And that’s the case with Scala: even Odersky admits that in Scala, “currying is more verbose and much less used than in other functional languages”. (Protip to Scala people: If one of the fundamental stunts of a style is pervasive in all the code but yours, you’re not in the same style of programming.)

What really gets me, though, is the claim that Scala is “an early example of a new breed of postfunctional languages”, because aside from the static typing, all the language features that Odersky trots out already exist in Perl. It’s hard to be a vanguard of a new breed of programming languages when there’s prior art from the 1980s.

Don’t believe me? The existence of a book on the topic unconvincing? Then let’s run the list of functional language features from Odersky.

  • Functions as first class values: check.
    sub apply(&$) {  # Take a function as an argument no problem
      $_[0]->($_[1]);
    }
     
    sub times2($) {  # Create a function to take
      print $_[0]*2 . "n";
    }
     
    apply(&times2, 3);
  • Convenient closure syntax: check
    my $x = 2;
    apply { print $_[0]*$x . "n" } 3;
     
    my $times_x = sub($) {
      print $_[0]*$x . "n";
    };
    $times_x->(3);
  • List comprehensions: check. (See perlfunc on list data.)
  • “Curried” function definitions and applications: check-ish.
    Okay, so calling this a “check” on Scala is a bit of a reach (cite, cite, cite, although note thishere is a more sympathetic run-down on Scala currying). Ignoring the foo(2,_:Int) syntax for a moment, we can implement basically the same style of “‘curried’ function definitions” such as Scala’s List#foldLeft.

    sub add {
      my $x = shift;
      return sub { $x + shift };
    }
    add(2)->(3);  # Okay, so you do need an extra ->

    In the case of our apply function above (where we take a function as the first argument), it’s even easier.

    apply { print $_[0]*$x . "n" } 8;

    Now, there isn’t really argument skipping (i.e.: foo(_:Int,3)) as a syntax feature, and there isn’t a built-in curry function, but if you want Scala’s Function.curried in perl, here it is:

    # This code released under Creative Commons 0 and WTFPL.
    sub curry(&@) {
      my($f,@args) = @_; 
      return sub { $f->(@args, @_); };
    }
     
    sub add($$) {
      return $_[0] + $_[1];
    }
     
    my $curried = curry(&add, 2); 
    print $curried->(3) . "n";
  • Lazy evaluation: check. See Scalar::Defer for lazy val equivalents and Tie::LazyList for lazy seq equivalents. People generally use a double-return approach for generators (which I realize are different than lazy seqs and only kinda-sorta lazy).
  • Pattern matching: check (okay, check-ish). See Switch. The decomposition isn’t there, which is the biggest weakness. But the general cumbersomeness and lack of real algebraic data types hamstrings the coolest parts of pattern matching anyway, so I’m calling it a draw. (This should be read as a generous and sympathetic ruling for Scala: Cedric Beust, for instance, rails against pattern matching/case classes and says “it’s hard for me to see case classes as anything but a failure“.)

In addition, perl’s got a few features in its favor for functional programming, like more flexible arguments, autovificiation, list/argument coercion, and dynamic symbol table mangling. Since perl also has OO capabilities, perl is at least as convincing a “post-functional language” as Scala. But there’s even more in common between the two than that.

Odersky’s “post-functional language” is really a subtype of Larry Wall’s “post-modern language”: it’s an attempt to create a language that is a grab-bag of multiple paradigms. And when you do that, you’re just begging for the complaints that you hear leveled against both perl and Scala: it’s too complicated, its syntax is weird, it’s too magical, people write in entirely distinct subsets of the language, etc. (cite, cite, cite) Now, those who master the language (or master their favorite subset of it) love the TIMTOWTDI aspect. But it also means that the language is left as a jack-of-all-trades, master of none. Yes, Scala and perl integrate a lot of powerful tools from functional languages—but learning OCaml still blew my mind, despite knowing perl for years. As I started off saying, Scala is not a functional programming language. It is a statically typed object oriented language with closures.

Now, there is a sense in which Odersky is really onto something. The world of programming is forever transformed with closures and list comprehensions as being mandatory for new high-level languages, even if they’re otherwise object oriented. And software developers are going to need how to work with them effectively if they want to read code moving forward. Yes, after 20+ years, the rest of the programming world finally caught up to one of perl’s fundamental insights.

Tags: , ,

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    True enough. I’m just trying to wrap my head around how Odersky can really grok functional programming and then assert that it’s simply bolt-on features of a language. The difference between Haskell and Java is not simply one of syntactic sugar.

    Claiming he’s mistaken about how FP works is actually a generous take—claiming he’s clinging to the FP title for Scala for some ulterior reason when he should know better is much less generous. I was trying to give him the benefit of the doubt.

  • http://james-iry.blogspot.com James Iry

    Robert Fischer Finally Admits That Scala Is Functional (http://james-iry.blogspot.com/2010/03/robert-fischer-finally-admits-that.html)

  • Martin Odersky

    Robert, please, read my post again:

    http://www.scala-lang.org/node/4960

    Nowhere did I define functional programming exclusively by a list of features. Instead I had as my central definition:

    “A functional language makes programming centered around functions easy and natural.”

    A good test of this would be: is it common to write functional programs in the language in question? It’s usually far easier to tell whether a program is functional than whether a programming language is functional.

    By that standard Scala is clearly functional; many of the classes and modules written in it are purely functional and every cool idea from Haskell seems to be implemented sooner or later in Scala (not that we don’t invent some cool ideas of our own as well).

    I don’t know Perl well enough to be able to tell whether there are many functional programs written in it, and whether the expression of such programs is easy and natural.

    Thanks anyway for giving me the benefit of the doubt that it’s just misguidedness and not evilness that makes me insist on my ways.

  • bse

    Well, at least you admit you’re writing about “religious issues”, where arguments are replaced by belief. So by definition I can’t argue with your expression of belief. However I don’t have to agree with you.

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    How do you account for this quote?

    The first thing we cared about was to have as clean an integration of functional and object-oriented programming as possible. We wanted to have first-class functions in there, function literals, closures. We also wanted to have the other attributes of functional programming, such as types, generics, pattern matching. And we wanted to integrate the functional and object-oriented parts in a cleaner way than what we were able to achieve before with the Pizza language. That was something we deeply cared about from the start.

    Later on, we discovered that this was actually very easy, because functional languages have a fixed set of features. They had been well researched and well proven, so the question was only how to best integrate that in object-oriented programming.

    The Goals of Scala’s Design

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    Ugh. I need to turn down the maximum comment nesting. It’s getting out of control, and the box is getting REALLY skinny.

  • Martin Odersky

    Classical fallacy: I said: functional languages have a well-identified, fixed set of features. So being a functional language implies having that fixed set of features (or rather, some part of it because I was talking about the union of all functional languages). You think I said: having that fixed set of features implies being a functional language. But I never said that.

    Besides, that quote is not from the article I referred to. When I said “nowhere” I meant nowhere in that article, and I think that was clear from the context.

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    Okay, sure I’m taking your quotes from both your post on post-functional languages and another one on the design decisions behind Scala. Sorry if I overgeneralized your “nowhere”.

    I read that quote as saying, “I went out to implement a functional language. This was easy, because it turns out implementing a functional language means implementing a fixed set of features.” If that’s not what you meant, what did you mean?

    I’m uncomfortable with your broad definition because it leads to confusion. Scala’s extreme flexibility is exactly why I’m so resistant to labeling it a functional programming language — people learn the subset of Scala closest to Java and then think they know functional programming. I’m really, really tired of explaining to people that functional programming is more than list comprehensions.

    BTW, I want to say it straight to you, since I think things can get lost and there are some misconceptions floating around. I really do appreciate all you’ve done on the JVM languages—especially proving to folks who only know Java that there’s more to static typing than Java’s conception of it. And for all my pedantic resistance to calling Scala a functional language, it is one of my go-to languages for heavy lifting on the JVM. (My own experiments in programming language development have Scala cores.)

  • Martin Odersky

    Adding functional language constructs was relatively easy, because they are well understood. Making Scala an organic whole was hard. We can disagree whether this succeeded or not.

    The other question is, should a functional language force you to program in a functional style? Scala clearly does not, that’s why I prefer to call it postfunctional. I understand your uneasiness that some people might take this as an excuse never to get into real FP at all.
    Where we might differ is that I believe that in the long run Scala programmers will recognize the benefits of the functional programming style even if it is not forced down on them.

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    I’d say that if a language doesn’t force you to program in a functional style (or at least strongly encourage you that way), I’d consider it suspect as being a functional programming language at all. But I’m running on the stricter definition, as we’ve established.

    I hope that your sense of the future is true, and that the functional programming style becomes more widely adopted by way of Scala. And maybe being on the inside of Scala adoption, you’re seeing that. Sitting here on the outside, though, I’ve seen the contrary, which is why I’m such a stick in the mud. Well, that and I’m preternaturally disposed towards being a stick in the mud.

  • http://enfranchisedmind.com/blog/posts/author/bhurt-aw/ Brian Hurt

    Let me turn that question around: should object oriented languages impose an object oriented approach?

    There is a value to learning, and really using, a paradigm. Is that limit you just hit a limit of the paradigm, or just a limit of your understanding of the paradigm? Until you expand your understanding, you don’t really know- and the siren call of multi-paradigm languages is that they allow you to not expand your understanding by ditching the new paradigm early and often.

    There is another comparison (one you’ll like even less) to another multi-paradigm language- C++. The reason C++ “won” was because it was multi-paradigm (procedural and OO), allowing people who didn’t really understand OO to program in it. The reason C++ “lost” was because it was a multi-paradigm language, forcing people to deal with the old paradigm whether it was valuable to or not. The history of popular programming languages 1992-2007 was basically one of going ever less multi-paradigm and ever more pure OO.

    In other words, is Scala “post-functional” in the same sense that C++ is “post-Object-Oriented”?

  • http://gabrielsw.blogspot.com Gabriel C.

    If we’re talking about how to LEARN functional programming, I totally agree that Scala is not the best choice (neither OCaml… the O part is there, lurking in the back…). I would recommend a pure functional lazily evaluated language (i.e. Haskell).
    My experience was the opposite of Robert’s fears: I started learning FP with Scala (limiting myself to the functional subset of it) and then moved to learn Haskell (and I try to pick up as much as OCaml as I can on the way), but I guess I’m just weird: I’ve learned OO in C++ :)
    But somewhat I agree in the comparison: C++ is to C as Scala is to Java

  • joe poki

    Robert, I was actually talking about this kind of strong encouragement in my previous post when I said that scala was nudging you towards FP.

  • Martin Odersky

    If Scala had taken all of Java and then added FP to it, a comparison with C++ might have merit. But as it stands Scala
    has thrown out or generalized a lot of Java’s more crufty features. Where C++ is clearly much more complicated than C, the situation between Scala and Java is more balanced. Scala’s syntax is a probably a bit simpler than Java’s but its types are richer and more powerful. So you can say that Scala shares C++’s motivation — take a well established programming style and let it branch out into a new paradigm. But it shares neither its approach (to be an almost strict superset of a base language) nor its overwhelming complexity.

  • http://blog.juma.me.uk Ismael Juma

    Hey Robert,

    I found the following in your LinkedIn page:

    “Robert Fischer is a multi-language open source developer currently specializing in Groovy in Grails. In the past, his specialties have been in Perl, Java, Ruby, and OCaml. In the future, his specialty will probably be F# or (preferably) a functional JVM language like Scala or Clojure.”

    What, did you just call Scala a functional language?!

    Ismael

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    Busted!

    Really, that’s exactly what I’ve been discussing with Martin O.—if you know functional programming already, you can do functional programming in Scala. And if you’re on the JVM, like static typing, and want to do functional programming, you really don’t have a better option.

    BTW, in Scala, I’m still struggling a bit to do functional programming in a way that’s particularly friendly to Scala’s approach — algebraic data types and function manipulation a la curry/flip are standard stunts for me, and I feel like I’m swimming upstream when I try those in Scala. I really should look more into some open source projects—anyone have one in Scala that’s particularly functional?

    (In any case, I should update that since I’m now an .Net/F# developer.)

  • joe poki

    that is hilarious!

  • peter pnin

    Robert, are you under the employ of Flying Frog Consultancy, perchance?

    There is a certain Harropity, if you will, to your blog post.

  • http://enfranchisedmind.com/blog/posts/author/candide/ Robert Fischer

    Ouch. Of all the nasty things people have said about me because of this post (up to and including calling me a “douchebag”), that’s the first one to hit home.

  • joe poki

    Robert, I am sorry people called you names but I am not sure what you expected. You published a deliberately controversial post (or may I say a very trollish one?) where you wrote off a whole language using standard (and frankly boooring) points like “it’s too complicated, its syntax is weird, it’s too magical, people write in entirely distinct subsets of the language” for one particular issue you have with it and on top of this, you called the given language’s creator a numerous times pretty much clueless when it comes to FP.

  • http://pchiusano.blogspot.com Paul Chiusano

    Functional programming just means programming with pure functions (no observable side effects). The alternative is imperative programming – programming with observable side effects. OO is orthogonal – you can program using pure functions operating over objects if you like, or your functions operating over objects can have side effects. Of course, most real programs have some mixture of functional and imperative styles.

    There are some language features that make programming in a purely functional style more convenient, sure, but basically, with some discipline you can program in this style in any language, Scala (and Perl!) included. My personal opinion is that Scala makes FP pretty convenient – your argument seems to be merely that it’s not as convenient as it could be in some instances and therefore doesn’t get your official “functional language” stamp. I guess that is fine, although the distinctions you are making seem pretty arbitrary to me.

  • http://enfranchisedmind.com/blog/posts/author/bhurt-aw/ Brian Hurt

    I don’t see what the big deal with Object Oriented languages is. I mean, you can do Object Oriented programming in languages like C and Pascal- yeah, it’s not quite as convenient as doing it in a language like Java or Ruby, but there’s no real reason to switch, is there?

  • http://gabrielsw.blogspot.com Gabriel C.

    But maybe the only true “popular” OO language is SmallTalk, it could be argued that Java is not really OO, having primitive types and static methods, and many don’t really in a fully OO way…

  • http://gabrielsw.blogspot.com Gabriel C.

    Perhaps the difference is that C and Pascal don’t have “first class” objects and Java and Ruby have.

  • Paul Kaletta

    Well, I’ve never used Perl 4, but afaik it didn’t have a reference variable type and it didn’t have lexical scoping, so your examples would not have worked in the current form.

    Glancing at the Perl 4 manpages it seems that Perl 4 could already pass around code blocks, however it seems to me that you would have had to capture variables manually, which means that closures would have been difficult to do.

  • http://enfranchisedmind.com/blog/posts/author/bhurt-aw/ Brian Hurt

    They have structures, and function pointers, and you can put the function pointers in the structures giving you virtual methods. Inheritance is a little tricky, but no one is using that anyways these days, so no problem. Yeah, it’s a little bit klunky, but all the important stuff is there, so why do we need to leave C and Pascal again?

    If you haven’t figured it out, I’m making one of my little points by adopting this position.

  • http://[email protected] alex

    Real FP languages are dominated by parenthesis, prefix notation, and weak type systems.
    Scala is dominated by curly braces, infix notation, and a strong type-system.
    Ergo, Scala is clearly not an FP language.

    Seriously though… to remove all doubt, I propose Scala should introduce a new keyword, “functional”, that when applied to methods or whole classes, guarantees an absence of side-effects within that scope. That should end all discussion!

  • http://enfranchisedmind.com/blog/posts/author/bhurt-aw/ Brian Hurt

    By this definition, Haskell and Ocaml are not real FP languages. But thank you for playing…

  • http://gabrielsw.blogspot.com Gabriel C.

    You can’t call a language OO if it doesn’t have objects as “first class citizens” and you can’t call a language functional if you don’t have functions as “first class citizens”. You can draw the line on the other side and define a functional language as one in which you can program ONLY in a functional style.
    The middle road is defined in fuzzy terms of “encourages a functional style”, “is easier to program in a functional style”, “makes harder to program in a non-functional style”, etc.

  • ndh

    In learning the ways of the FP I’ve tried a handful of languages, and for actual pragmatic use outside of academic circle-jerks I find Scala to be at the top of the list of those I’ve tried though I still have a few more to go (Erlang in 2nd place). Whether it’s a perfect fit for a dictionary definition perhaps not, but does it actually matter?

    That aside, I found the Perl examples interesting to see that many FP concepts can be implemented in a handful of lines of [terribly awkward] code. Once again it seems Perl can do anything anyone else can: but in the absolute worst possible way :)

  • Cody Koeninger

    AFAIK Perl functions are merely lambdas (anonymous subs) that have been assigned to a named value (the code slot of the identifier’s typeglob). Likewise there’s no difference between a scalar holding a reference to a lambda and a scalar holding a reference to a function.

    users-computer:~ user$ perl -d -e 42
    DB sub function { print(“called a functionn”) }

    DB $function = &function

    DB $lambda = sub { print(“called a lambdan”) }

    DB *lambda = $lambda

    DB function()
    called a function

    DB lambda()
    called a lambda

    DB $function->()
    called a function

    DB $lambda->()
    called a lambda

    I don’t see any difference in invocation syntax there. If you want to write all of your subroutine definitions as

    *bar = sub { print(“bar”) }

    you’re welcome to.

  • http://blaisorbladeprog.blogspot.com/ Blaisorblade

    @Cody Koeninger: you missed the discussed point – you are still doing a conversion, when you type:
    $function = &function and *lambda = $lambda. No such conversion is needed in more functional languages, as already argued above by Daniel Spiewak.

  • http://www.weather.com Andrew Garman

    …if you’re still futzing around in Java-land, Scala (like Groovy and JRuby) is a productivity huge win that can leverage your existing infrastructure. Groovy’s probably the semantically closest, JRuby’s the conceptually cleanest, and Scala keeps the fetishized complexity and the BDSM aspects of Java.

    Currently evaluating FP alternatives on the JVM… this quote is the best
    I’ve found on any blog, tweet or news group post.

    So…I can sell it to all our java developers as a way to get more done without
    taking off the leather or putting away the whips.

    Not sure if Rich Hickey can compete with that kind of allure.

  • http://www.robertcfischer.com Robert Fischer

    I just don’t know enough Clojure to comment: you’d have to ask Brian about that.

  • Christopher E. Stith

    It certainly seems to work for Congress and their aides. There are also the celebrity tabloids, the “Teach Yourself “… “In 24 hours” authors, and whoever’s writing the majority of the morning TV news programs.

    Maybe it’s time someone made a bunch of money off of BS on technical and pseudotechnical babble (I mean besides the aforementioned awful programming tutorials and Slashdot).