I was excited that Pyret, which I was previously familiar with as a Racket #lang, had evidently gained enough independence to warrant its own website that does not even mention Racket. What a vindication of the #lang system! Then I read this:
"Ultimately, Racket’s #lang facilities, though designed to create new languages—and a great prototyping ground for Pyret—proved to not be quite enough to support a language creation process of the scale of Pyret"
I find this rather sad. The PLT group, dogfooding their flagship product's unique selling point, was not able to achieve their goals and switched to Javascript. I hope that one day it may once again be a Racket #lang.
I vaguely get that the idea is an extension of the Unix "shebang" convention to files which define modules. It might be preferable to a system based on file extensions or file associations (it's certainly less brittle). I guess the question is whether it's better than directives of the form "import(lang, file)", in other words, explicitly naming the interpretation language for each file that's imported. Naturally one could set a default so that a plain "import(file)" would do the right thing. A project full of files in extremely heterogenous languages would make make #lang more appealing, but it would also be possible to build some kind of file association system within Racket—personally I would prefer to put the #lang info in the filename or the path than stick it ("intrusively") into the file.
I think they were using "#lang system" as a synecdoche[0] for "Racket's language support". Racket's USP is definitely that it's a platform for language design and implementation[1].
On the other hand, I don't think it's a big deal they moved off of Racket. I actually think it's a success story that they were able to build on the platform and then move off.
#lang is much more than shebangs. Racket #langs are implemented on top of Racket - they can be nothing more than alternative syntaxes, or they can have different semantics (written in Racket). This has the effect of rendering different Racket #langs interoperable within a project. This is wildly powerful - someone reimplemented Python entirely as a #lang, for instance, which meant that Racket could use (pure) Python libraries directly, and vice versa.
It's an entirely different philosophy, which aims to allow to use "the best tool for the job" even within a project. Need a DSL? Write one!
I understand all that. You don't seem to have understood my comment, which was asking questions about the suitability/fitness of the "#lang" mechanism, rather than questioning the idea of polyglot code on top of Racket.
I guess it seems to me like it hasn't been established whether the "#lang" mechanism is a brilliant invention, or something cheesy. My initial reaction is that it seems cheesy.
I'm not sure I'm following, then. You think the basic idea of polyglot programming on top of Racket is sound; but that's what #lang does, and is the interesting bit of what we're talking about. The fact that it's specified with a line at the top of the file is a boring implementation detail - albeit the correct decision in my view, since the information about what language a file is written in clearly belongs with that file - exactly the same reason we have shebangs. But it seems like a minor thing to pick on to deem it "not Racket's USP".
If I understand correctly, the main problem was that the Pyret project wanted Pyret to run directly in the browser, so they needed a JavaScript-backend. Since Racket doesn't have a JavaScript-backend, they wrote a compiler in Pyret that compiles Pyret to JavaScript.
I don't see the lack of a JavaScript-backend as failure of the language building facilities of Racket. In fact, the ability to quickly build a new language in Racket, made it possible to try out different features quickly. Also, it made bootstrapping possible.
However, they don't have to turn around and slag it in their documentation, which seems like a dick move.
Switching to Javascript has nothing to do with "scaling". They wanted to go to JS, and simply jettisoned their bootstrapping booster rocket once they got there, because they didn't want Pyret users (including maintainers) to have a dependency on a Racket installation that is otherwise irrelevant to the running code.
"Ultimately, Racket’s #lang facilities, though designed to create new languages—and a great prototyping ground for Pyret—proved to not be quite enough to support a language creation process of the scale of Pyret"
I find this rather sad. The PLT group, dogfooding their flagship product's unique selling point, was not able to achieve their goals and switched to Javascript. I hope that one day it may once again be a Racket #lang.