> Besides, config languages never have the level of polish (re: tab completion and static analysis) that the main language does.
They don't have to!
> Why deny your users the niceties that you set up for yourself?
Don't do this, of course. :)
For instance, Nickel gained[1] an LSP implementation years before it hit version 1.0. When editing Nickel configs,byou get all the stuff you'd expect for a programming language: syntax highlighting, type hints, autocompletion, etc. But beyond static analysis, Nickel can also validate your configurations dynamically, according to arbitrary predicates-- and that too, integrates with your editor through the language server. Nickel ships with a command-line evaluator, REPL, pretty printer for data structures, code formatter, documentation generator, and CLI tab completions for Bash, zsh, Fish, PowerShell, and Elvish. Upstream distributes the tooling (which is, Go-style, a single executable with subcommands) as a Nix flake, a Docker image, and static executables for Linux aarch64 and x86_64 (< 30 MB, for now at least). Nickel doesn't yet have a package manager[3], but other than that it honestly has a better tooling story than many general-purpose programming languages currently in widespread use.
CUE, an older (but not old) configuration language (very cool in its own right), likewise has great tooling. It ships with a command-line evaluator, a pretty printer, a formatter, package management based on Go modules (i.e., it has both a `mod` and a `get` subcommand), a few refactoring tools, and tab completions for bash, zsh, and fish.[4] It currently lacks a language server[5] but there is recent and ongoing work to enable that.
There's no reason that a configuration language can't have first-rate tooling!
There is a new crop of really innovative, powerful configuration languages out there. I hope they start catching on.
> This way all your users are suitably placed on a slippery slope where one edit in the same language but to another file will transition them from user to contributor.
> Jail them in a separate language and they'll stay users forever.
This, I admit, deeply appeals to me. It's one of the things I really like about the Nix language, wart-y though it may be: the tools you use to configure your NixOS computer are the same ones you can go on to use to extend NixOS. I think that's an enormous strength.
But I also like for simple configuration to look really simple, and not trouble you much with the details of the language (or, God forbid, configuring the runtime). Perhaps programmable configuration languages, then, are good fits for tools like build systems and packaging systems (where the 'upstream' code users might next want to play with is also essentially fancy, expressive configuration in a special domain), but embedded DSLs might be better for applications (and work for packaging and build systems, too).
Re: creating a continuous road for users to hack on the software they use, GNU is really trying to do something special there: with Guix you have application configuration, package management, and system configuration handled in a DSL in Guile Scheme. But then Guix itself is also written in Guile! And at the same time, Guile is the official extension language of the GNU project, so GNU applications all (hopefully?) have Guile bindings, so you can use the same language to plug into (hopefully?) all the other GNU applications on your system. It's a cool idea.
> there's no reason that a configuration language can't have first-rate tooling!
Perhaps I shouldn't have made it about the configuration language's maturity. What I mean is that however mature it is, it's still yet another thing. At least from what I've been seeing lately, fewer tools > better tools.
Suppose that on first run you generate a config file like this:
from configtypes import Strategy
def strategy() -> Strategy:
return Strategies.balanced
And then the user maybe changes it to
from datetime import datetime
from configtypes import Strategy
def strategy() -> Strategy:
# be aggressive on Tuesdays
if datetime.today().weekday() == 2:
return Strategy.aggressive
return Strategy.balanced
But actually it's `Strategy.assertive` not `Strategy.aggressive`, so this is an invalid configuration. I'm sure you can set up an editing experience for the user which gives them a red squiggly when they try to reference a nonexistent value, or which gives them an error that indicates a line number when they try to use it. But those are things you want for your dev environment anyway, right?
And maybe the user is curious: "how is this used?". There are editor actions like "go to definition" or "find references" which they can use to find more information. Those will be interrupted if the language is different.
I just feel like whatever time you might spend on integrating the configuration language is going to be better spent on docs, error messages, or making it easy to summon a well-configured editor environment (which is the same both for devs or config authors).
---
I've experienced what you're describing with nix. It's magical. I haven't toyed with Guix yet, but I want to eventually write helix plugins (it uses a scheme for its plugin language, or it will...), so maybe I'll give Guix a try when I get to that "learn scheme" item on my todo list.
For now I'm focused on how to help users who aren't accustomed to navigating a project with files in more than one language (scientists mostly), and I just don't think the juice is worth the squeeze re: creating more than one world, one for them and one for "proper" devs. Maybe maybe I'm overlooking the use case of large non-user-serviceable things because I wish things in general were smaller and more user-serviceable.
> What I mean is that however mature it is, it's still yet another thing.
That's definitely true. And I didn't have scientific applications in mind as I wrote, either. I was thinking of domains where external DSLs for configuration are already predominant, and also of DSLs that I've enjoyed working with and their particular advantages. But everything you're saying makes perfect sense to me.
> And maybe the user is curious: "how is this used?". There are editor actions like "go to definition" or "find references" which they can use to find more information. Those will be interrupted if the language is different.
Right, the validation errors your config lang hands you might tell you what's wrong, but they won't be Python or R or $PROGLANG errors. It would be extra and special work to somehow get an editor to know how to jump between those two worlds, and I'm not even immediately sure how I'd try to do it if I wanted to.
> For now I'm focused on how to help users who aren't accustomed to navigating a project with files in more than one language (scientists mostly), and I just don't think the juice is worth the squeeze re: creating more than one world, one for them and one for "proper" devs.
I have to admit that I think you might be right, partial as I am to some fancy new config langs. But on some level it's already the case that those two separate worlds exist, right? Polyglot projects and companies are super common in software.
> that "learn scheme" item on my todo list.
I'm doing this right now in a real casual way with some friends! We're reading a chunk of The Little Schemer every other week. The book is easy, and that pace is gentle. If you're interested in joining or following along, feel free to DM me on NixOS Discourse.
I think that we're each focusing on the right thing for the domains that we're thinking of, they're just different domains. When I first heard about Nickel I was like "wow, I'm gonna use this everywhere"--because I too get excited about fancy new languages--and I think that was premature. I'm looking forward to when the opportunity arises, but I don't think it'll become everywhere.
Config languages are definitely the way to go for system config and polyglot projects and maybe also places where the config/project divide is natural in some way such that users reasonably can spend their whole life without writing only config code (kubernetes comes to mind). I just tend to lean somewhat radically away from those things... to such an extreme that I've been thinking about making the code editor (and all its various configs) a project-level dependency instead of a system config.
I know it sounds crazy, but If you bundle the tools needed to work on a project into that project, then wherever it ends up, the user has what they need to work on it, even if the internet is down: Little dev-focused operating systems bundled with each package so that the actual operating system needs to be interacted with hardly at all. Something something right to repair...
---
Re: The Little Schemer, I actually have a copy of that on my desk, I have spent less than 5 minutes on it. I'll find you on NixOS Discourse :)
> I've been thinking about making the code editor (and all its various configs) a project-level dependency instead of a system config.
> I know it sounds crazy, but If you bundle the tools needed to work on a project into that project, then wherever it ends up, the user has what they need to work on it, even if the internet is down: Little dev-focused operating systems bundled with each package so that the actual operating system needs to be interacted with hardly at all.
I think it's totally sensible. Presumably some people will always like to bring their own editor, and for that, having direnv supply an language server, compiler or interpreter, etc., plus including a generic editorconfig file is nice. But why not go all the way and provide ready-to-use, pre-configured VSCodium and Neovim?
I'm planning on putting together something like that for my roommate who is a non-programmer but interested in learning to code. Right now he's working through Learn Enough Developer Tools to Be Dangerous to get some Linux and CLI basics down, then we're gonna do Linux From Scratch together and some programming (starting with The Little Schemer and hopefully ending with SICP). I'm making Nix environments for all the programming books I'm currently reading, and the Scheme one will bundle Emacs and DrRacket.
I was also motivated in this direction by some friends I want to teach to code. There's so much for a newbie to overcome that has nothing to do with code.
What I'm struggling with right now is:
How do you make supplying your own vim, say, versus the one that's default for this project... How do you make that feel like supplying a nondefault "Editor" type argument to a function? I'm aware that flake inputs support that sort of trickery via input overrides from the CLI, but there's nothing about that (besides perhaps the name of the input) which ensures that you only put editors in that slot. And then correspondingly, editors are a bit too varied to be interchangable like that. If a project defaults to VSCodium and I want to inject helix... that's like gonna take some hackery which prevents it from feeling like dependency-injected magic.
I hadn't planned on something so neat. I was just thinking I'd put them on the path with nonstandard executable names like `pvim` or `pcode`, and then people could use them in combination or not, individually enable-able as devenv modules. One of them could be aliased to `edit` or something like that if configured as the default editor, though.
and users would set or modify that in devenv.local.nix¹.
And the editors module would handle constructing the customized editor packages from the specified base packages, lists of plugins/extensions, included config, etc., as well as wrapping them so that they are called by the names `pvim`, `pcode`, `edit`, etc., and pointing them to gitignored RC files that live in-repo which users can edit.
They don't have to!
> Why deny your users the niceties that you set up for yourself?
Don't do this, of course. :)
For instance, Nickel gained[1] an LSP implementation years before it hit version 1.0. When editing Nickel configs,byou get all the stuff you'd expect for a programming language: syntax highlighting, type hints, autocompletion, etc. But beyond static analysis, Nickel can also validate your configurations dynamically, according to arbitrary predicates-- and that too, integrates with your editor through the language server. Nickel ships with a command-line evaluator, REPL, pretty printer for data structures, code formatter, documentation generator, and CLI tab completions for Bash, zsh, Fish, PowerShell, and Elvish. Upstream distributes the tooling (which is, Go-style, a single executable with subcommands) as a Nix flake, a Docker image, and static executables for Linux aarch64 and x86_64 (< 30 MB, for now at least). Nickel doesn't yet have a package manager[3], but other than that it honestly has a better tooling story than many general-purpose programming languages currently in widespread use.
CUE, an older (but not old) configuration language (very cool in its own right), likewise has great tooling. It ships with a command-line evaluator, a pretty printer, a formatter, package management based on Go modules (i.e., it has both a `mod` and a `get` subcommand), a few refactoring tools, and tab completions for bash, zsh, and fish.[4] It currently lacks a language server[5] but there is recent and ongoing work to enable that.
There's no reason that a configuration language can't have first-rate tooling!
There is a new crop of really innovative, powerful configuration languages out there. I hope they start catching on.
> This way all your users are suitably placed on a slippery slope where one edit in the same language but to another file will transition them from user to contributor.
> Jail them in a separate language and they'll stay users forever.
This, I admit, deeply appeals to me. It's one of the things I really like about the Nix language, wart-y though it may be: the tools you use to configure your NixOS computer are the same ones you can go on to use to extend NixOS. I think that's an enormous strength.
But I also like for simple configuration to look really simple, and not trouble you much with the details of the language (or, God forbid, configuring the runtime). Perhaps programmable configuration languages, then, are good fits for tools like build systems and packaging systems (where the 'upstream' code users might next want to play with is also essentially fancy, expressive configuration in a special domain), but embedded DSLs might be better for applications (and work for packaging and build systems, too).
Re: creating a continuous road for users to hack on the software they use, GNU is really trying to do something special there: with Guix you have application configuration, package management, and system configuration handled in a DSL in Guile Scheme. But then Guix itself is also written in Guile! And at the same time, Guile is the official extension language of the GNU project, so GNU applications all (hopefully?) have Guile bindings, so you can use the same language to plug into (hopefully?) all the other GNU applications on your system. It's a cool idea.
--
1: https://github.com/tweag/nickel/pull/405
2: https://www.tweag.io/blog/2024-05-16-nickel-programmable-lsp...
3: https://github.com/tweag/nickel/issues/1585
4: https://cuelang.org/docs/reference/command/
5: https://github.com/cue-lang/cue/issues/142