Apple Releases Pkl, a Configuration-as-Code Language (pkl-lang.org) 73
Apple, in a blog post: We are delighted to announce the open source first release of Pkl (pronounced Pickle), a programming language for producing configuration. When thinking about configuration, it is common to think of static languages like JSON, YAML, or Property Lists. While these languages have their own merits, they tend to fall short when configuration grows in complexity. For example, their lack of expressivity means that code often gets repeated. Additionally, it can be easy to make configuration errors, because these formats do not provide any validation of their own. To address these shortcomings, sometimes formats get enhanced by ancillary tools that add special logic. For example, perhaps there's a need to make code more DRY, so a special property is introduced that understands how to resolve references, and merge objects together. Alternatively, there's a need to guard against validation errors, so some new way is created to validate a configuration value against an expected type. Before long, these formats almost become programming languages, but ones that are hard to understand and hard to write.
On the other end of the spectrum, a general-purpose language might be used instead. Languages like Kotlin, Ruby, or JavaScript become the basis for DSLs that generate configuration data. While these languages are tremendously powerful, they can be awkward to use for describing configuration, because they are not oriented around defining and validating data. Additionally, these DSLs tend to be tied to their own ecosystems. It is a hard sell to use a Kotlin DSL as the configuration layer for an application written in Go. We created Pkl because we think that configuration is best expressed as a blend between a static language and a general-purpose programming language. We want to take the best of both worlds; to provide a language that is declarative and simple to read and write, but enhanced with capabilities borrowed from general-purpose languages. When writing Pkl, you are able to use the language features you'd expect, like classes, functions, conditionals, and loops. You can build abstraction layers, and share code by creating packages and publishing them. Most importantly, you can use Pkl to meet many different types of configuration needs. It can be used to produce static configuration files in any format, or be embedded as a library into another application runtime.
We designed Pkl with three overarching goals:
To provide safety by catching validation errors before deployment.
To scale from simple to complex use-cases.
To be a joy to write, with our best-in-class IDE integrations.
On the other end of the spectrum, a general-purpose language might be used instead. Languages like Kotlin, Ruby, or JavaScript become the basis for DSLs that generate configuration data. While these languages are tremendously powerful, they can be awkward to use for describing configuration, because they are not oriented around defining and validating data. Additionally, these DSLs tend to be tied to their own ecosystems. It is a hard sell to use a Kotlin DSL as the configuration layer for an application written in Go. We created Pkl because we think that configuration is best expressed as a blend between a static language and a general-purpose programming language. We want to take the best of both worlds; to provide a language that is declarative and simple to read and write, but enhanced with capabilities borrowed from general-purpose languages. When writing Pkl, you are able to use the language features you'd expect, like classes, functions, conditionals, and loops. You can build abstraction layers, and share code by creating packages and publishing them. Most importantly, you can use Pkl to meet many different types of configuration needs. It can be used to produce static configuration files in any format, or be embedded as a library into another application runtime.
We designed Pkl with three overarching goals:
To provide safety by catching validation errors before deployment.
To scale from simple to complex use-cases.
To be a joy to write, with our best-in-class IDE integrations.
Looks modestly useful (Score:2)
It seems like it would have a reasonable number of specialized use cases. But It's not something I've wanted to do.
This is for things like having a common base file that you can emit as JSON or YAML or ...
We desperately need SOMETHING like this (Score:5, Interesting)
For all the "innovation" in programming languages, what we need is a new data definition language. JSON is dogshit...like absolute DOGSHIT. Why? No comments, schemas are an afterthought, and there's no way to determine the intent of a JSON payload....foo.json...what is it?...who knows...you figure it out. There's no way to bind a class or schema to it.
XML? Honestly, XML with attributes is fine...it's just a bunch of idiots, mostly IBM-ers wrote the world's worst XML...as if by a dare and it became popular among many early 2000s frameworks and now everyone HATES XML because they saw it designed by the world's most sadistic idiots. Also, it's fair to say with with namespaces and multiple schema inheritance it was over-engineered....so yeah, we went from a sadistic over-complicated format to one too simple to be useful.
Honestly, most programming languages do a better job than XML or JSON. Groovy is a great example. Pkl seems OK. I don't like giving too much flow control because it's a security vulnerability waiting to happen, but we'll see...maybe they nerfed it enough.
I want constructors, factory methods, and strong typing....and minimal syntax. From that tiny post, it seems to accomplish those. The devil is in the details, though....most likely, if this gets popular, some idiots will make it as painful as XML got to be.
Re: We desperately need SOMETHING like this (Score:2)
Re:We desperately need SOMETHING like this (Score:5, Funny)
Honestly, most programming languages do a better job than XML or JSON.
you have to admire the audacity of our progress: first we moved configs out from the language to data stores because they shouldn't be there, then after struggling for years with ddls, specs, document models and validators we come to the realization that we need to create ... a new language to handle configs.
Re: (Score:3)
I don't like giving too much flow control because it's a security vulnerability
Yes, but a more risky thing:
read("env:DATABASE_PASSWORD")
They have hooks to reach outside the configuration to get data, which has historically a gigantic security nightmare, and at this point I suspect would be an absolute non-starter for such an initiative.
For data definition, I suppose you have Protobuf, though it's in binary on disk, but if you really really want schemad data definition.
However, I would suggest there are plenty of scenarios where a schema-less strategy is just fine. If you absolutely *must* have schema in such a f
Re: (Score:2)
I think the real problem is XML just started to be used for like EVEYTHING. Hey its tree like, lets use it for widget configs! Hey it has inheritance, so l
Re: (Score:2)
That is to have a strict schema so programs don't just crash when loaded with a bad configuration?
If that was the goal the way people implemented it served exactly the opposite. A typical key/value pair .conf with some mis-typed parameters typically would throw a warning or two and fill in some defaults for anything you missed. Programs which used XML based configs on the other hand were pretty much guaranteed to stack dump on an unhandled exception. It was a running joke about Apache/Tomcat servers that they'd always be down because someone put a & in a field somewhere and now the XML interchange f
Re: (Score:1)
Re: (Score:2)
Honestly, most programming languages do a better job than XML or JSON. Groovy is a great example.
YAML is decent enough. Also, Python and Ruby are excellent at creating configuration DSLs (I've used Python myself for that purpose.)
You are right regarding XML and JSON (though I always prefer JSON to XML when I'm forced to make a choice.)
Json crumbles under serious work (Score:3)
You are right regarding XML and JSON (though I always prefer JSON to XML when I'm forced to make a choice.)
JSON was designed as a means of transferring state between two running systems who know each other well. For that, it's well-suited. The problem is we turned into things it was never designed for.
For example, any system that uses JSON over time. Eventually, you'll need versioning. Eventually, you'll run into data and not be certain which version it was. There's no meta-declaration, like there is with XML. The 2 fatal flaws of JSON is it can't support comments and there's no way to identify what it i
Re: (Score:3)
The core problem with XML is that it is too complicated to use. The core problem with JSON is that it is not complicated enough to be usable. This isn't a coincidence...
Re: (Score:1)
The 2 fatal flaws of JSON is it can't support comments and there's no way to identify what it is.
That is nonsense. You just put a
ignore_this_comment : "this is a silly comment"
Into where ever you want it.
Some parsers fail on unknown keys (Score:2)
The 2 fatal flaws of JSON is it can't support comments and there's no way to identify what it is. That is nonsense. You just put a ignore_this_comment : "this is a silly comment" Into where ever you want it.
I hope you set your Jackson settings correctly...or else that will throw an error unless your schema is designed to handle it. Some parsers fail on unknown keys. What if you want to comment out a section to test one property in isolation? You're coming up with creative hacks to get around a feature that every serious tool has and this one tool omitted for one reason or another.
Re: (Score:1)
Well,
if you want to be a pro, then you commit your file to a local branch.
DELETE the damn stuff you want to comment out. And then continue.
After you figured the problem you revert the commit or check the previous version out.
In our times a comment should be what it is: a comment about what is going on. Not hidden code where no one really knows why it is commented out.
counterpoint: Optional parameters (Score:2)
Well, if you want to be a pro, then you commit your file to a local branch. DELETE the damn stuff you want to comment out. And then continue. After you figured the problem you revert the commit or check the previous version out.
In our times a comment should be what it is: a comment about what is going on. Not hidden code where no one really knows why it is commented out.
That's pretty macho, but not very understanding of a typical work environment. Take a config example.
//"debugMode" : true //enable this in your IDE to see more details in your stacktrace. Do not enable in production as it will make the system log too much noise and slowdown the user experience and make it harder to find actual bugs.
{
}
Do you have coworkers? Do yours actually read your docs? Mine don't. Half of them barely speak English or started recently or there's a bunch in different contine
Re: (Score:1)
With git you always can locally make a local commit. No coworker sees it.
Re: (Score:1)
I think modern development should rely more on proper usage of version control and branches.
It does not make sense anymore that a config file contains a kind of IF, that shell scripts have IFs depending on architecture, platform or OS. Same for make files or #ifdef's in source code.
The dart/flutter people use the concept of streams. One is basically a release branch, two more are alpha and beta development streams.
Re: Json crumbles under serious work (Score:2)
I had the same idea once, but then you need to account for that in whatever is reading and parsing the JSON. This gets extra fun in nested structures where you may have written recursion to deal with them, and now you have to explicitly branch on that key when encountered.
It's not the biggest problem, but it's still a problem that makes for shitty code instead of a shitty data file.
Re: (Score:2)
When you get foo.json...was it from this year?...last year?...is it referring to the accounting system's foo?...procurement's foo?...sales foo?...is it even a foo?
Ask Dr. Seuss.
Re: (Score:1)
No comments, schemas are an afterthought, and there's no way to determine the intent of a JSON payload....foo.json...what is it?...who knows...you figure it out. There's no way to bind a class or schema to it.
Plain wrong.
Obviously you can bind a schema and/or class(es) to it.
That is the whole point of JSON.
Re: (Score:2)
Ansible user here, and a happy one too. What problem does Pkl solve that Ansible doesn't?
https://en.wikipedia.org/wiki/... [wikipedia.org]
Re: (Score:2)
JSON's lack of support for NaN, Inf, and -Inf in numeric fields is also a major problem.
Python pickle (Score:3, Informative)
Letâ(TM)s just ignore the naming collision and go with pickle anywayâ¦
Re: Python pickle (Score:1)
I have no idea why the spelling of my post was mangled in such a brutal manner!
Re: Python pickle (Score:4, Insightful)
You posted from an iDevice that wants to send unicode to a textarea, and further, likes to go for 'weird' unicode for certain punctuation despite that punctuation having a much more universal ASCII variant.
Re: (Score:2, Insightful)
You posted from an iDevice that wants to send unicode to a textarea, and further, likes to go for 'weird' unicode for certain punctuation despite that punctuation having a much more universal ASCII variant.
And which EVERY other Site on the Internet handles with aplomb.
But let's all just blame Apple for using a STANDARD as intended. . .
Re: (Score:2, Insightful)
Well, it's true that slashdot should be able to handle unicode.
However, it's also a bit ridiculous for "Smart Punctuation" to send nitpicky variants of various punctuation instead of the ASCII versions.
Re: (Score:2)
Well, it's true that slashdot should be able to handle unicode.
However, it's also a bit ridiculous for "Smart Punctuation" to send nitpicky variants of various punctuation instead of the ASCII versions.
The rest of the Internet handles it; so I'd say Slashdot is the clear Outlier, not Apple.
Especially ridiculous, since Slashdot's readership us clearly multinational in scope.
Re: (Score:3)
I’ve made the point a lot. Go find another website with this same bug as slashdot. You won’t. Slashdot is the only site on the entire internet with this problem.
Re: (Score:2)
I’ve made the point a lot. Go find another website with this same bug as slashdot. You won’t. Slashdot is the only site on the entire internet with this problem.
But they insist:
1. That it does recognize certain UNICODE.
2. It is "working as intended".
So, whatcha gonna do?
Re: (Score:1)
Re: (Score:3)
At this point write a damn browser plugin to fix it.
Re: (Score:2)
I’ve made the point a lot. Go find another website with this same bug as slashdot. You won’t. Slashdot is the only site on the entire internet with this problem.
Indeed. Yet Slashdot is one of the few sites that's actually immune from having an unreadable fucked up comment section ruined by trolls. If you want to support Unicode then to support it properly opens you up for a world of hurt from trolls using the various modifiers to change how your text is presented. So you either support standards, or you don't. I'm in favour of Slashdot actually supporting standards. Maybe Apple can learn that the apostrophe has a perfectly fine ' character they should be using, jus
Re: (Score:1)
Re: (Score:2)
And which EVERY other Site on the Internet handles with aplomb.
But let's all just blame Apple for using a STANDARD as intended. . .
Yes let's blame Apple. See that apostrophe that I used there? It was used from an actual standard compliant browser. And no there are plenty of examples across the web which don't support unicode, and even more that support unicode which then get horrendously fucked in the process of supporting unicode, my favourite troll character being U+200F the right-to-left modifier.
Oh you don't want to support U+200F? That makes you non-standards compliant. You do want to support it, expect every idiot troll to make y
Re: (Score:2)
Re: Python pickle (Score:2)
Because you are using an Apple device with "smart quotes" turned on which emits Unicode curly quotes, and Slashdot doesn't support Unicode. See: signature.
Re: (Score:2)
PHP also has PECL. Who the hell wants their product to be even slightly confused with PHP?
Unfortunately... (Score:4, Informative)
It's a reinvention of Ansible without system configuration management because NIH.
While it's good for application-internal configuration management, it doesn't go nearly far enough.
And it doesn't support TOML, INI, or conf files.
This is some corporate FOSS-washed project that isn't all that useful.
Re: (Score:3)
It doesn't go far enough in targeting something like ansible, with actual facilities to do meaningful stuff rather than just abstract.
But it goes too far in letting a mere 'config file' do things that most security folks agree is not things a 'config file' should be able to do.
Re: (Score:2)
It's a reinvention of Ansible without system configuration management because NIH.
While it's good for application-internal configuration management, it doesn't go nearly far enough.
And it doesn't support TOML, INI, or conf files.
This is some corporate FOSS-washed project that isn't all that useful.
So don't use it. We won't mind.
Re: (Score:2)
Re: (Score:2)
I don't use lots of things - eg. nodejs, swift, and this. Doesn't stop someone else thinking it's the greatest idea in the world and then lumping me with it to run on their production network for them.
As an easier example, some apps use a Json config file - after all, it makes it easier for the developers because either the config is valid or it's not - and if it's not, they don't have to do much - they can just exit() and they're good. The thing is, for us sysadmins, having to template out Json is tedious
Re: (Score:2)
I don't use lots of things - eg. nodejs, swift, and this. Doesn't stop someone else thinking it's the greatest idea in the world and then lumping me with it to run on their production network for them.
As an easier example, some apps use a Json config file - after all, it makes it easier for the developers because either the config is valid or it's not - and if it's not, they don't have to do much - they can just exit() and they're good. The thing is, for us sysadmins, having to template out Json is tedious - not impossible of course, but tedious because you need to put commas in some places and not others, you've got to close braces etc. Even just using Yaml would be simpler for the sysadmin, but for some reason, oh no - it's gotta be json. Very much a "first world problem", but it feels like an entirely avoidable one.
I've had the misfortune to have to deal with, and manually bugfix(!!!) some Json code and traffic. Not pleasant; but manageable, I guess. . .
I haven't messed with Swift yet.
Re: (Score:2)
I thought the same. The example of sucking up an environment variable just isn't needed - whatever that secret is, Ansible already handles it. The same with 'profile' stanzas - Ansible just does a for...next to burp out all you need as many times as its needed.
My recent experience suggests very few people "get" modern sysadmin. They still think 20 years out of date where you log on to each server and edit the configs by hand - which we just don't do any more. Sadly, people think they know it all, and so go
Re: Unfortunately... (Score:2)
Reading the description, it reminds me of AWS CloudFormation where you can have inline functions in a JSON file.
And then I had a PTSD flashback of working in CloudFormation templates, which was followed by curling into the fetal position below my desk.
I see no big need for this, unless you are just really hell-bent on not using ansible, chef, terraform, kubernetes+kustomize, or half a dozen other viable solutions that already exist.
Uh oh... (Score:2)
password = read("env:DATABASE_PASSWORD")
So here we hit the problem of so many configuration languages: you *want* features but once you have a fully featured configuration language that someone uses in a data exchange format, trouble happens in security.
So while it might be *nice* to have logic and capabilities in your configuration language, it's also *risky*. So long as it was scoped to only be 'programmer level access, not actual final configuration', but they lead with an example of using it as a config file.
Also, bold choice to use the name
Can someone break this down for me? (Score:2)
I read the blog post but I obviously lack the prerequisites here as I am not a developer, so can someone point out what is meant by "configuration" in this context? I assume it's not configuration of your systems hardware? Is it configuration of the language? The IDE? I always tell me people no such thing as a stupid question but I sure feel it now.
Re: (Score:2)
Like 'ini' files, and some people have taken up yaml as a more formally defined language that can look like a config file, and of course 'toml' for the less maddening whitespace and more like 'ini' but with a formal grammar to describe it.
Apple here is trying to 'code up' a config file, with more comprehensive support for describing what keys are allowed, what types of data the key should have, and constraints on the values baked into the specification. This can be nice and good, though I'm not sure this i
Re: (Score:2)
Thank you, .ini files makes perfect sense and it's easy to forget that something as simple looking as an .ini file is in fact a language unto itself, just usually don't consider it that.
Followup dumb question,
If this is "configuration as code" does it get compiled as code? Or are they just describing the fact that it follows the form of writing code?
Re: (Score:3)
It's not getting compiled, it's just "configuration file that can upgrade to 'code' by virtue of having things like if statements and for loops"
Analogous to a 'batch file' or 'shell script'. In it's most basic form, just a series of commands. But then you start throwing in 'code' features to do things like make one thing depend on another, or to do the same thing many times with one little change.
Re: (Score:2)
Thank you, I really appreciate it.
Re: (Score:2)
This sounds horrible. Can somebody explain what actual use this thing has to cause the need to create yet another complexity? is this a replacement for an installer shell script?
So a configuration file now needs an interpreter instead of just a parser so conversion to another kind is now nearly impossible?
Re: Can someone break this down for me? (Score:2)
nothing new here. (Score:2)
Did EU approve before release? (Score:1)
Oh wait, sorry, wrong article. Never mind.
don't want to be in a pickle (Score:2)
The idiom "in a pickle" means to be in a difficult or challenging situation, often one that is hard to get out of. When you find yourself "in a pickle," you are dealing with a problematic or awkward circumstance.
Comparison with other languages (Score:3)
https://pkl-lang.org/main/curr... [pkl-lang.org] JSON, YAML, XML and Jsonnet (https://jsonnet.org/), HCL (https://github.com/hashicorp/hcl), and Dhall (https://dhall-lang.org/),
And again
https://github.com/apple/pkl/d... [github.com]
with Dhall, Nickel (https://nickel-lang.org/), jsonnet, CUE (https://cuelang.org/)
Obligatory XKCD (Score:2)
Obligatory XKCD [xkcd.com]
Rust is better (Score:1)
Rust solves all of coding's problems forever and is the easiest language to learn and fastest to deploy.
No other language has been as good as Rust, will be as good as Rust was yesterday, or will be better than Rust tomorrow.
Apple's pickle can only cause plain iron to rust even more. That's what briny juice does, you know, it contributes to the rust ecosystem.
Rusty Fanatic
Looks like Python's Pydantick knock-off (Score:2)
There is already solution for such case - it is called Pydantic.
And it looks quite similar to what they are showing...
Re: (Score:2)
The "Pydantic" looks neat.
The Apple's thingy, in a nut-shell, is a preprocessor for configuration. I did a lot of similar (handcoded) stuff using Ruby & Python too. (First & last time I touched the Ruby.)
Some geniuses also used PHP for the similar effect. In the end, it's all about how deep you want to go to validate the data, and how custom is your custom data model and types.
... But every time I touched the stuff in the past, the thought always crossed my mind: why the heck XML/XSD/XSLT had to be
They just need to go one step further (Score:4, Interesting)
All file formats become programming languages. They just need to go one step further [slashdot.org].
Can you image Apple devices? (Score:2)
Niche? (Score:2)
This sounds like NIH basically?
Heck, I have one puppet class with an embedded puppet template that gets facts from a ruby module and writes out a Lua config file - some kind of window manager setup IIRC.
Seems like we have a few options. That's OK, once in a while an oddball language idea gets legs. Most often they evaporate.
A configuration language should be simple (Score:2)
just use Perl (Score:2)
Perl is ideal for creating configuration. It has many language features designed for processing text. It has tons of shorthand variables that allow you to do complex things with very little code. Its use of character prefixes for variable types allow you to easily intersperse variables in data. And it's designed to have multiple ways to solve problems, so it's very flexible. It's also not the most complex language so it's relatively easy to learn. You can also embed the interpreter in another program.
Pki eally? (Score:2)