Skip to content

The Fall of YAMLScript!

(or Exciting YS News for Fall 2024!)🔗

Greetings!

It's been over 3 months since the last blog post here.

Just to be clear, the YS/YAMLScript project is alive and fantastic!

We've just been busy as hell on 2 very big things: Exercism ⧉ and KubeCon ⧉.

To be successful in both of these endeavors, YS needed to be amazing both as a programming language (Exercism) and as a data language (KubeCon).

There's so much new stuff to talk about, and I promise to write about all of it after things get back to a normal pace.

Today let's talk about Exercism, KubeCon and the positive impacts they've had on YS.

Exercism🔗

When I learned about the "free programming language learning website" called Exercism ⧉ in June, it supported 71 language tracks ⧉. I hoped that YS/YAMLScript could become number 72.

I was in Las Vegas at The Perl and Raku Conference ⧉ where I met Daniel Mita who is a maintainer of the Perl ⧉ and Raku ⧉ tracks on Exercism. Over breakfast, he told me all about the Exercism platform and and it sounded great. I asked him if it would be possible to add a YS track to Exercism.

Some hours later he had started the process to make that happen...

Fast forward to mid September; the YAMLScript track is live as number 74 ⧉!

The YAMLScript track ⧉ currently offers 60 exercises of varying difficulty levels. Try it out and let us know what you think!

How Exercism Made YS Better🔗

To get the YS track launched it was required that at least 20 exercises be set up. We launched with 42, of course!

To set up an exercise for a track, you need to write a program that passes a series of tests that are provided for the exercise. Personally I wanted these YS solutions to be as simple and beautiful as possible.

The main way YS has grown as a language is from actually writing code with it and seeing what works beautifully and what doesn't. If something can be done to make the language better, we do it.

I can confidently say that YS produces some of the most beautiful solutions to the Exercism exercises. In fact I wrote a program ⧉ (in YS) that compares the YS solutions to all the other languages on Exercism as a markdown page and then posts it as a GitHub gist page.

I won't share these gists here, because that would be a spoiler for the Exercism platform. But since all these solutions are publicly available and open source, you can run that program yourself and see the results.

However I did think of a way to do something similar that I can share here...

I wrote a another (nearly identical) program ⧉ that compares YS solutions to other languages for the massive open source code site Rosetta Code ⧉.

Here's a few example gists that compare Rosetta Code YS solutions to those in the Exercism set of languages:

As you can see, YS programs turn out to to be very clean, simple and easy to read.

If you are going to enhance your YAML files with functional logic, it's reassuring to know that YS has the full power of a mature programming language behind it (Clojure), presented in a way that is not only as easy to read as YAML, but is actually 100% valid YAML.

Let's segue to KubeCon, where we focus on YS as a way to tame large YAML configuration files for Kubernetes and other software frameworks.

KubeCon🔗

In early June I proposed a 90 minute YS tutorial for KubeCon and it was accepted! I'll be presenting it a week from this Friday (November 15th in Salt Lake City). If you are going to be at the conference, I hope we get a chance to talk.

The YS needs for Kubernetes ⧉ (and other) config files are very different from its needs for Exercism. Kubernetes is configured with YAML, as are many other software frameworks used by Cloud Native ⧉ programmers.

While YS is a great as a programming language, one of its main goals has always been to fix as many of the problems people have with YAML as possible.

Of course, we are aware that there are already lots of tools that help with managing large YAML files for various applications. YS is not trying to replace those tools, but to augment them.

For instance, many people use Helm ⧉ to manage their Kubernetes. Helm is a complete package manager for Kubernetes, that comes with a Go based templating system.

YS can be used instead of (or in combination with) Helm's templating system.

Let's look at an example of using YS in a Helm chart template.

Here's a snippet of a template file created by helm create mychart:

  {%- raw -%}
  spec:
    {{- with .Values.volumes }}
    volumes:
      {{- toYaml . | nindent 8 }}
    {{- end }}
    {{- with .Values.nodeSelector }}
    nodeSelector:
      {{- toYaml . | nindent 8 }}
    {{- end }}
  {% endraw %}

With YS, you can write this as:

  spec:
    :when+ Values.volumes.?:: {volumes:: _}
    :when+ Values.nodeSelector.?::
     nodeSelector:: _

Not only is the YS version much shorter and easier to read, its also valid YAML. That means it can be maintained using other YAML tools, like yamllint. It also gets rid of the need for the nindent explicit indenting function, which is a common source of errors in Helm templates.

If you want to see an entire Helm chart written converted to YS, we made a repository ⧉ with 2 commits: one with the original Helm chart and one after the conversion.

Or you can simply look at this gist ⧉ which shows the changed files side by side. This gist was also generated by a new YS program called sbs ⧉ (side by side).

How KubeCon Made YS Better🔗

Just because YS is a great programming language, and embeddable in YAML, doesn't mean that it will be a natural fit for every common config file need.

As we reviewed the needs for using YS in Helm charts, we found a couple things that could be improved.

The first was being able to load external data and define variables in a file that is just data.

As you might recall, to use YS in a YAML file, you need to use the special !yamlscript/v0: tag at the top. Note the difference between that tag and the !yamlscript/v0 tag that we need to start a YS program.

The trailing slash means to start off in "data mode" instead of "code mode". This means the the unquoted word foo is a string, not a variable or function.

But say you want to load another file and use its contents as data (without having any effect on the YAML data you are defining in the current file).

Now you can do that (at any lexical level) with a normal YS assignment.

!yamlscript/v0:

default =: load('defaults.yaml')

foo:: default.foo.take(3)
bar:
  more =: load('more.yaml')
  baz:: more.stuff * default:count

The =: pairs are code expressions that define new variables. The first one above applies to the entire file and the second one applies only to the map that contains it.

Another thing we found was that we needed a way to conditionally include key/value pairs in a YAML map.

We added a general :: form that evaluates any code. If the evaluation result is a map, it is merged into the current map at the current position. If it evaluates to nil, nothing is added.

!yamlscript/v0:

foo: 1
::
  when rand(2) > 1::
    bar: 2
baz: 3

This is a very powerful way to conditionally add data using any logic you want.

However its a bit verbose for simple cases. Consider this:

!yamlscript/v0:

data =: load('data.yaml')

foo: 1
::
  when data.key1::
    bar: data.key1
baz: 3

We added a shorthand for this common case:

:when data.key1::
 bar: data.key1

You can collapse the :: and the when function line into a single line.

But notice the glaring redundancy of needing to write data.key1 twice?

We addressed that with the new when+ function.

:when+ data.key1::
 bar: _

It sets the _ variable to the value of its condition expression data.key1.

Finally we can collapse that to a single line by simply using YAML's flow style:

:when+ data.key1.?:: {bar: _}

Conclusion🔗

This Fall, the YS project is in a really good place. It's both a great programming language and a great data language. While it is already extremely capable, we are always looking for ways to make it better.

I'm very excited to present all this next week at KubeCon. And again, I hope to see you there!