YS Mode Switching

I've mentioned YS "modes" in passing several times in this series.

YS has 3 modes: data, code, and bare.

Fully understanding modes is one of the most important things to understand about YS.

Today I want to go deeper on the details of modes. This will make everything else much easier to explain going forward.

Before Modes... Nodes

We need to cover a bit of "YAML Vocabulary" first ...

When talking about YAML and YS we use specific terms like "mapping" that may go by other names in other programming languages and contexts. Each quoted term below is an official YAML vocabulary term and I'll use them consistently in all my YAML and YS writings. The YAML specific terms here all come from the official YAML 1.2 specification  .

YAML has a fundamental concept called "nodes", and it has exactly 3 "kinds" of them:

• "scalar" - A single value like a string, number, boolean or null
• "sequence" - An ordered list of 0 or more nodes
• "mapping" - A set of 0 or more key-node/value-node pairs

When we talk about a YAML "node", we are talking about one of these 3 kinds.

Let's call out a few more YAML vocabulary terms while we're at it:

• "stream" - A complete YAML text or file is called a stream
• "document" - A top level node in a stream is called a document. A stream can contain 0 or more documents.
• "tag" - Every YAML node has a tag, either found explicitly in the YAML text using the ! prefix or implicitly resolved by a YAML "loader".
• "anchor" - A YAML node can be assigned an anchor name with the & prefix.
• "alias" - A named reference to an anchored node using the * prefix.
• "loader" - An implementation of the process of turning YAML text into an application's desired data structure. A loader is a stack of "parser", "composer", "resolver" and "constructor" components. We don't ever say "parser" or "parse" when we mean "loader" or "load".

The YS Compiler is actually a YAML Loader

The YS compiler is a special YAML loader that has a layered stack of transformations (including all the ones above), but instead of creating a mapping or a sequence node, it it creates a Clojure AST.

Back to Modes

Whenever you are looking at a node in a YAML file that is YS enabled, you need to know if that node is plain data, or a code expression.

If you see a line in a file like this:

say: Hello

Is that a piece of data like {"say": "Hello"} or a Lisp code expression like (say "Hello") or (say Hello)? In the last snippet, the Hello is a variable symbol.

The answer is "it depends"!

It depends on the mode of each node!

Starting from the Top

One of the important concepts in YS is that every existing YAML config file out there is a valid YS. When a YS loader loads that file it should produce the same result as your regular YAML loader.

To use YS code anywhere in a YAML file, you need to explicitly enable it by tagging a document node with a special YS tag, like !YS-v0. Without that tag somewhere in the stream, every document will be in "bare" mode.

Bare mode just means that the document node and every node within it is just a plain YAML data node.

If we tag a document with !YS-v0 then the document node is in "code mode". That means every node within it is part of a code expression, unless you "switch" it to "data mode".

If we tag a document with !YS-v0: then the document node is in "data mode". That means every node within it is just plain data, unless you "switch" it to "code mode".

Switching Modes

The way we switch between code mode and data mode is by using the ! tag. That's just a single ! character with no word following it.

Let's see an example:

!YS-v0  # Start this document in code mode

say: "Hello"  # The key and value are both code
say: ! Hello  # The value node of this pair is data (a string)
say: Hello    # Both code. Hello is a variable
say: !
  say: Hello  # This mapping and all its nodes are data
say: !
  say: ! Hello  # The mapping is data but Hello is code

When using YS in YAML config files you end up switching modes a lot. Having all these ! tags in your config file is a bit ugly.

YS has a cleaner way to switch modes. Wherever you see : ! as in foo: ! bar, you can say foo:: bar instead. They mean exactly the same thing. The compiler turns :: into : ! internally.

Let's see the above using the :: syntax:

!YS-v0
say: "Hello"
say:: Hello
say: Hello
say::
  say: Hello
say::
  say:: Hello  # The mapping is data but Hello is code

Document Tags

Every document in a stream is bare mode by default. Even if you've enabled data or code mode on a previous document. IOW, you need to turn data or code mode on explicitly for every document.

The 2 tags for doing this are !YS-v0 and !YS-v0: for code mode and data mode respectively.

However if you have more than one code mode or data mode document in a stream, you only need to use those tags on the first one.

After that you can use the alternate tags !code and !data instead.

Example file:

--- # Bare mode document
a: b
--- # Another bare mode document
c: d
--- !YS-v0  # Code mode document
e: f
--- # Another bare mode document
g: h
--- !code  # Code mode document
i: j
--- # Another bare mode document
g: h
--- !data  # Data mode document
k: l

Remember when loading a file like this YS will return the evaluation of the last document in the stream.

Code Mode Nodes

YS code can be fully expressed using "block mappings", and scalars (plain, quoted or literal).

More YAML Vocabulary

• A "block mapping" is the normal style using indentation for scope.
• A "block sequence" is the normal style using - for each item.
• "flow" mappings and sequences use the {} and [] syntax like JSON.
• A "plain" scalar is unquoted
• A quoted scalar uses double quotes or single quotes
• A "literal" scalar uses a | character and no quotes
• A "folded" scalar uses a > character and no quotes
• All scalars can be multiline

Block sequences, flow mappings and flow sequences can never be in code mode. The benefit of this is that whenever you see one of these styles, you'll know you are not in code mode.

However, this is where things get interesting. YS code mode expressions (parsed from plain scalars)also uses {}, [], "" and '' syntax for mappings, sequences and strings.

Let's look at some examples:

!YS-v0  # code mode
a =: [1, 2]  # Error, flow sequence in code mode
b =: {a: 1, b: 2}  # Error, flow mapping in code mode
c =: "Hello" + ", World"  # Error, invalid YAML. Content after ending quote
d =:  # Error, block sequence in code mode
- e
- f

Here's how we fix those errors:

!YS-v0
a =: + [1 2]  # Make these into plain scalars (for YS expressions)
b =: + {a 1, b 2}
c =: + "Hello" + ", World"
d =::  # Switch to data mode
- e
- f

When a YAML node doesn't start with a YAML syntax character, it's a scalar.

The + in front of a YAML syntax character tells YS that this scalar is an an expression and the + should be ignored before parsing the rest of it.

It's a YS escape character.

Note how I removed the , in [1 2] and the colons from {a 1, b 2}.

Commas are whitespace in YS code (but they are nice to use when separating mapping pairs).

Tag Function Calls

Yesterday we saw how to use !:<fn> to call a function.

Let's take a closer look.

- foo: !:bar
  - a
  - b

Here we see - so we know those nodes are in data mode.

That means that bar is a (code) function that expects a sequence argument. So we snuck in a function call without needing to switch to code mode and back again like this:

- foo::
    bar::
    - a
    - b

The tag call is much nicer.

But what if we wanted to have the argument for bar be code?

We can do that by using the :: syntax:

- foo: !:bar:
    a: b

By adding a : after the tag name, we are switching the argument to code mode. In this case the argument is the Lisp expression (a b) (calling function a with argument b).

There's even more stuff we can do with tag function calls, but since they don't involve modes, we'll save that for another day.

Code Mode in Data Sequences

I forgot to mention how you switch to code mode in a YAML block sequence (always in data mode).

!YS-v0: start in data mode
foo: !:bar
- 42
- ! 6 * 7

Well actually I did tell you. You just use the ! tag switcher.

I just forgot to mention there's no sugar like :: for that.

Assignments in Data Mode

YS wants to help you make your YAML files dynamic, while keeping them mostly the same as they were before.

In data mode you can use assignments anywhere and they will be scoped to where they are defined.

!YS-v0:  # data mode

vars =: load("vars.yaml")

a: 1
b:: vars.abc
c:
  var =: load("vars2.yaml)
  d:: vars.abc2
e: vars.def

That's Most of It

There's a few more special cases, but if you've followed along this far, you'll be able to understand them when I talk about them later on.

This might all seem like a lot to take in, but once you've gotten the hang of switching modes, I think that you'll find YS is pretty easy to use.

Comments