YS Streaming Mode
We've seen before that YS can work with YAML files that contain multiple documents. "Document" is the YAML term for a single top level node (object).
YS returns the last document in the file by default.
YS can also load all the documents in the file.
This is called streaming mode, and we'll learn more about it today.
Multiple Documents🔗
Like I said, YS returns the last document in the file by default:
$ ys -Y <<<'
---
foo: 111
---
bar: 222
'
bar: 222
This makes sense, because the other documents could be data sets, code definitions or YS modules that get used by the last document to produce its desired value:
$ ys -Y <<<'
!YS-v0
--- !code
foo =: 111
--- !data
bar:: foo
'
bar: 111
But what if we want to load multiple documents?
We can do that by using the -s flag:
$ ys -s -Y <<<'
---
foo: 111
---
bar: 222
'
---
foo: 111
---
bar: 222
What happens if some of the documents are code mode and others are data mode?
$ ys -s -Y <<<'
!YS-v0
---
foo: bar
--- !code
foo =: 111
--- !data
foo:: foo
--- !code
say: "Hello, $foo!"
--- !code
(1 .. 3) + (10 .. 12)
--- !data
bar:: foo
'
Hello, 111!
---
foo: bar
---
foo: 111
---
- 1
- 2
- 3
- 10
- 11
- 12
---
bar: 111
If you count carefully, you'll see that there are 7 documents in the input, but only 4 documents in the output.
YS will only return the documents whose value is representable as JSON and only
if the value is not nil.
In one of the documents, we just printed a line with say.
Let's see what the return value of say is:
$ ys -pe 'say: "Hello"'
Hello
nil
It looks like say returns nil.
That means the value of that document is nil.
Since the value of that document is nil, YS will not load it.
What about the variable assignment?
$ ys -pe 'foo =: 123'
#<Var@21b42ece: 123>
That returned a Clojure var object. Since this is not something that can be represented as JSON, YS will not load it.
The stream function and the _ variable🔗
As YS evaluates a YAML file, it keeps track of the values of each document. Any document can access the values of the previous documents.
The stream function returns a list of the values of the previous documents.
The _ variable contains the value of the previous document.
It's short for stream():last or stream.$.
Consider this example:
$ ys -Ye '
!YS-v0
---
foo: 111
bar: 222
--- !code
merge stream().$::
baz: 333
--- !data
number:: _.foo * _.bar * _.baz
--- !code
say: stream():yaml/dump'
- foo: 111
bar: 222
- foo: 111
bar: 222
baz: 333
- number: 8205786
In the final document, we print the values of the previous 3 documents in YAML.
The second to last document calculates the product of the values from the previous document, whose value is the result of a merge using the document before that.
If you study that example carefully, you'll see how to use the stream function
and the _ variable.