A tiny language called Z

“The main idea seems clever, but also too clever” — reddit

A tiny, strict, impure, curried, dynamically typed (although that may change), partially applied language with rather peculiar syntax.

Markdown’s insight

First, I want you to recall Markdown. You've seen it even if you haven't ever written any. And you'll know that there's a particular feature in the Markdown syntax, which is how to embed code. It's exceedingly simple; obvious, even: you indent your code four spaces, and then you can write whatever you want!

Hello world! Here comes some code!

    Here is some arbitrary code! f.x()/f23(); // Zaha!

And now we're back to normal text...

What I realised was special about this idea, is that you can put anything in there. And it doesn't affect, in any way the source code surrounding it! Now, that is a very powerful idea. Let me show you what I mean.

Z-expressions

I'm going to show you a tiny language called “Z” which I have used to illustrate the concept.

Z has very, very simple syntax. Weird, but simple. Here's how it works, function application is of the following form:

name argument

And that's taken to an extreme, because this code,

foo bar mu zot

actually groups like this:

foo (bar (mu zot))

(Note: there are no parentheses in Z. Zero.)

Which, if you think about it, is the natural grouping for the definition of the name argument syntax I gave above.

To pass additional arguments to a function, the arguments are put on the next line and indented to the column of the first argument:

foo bar
    mu
    zot 

This means that the function foo has three arguments. This rule applies everywhere, so I can, of course, write:

foo bar mu
        zot
    bob

This means that the function foo has two arguments, and the function bar has two arguments.

I call these “z-expressions”. Lisp is curly, curvy. It has its s-expressions. Z is jagged and sharp. And weird.

Special operators follow the same rules. Now I'll show you some of those special operators.

Z’s built-in operators

The defun special operator takes two arguments: a list of names, the first of which is the name of the function, and a body for the function. Here's a function that appends two lists:

defun ap x y
      ++ x
         y

All Z functions are curried and partially applied, like in Haskell, so the above is equivalent to

def ap
    fn x
       fn y
          ++ x
             y

but that doesn't matter for this introduction. We also have if and do:

if foo
   bar
   mu

do this
   that
   those

Note, if you will, that these special operators interpret their arguments in a non-function normal-order way. They interpret their arguments syntactically!

We also have some number 123 syntax, "strings" and unit, as in nothing, null, empty, voidness, niente.

Defining macros

Aha! La pièce de résistance! We also have a defmacro operator with the specific task of allowing us to define new syntax. Observe…

defmacro -- _
         "unit"

Voilà! We have defined the name -- which will take an argument _ and return the string "unit".

All macros take in a string, which is all the source code that can be arguments to it, which, as we know, is done by indenting. And all macros output a string that will be put in place of that macro call, and will be re-parsed by the language.

In the case of our -- macro, however, we're just returning unit, a no-op. We've defined our own comment syntax.

-- A simple function, that is used inside the macro below.
defun ap x y
      ++ x
         y

Tada! There's a function with a comment! That comment syntax, we just made it up! We can also use this function, ap inside other macros, which is typical of the Lisp family of languages. And now let's do that, and define a more complicated macro:

The when macro

-- A messy macro (because it uses string manipulation),
   but demonstrates the idea well enough.
defmacro when input
         fn blocks
            ap "if"
               ++ z:indent-before 3
                                  car blocks
                  ++ "\n"
                     ++ z:indent 3
                                 car cdr blocks
                        ++ "\n"
                           z:indent 3
                                    "unit"
            z:blocks input

Here we can see that I have provided some helper functions for getting the set of “blocks”—i.e. arguments in an application—and I'm passing that to the anonymous function starting at fn blocks, then I am constructing a string which is returned.

Can you tell the aim of this macro? It's to let us write this:

when = 1
       1
     print ++ "The number is: "
              when true
                   show 123

See how it looks native? Macros within macros are fine!

The string macro

A common problem in programming is how to write strings of text in a non-annoying way. Often we put up with our strange ways of escaping strings. In Z, you don't have to!

This is the normal way to use strings:

 print "Hai, guys!"

Here we define a macro to make writing strings easier, called :, it's meant to read like typical English, and lets you write arbitrary text as long as it's indented to the offside column.

defmacro : input
         z:string input

Here I provided a utility to make a string into "string", so that whatever is passed as input into the macro will be returned verbatim, but in string syntax. Ready? LOOK NOW!

-- Example with print:
print : Hello, World!
        What's going on in here? 
Isn't that just wonderful? It reads like a script! And that, is exactly the insight that Markdown had. Again, it works just fine with other function application:
defun message msg
      do print : Here's a message
         print msg
         print : End of message. 

And you can use it:

message ap : Hello,
           ++ " World! "
              : Love ya! 

Except you wouldn't write it like that, you'd just write:

message : Everybody dance now!

Defining some functions

Enough awesome for now. Let's take a breather from all that excitement and look at some boring pure functions. This is what code in Z looks like.

-- Map function.
defun map f xs
      if unit? xs
         unit
         cons f car xs
              map f
                  cdr xs

-- ["foo","bar"] → foo\nbar\n 
defun unlines xs
      if unit? xs
         ""
         ++ car xs
            ++ "\n"
               unlines cdr xs

-- Take the first n elements of list xs.
defun take n xs
      if = n
           0
         unit
         if unit? xs
            unit
            cons car xs
                 take - n
                        1
                      cdr xs

-- Take all but the last element of a list.
defun init xs
      if unit? xs
         unit
         if unit? cdr xs
            unit
            cons car xs
                 init cdr xs

-- Take the last element of a list, or return a default.
defun last def xs
      if unit? xs
         def
         if unit? cdr xs
            car xs
            last def
                 cdr xs

Isn't programming without pattern matching completely boring!? Sadly, we won't be defining a pattern matching syntax in Z today, because writing a decent pattern macher is non-trivial. And writing a crappy one is embarassing.

So we can use those functions, and all works as expected:

-- Print the blocks of foo and bar with ! on the end.
print unlines map fn x
                     ++ x
                        "!"
                  z:blocks : foo
                             bar

-- Use of take function.
print unlines take 3
                   z:blocks : foo
                              bar
                              mu
                              zot

Regular expressions

Here's another, easy use-case for macros: regular expressions! Let's experiment a little.

Our basic regex functions from the standard library are regex:match and regex:new. And regex:match returns a list of matches as marked by the (foo) syntax of regular expressions.

print regex:match regex:new "(abc)"
                  "abc"

We're already macro coinnnoisseurs (get it?) by this point, so let's dabble with some nicer syntax:

defun ~~ regex string
      regex:match regex
                  string

print ~~ regex:new "(def)"
         "defghi"

What do we think? Not bad? It's shorter to write the match, at least. But building the regex is still cumbersome. Let's make a macro for that!

defmacro rx input
         ++ "regex:new "
            z:string input

print ~~ rx Age: (.*)
         "Age: 123"

Bit nicer, but not amazing.

Let's maybe skip the whole composing part and merge in the matching together:

defmacro ~ input
         fn blocks
            ++ "~~ rx"
               ++ z:indent-before 6
                                  unlines init blocks
                  ++ "\n"
                     z:indent 3
                              last ""
                                   blocks
            z:blocks input

print ~ Age: (.*)
        "Age: 666"

Now we're cooking with gas! That looks like a million dollars, pre-recession!

print ~ Age: (.*)
        ([a-z]+)
        "Age: 777\nlalala"

Oh, fancy that, we can even write multi-line regexes. God damn, that's some delicious awesome sauce. Can I get another bottle, waiter?

print ~ Age: (.*)
        ([a-z]+)
        : Age: 999
          beep!

Ah, of course. It even works with other macros. How's that for a slice of fried gold?

Editing

Another aspect of Z-expressions which is totally suave is that editing it can largely be made trivial. Question: how do you capture the starting and ending positions of the current node in Lisp or any other language?

(lorem ipsum-lorem-ipsum ()
  (foo-bar)
  (let* ((zot (biff-pop))
         (zar-zar (beep "%s.bill" bob)))
    (if (ben-bill-bocky doo-dar)
        (let*| ((foo (foo-bar-mu-zot))
               (bar (ipsum-lorem))
               (ipsum (cdr (assoc 'cakes lorem)))
               (lorem (cdr (assoc 'potato lorem)))
               (ipsum (cdr (assoc 'ipsum lorem)))
               (lorem (cdr (assoc 'lorem lorem))))
          (if bob
              (progn
                (bill ben)
                (the cake is a lie)
                (the game))
            (message "Structural integrity is not secured.")))
      (message "Data, because it's polite." cakes))))

If you're just after the let, what do you do? The usual thing. You start looking for a start parenthesis. You find it. Then you start walking forward, looking for a closing parenthesis. Every time you encounter an opening parenthesis, you push it onto a stack. Every time you encounter a closing one, you pop it off the stack. Unless you encounter an opening string, or character escape, in which case you wait until you encounter another, non-escaped string, and continue… Sorry, was I boring you? Yeah, me too. I thought I could make it, but I can't.

However, in Z. It's easy. You go to the starting column, identified by the first non-whitespace character. Then you go up and down a line and do the same thing until the starting column is not equal to or greater than this one. Done. You have the whole z-expression. You want to move it? Easy, you cut it out and paste it, and add or remove spaces according to the new offset. Worried about indentation styles? There are non in Z. It's impossible to have indentation styles. There is only one indentation.

Future Work

Quasiquotations

We would be nothing if we did not learn from history. And Lisp has a lot of history, and it has taught us about quotation and quasiquotation, and how convenient it can be over strings. And I agree. That's why, next, I will implement this syntax:

defmacro when cond body
         ` if , cond
              , body
              unit

Of course, it follows the same syntactical pattern as all Z-expressions, but the same semantics as in Lisp. However, this is merely syntactic sugar. The real power in Z lies in its reliance on indentation to denote regions of text.

A “math” macro

In Z, you indent for many-argument functions. That can be boring for functions involved in maths, for which the arguments are often simple, other expressions of the same order. For that, a math macro is entirely appropriate. For example, #:

def x # x²-y²×(2xy+x²-y²×(2xy+c))

Why not?

Implementation

There's an implementation here, but I wouldn't try it, it's too buggy awesome for you, I'd just look at it, and try to imagine the bodacious vibes kicking off it. Alright?


© 2013-01-01 Chris Done <[email protected]>