Quick Tour for Converts

TONAL is Not A LISP

You may have taken a peek at the Quick Tour for Beginners and felt terror at seeing all the parentheses, but rest assured – TONAL is Not A LISP. Yes, parentheses are the only block delimiter, but TONAL shares more design philosophy in common with C++ than perhaps any other language.

TONAL does not have an obsession with homoiconicity, or “code is data”. Lists are syntactical containers only. There are no macros, or gensyms, or call-with-cc. There aren’t fifty different forms of let.

A pragmatic conceptual descendant of C++, TONAL has the same goal of zero-overhead-abstraction and being close to the machine while providing high-level constructs. Performance matters, because performance is usability.

TONAL asks the question – what if a language was designed to have compile-time, generic programming with reflection from the start. Being able to program at compile-time and access the language syntax is a prerequisite for a modern programmer’s toolkit. It enables higher levels of safety and usability with no sacrifice in performance. It somewhat necessitates a reduced syntax – hence the parentheses.

Types – the familiar

(type Derived base (Base)
      (readers '.*')
      (func getX (return x))
      (readers)
      (let x 1))

Types are like C++ classes. Functions not virtual by default. Every member and base type private by default. Access specifiers. Multiple inheritance allowed. Nested type definitions. Dot notation to access class members. Declare certain external-to-class types and funcs as friends.

There are special members functions – constructors, destructors, conversion operators. They are all explicit, in C++ parlance, because all the problems with implicit construction and conversion are well documented.

Types – the unfamiliar

Access specifiers are split into readers and writers. Safely make some members publicly readable, but not writable. Declare some members readable and/or writable only by some funcs from the same class.

The readers and writers commands delineate regions of access until the next respective readers or writers. A reader or writer command with no arguments resets everything to the equivalent of C++‘s private specifier. The reader/writer state can be pushed or popped like a stack for readability.

Types are also namespaces. Types do not have static members, but a type is an object that is initialized. A member referred via the type is distinct from it referred via an object of that type. Members made inaccessible for the object is the same kind of global as a static member.

Types can derive from anything, even functions, integer types, types with a specific value.

Types – the weird

One idea imported from Java is the ability for a anonymous class to grab (capture) a type‘s enclosing object. Types enclosed inside a func can similarly grab named variables as a closure.

One idea imported, that couldn’t make it into C++ but made it into Rust, is destructive move. So there is no need to write a move constructor. Objects that are moved are not destroyed, but ownership is passed to the move destination. If lexically-scoped, the stack memory of a moved object may be reclaimed and reused for the next object.

There are no assignment operators either. For copying into an existing object, the copy constructor is used for both purposes.

TONAL introduces the redirection operator which is a member func which takes a present-tense (compile-time) atom (string literal) as its first argument. This makes it easier to write wrapper types that can look like the wrapped type when used.

Func – the familiar

(func ytown (town-name atom)
     (return (+ 'Won't you take me to ' town-name '?')))

Funcs are non-virtual by default, like in C++. Object funcs can be virtual and can override virtual funcs.

Funcs can be a free function – subdominant, or a member function – dominant. Subdominant funcs are defined either in a namespace type, or as a lambda/closure. Lambda funcs can grab (capture) variables by name reference or value. Object funcs cannot grab since they already have access to their enclosing type‘s members.

Funcs, both subdominant and dominant, can be resumable functions like C++‘s resumable functions (often conflated with coroutines).

Func – the unfamiliar

One idea imported from multiple languages is uniform function call syntax. Member funcs can be called by their qualified name, with the object or type they are enclosed by – their dominant form, or they can be called as a free function – their subdominant form. The subdominant form always tries to search for the dominant equivalent of the function in the object of a command before searching in enclosing scopes.

Parameter types must be matched exactly as there is no implicit conversion in TONAL – but for one exception. Pruning (overload resolution) fails if there are more than one func that matches the given arguments because the programmer’s intention is ambiguous. When there are no overloads, there is no ambiguity, so only then is it done.

Arguments to funcs are lazily evaluated, so temporaries are not constructed unless read. This allows for short-circuiting functions without the use of macros.

Funcs – the weird

One idea imported from Java is object method references. Using the object dominant name of a func in subject positions in a command implicitly grabs the object as the context for that instance of the member func.

One idea imported from LISP is multimethods. Funcs with virtual parameters of a type that has virtual functions perform multiple dispatch at runtime when an object‘s most derived type is not known. Virtual parameters are the only way to down-cast an object at runtime in TONAL. Downcasting is a code smell – a codour. It is a necessary evil in C++ because funcs are not virtual by default, so you can’t simply lift a func into a base class without hurting performance in the common case. It could also pollute the base class with unnecessary detail. When we downcast, we normally want to access some derived type specific function or data to do a type specific thing, so multimethods is what we really want anyway. Downcasting was just a means.

Compile-time programming

TONAL denotes “compile-time” as present-tense. A present-tense value has a value known to the compiler. TONAL has six archetypes, similar to the concept of fundamental types, except they do not have a fixed machine representation. They are archetypal categories of all data types. They, and any value that can be converted to them, can be used in present-tense contexts.

By having tensor archetypes as present-tense values, TONAL can provide a uniform syntax for compile-time generic (templates) code and normal code. All code is compile-time generic if they never read future-tense values. It is as if we used the type system to encode the present-tense/future-tense divide without special syntax for templates/generics, constexpr, if constexpr/static if, if_constant_evaluated, etc.

Present-tense, Present-time … … …

To round off support for present-tense programming, TONAL supports present-tense exceptions. This makes it possible to explicitly direct the compiler – such as programmatically pruning overloads – using normal code, instead of relying on tricks like SFINAE.

C++ templates have typename and auto for type and non-type template parameters. For functions, they are also separate from the function’s normal arguments. The TONAL tensor archetypes can have parameters that are (or present-tense values that can be converted to) any of the TONAL archetypes. Type parameters are the equivalent of typename template parameters, and the rest of the archetypes are equivalent to auto template parameters.

One idea similar to D is that there are no primary templates. Template resolution is exactly the process of overload pruning. Partial template specialization is achieved simply by parameters defined as a present-tense value. SFINAE really is just a special case of overload pruning. That’s why it’s unnecessary to have a template syntax.

Present-tensor programming

TONAL reduces compile-time U generic programming to overload pruning and operations on present-tense values. Overload pruning works the same for the tensor archetypes to further reduce the friction between functions and classes we experience in C++ generic programming.

Tensor parameters can be variadic (pack), or have a default value, or both. Pack parameters are present-tense lists, like C++‘s template parameter packs. TONAL introduces the @ token to allow the caller to skip a pack or defaulted parameter.

Tensor parameters can also be special rules – called mediant forms – that are like C++‘s concepts. For more complex matching rules, TONAL defines a few present-tense exceptions that can be used to control the pattern matching.

Names can contain almost any character. TONAL does not reserve as many special characters for C compatibility. Operator overloading is just normal overloading because there are no special operators. Again, this further reduces friction between classes, functions and operators we experience in C++ generic programming.

What do we want? Reflection! When do we want it? In the present-tense!

Present-tense programming U reflection can be as powerful as homoiconicity. You can calculate anything in the present-tense, use those present-tense values with tensor archetypes to instantiate new types – in-effect, generating new types; and being able to calculate and generate based on a program’s structure. TONAL even supports certain included files as present-tense.

TONAL‘s reflection will always be present-tense, so it will not bloat or slow down a program. Only the final results of reflection need to be kept to actually do something useful with future-tense tasks. Imagine being able to write a compiler for a domain-specific language, with the grammar specified as an atom, that runs in present-tense and converts it into an interpreter.

The combination of present-tense programming and reflection can be used to write unit-testing frameworks, fuzz-testing frameworks, parsers for regex and domain-specific languages, database object-relational mappings, implementing protocols directly from a specification, API versioning, symbolic algebraic computation (expression templates in C++).

You can’t silence me – I’m immutable!

Functional programming languages like Haskell or Clojure, and new languages like Rust makes everything immutable by default. Older languages like C++ are mutable by default, except for non-reference lambda captures. Rust and C++ makes mutability part of their type system. Rust also makes it part of its lifetimes, through its borrow checker delimited by scopes and lifetime annotations.

TONAL breaks away from the tradition of making const-ness part of the type system. TONAL keeps the importance of present-tense checking that reads and writes are valid. TONAL definitely keeps the lexically scoped lifetimes. TONAL introduces lexically scoped mutability control flow.

A locally constructed variable begins non-past-tense as mutable. TONAL assumes you created a local variable to modify it for a while. When a value is passed to a func, or is grabbed by a func, all non-object position arguments begin as immutable references. TONAL assumes func arguments are mostly just for reading. For the argument in a command‘s object position (whether dominant or subdominant) it keeps its local mutability status.

Values don’t have to stay [im]mutable forever. TONAL‘s system of levers to push and pull gives you total control of mutability over a value’s lifecycle.

Total Ownership of Waste And Lifecycle

The push/pull keyverbs alter the mutability of a variable temporarily. These are not the same as const-ness casts. Casts are used to break out of restrictions. These keyverbs expresses your intention to the compiler of how you’re using an object, and where, without breaking any guarantees.

The compiler doesn’t need to guess a programmer’s real intention from their professed intention through a declaration’s [im]mutability and ref-ness. The compiler uses your intention to decide: where to make something immutable; whether early destruction is possible; whether to pass by value, [im]mutable reference; when to construct; when to copy or move – for example. All of this is enforced in present-tense.

This also eliminates the need for C++‘s expert-level value lifetime categories. No more r-value or l-value confusion. No more combinatorial blow-out of function definitions differing by only const-ness or ref-ness of parameters and enclosing object. Nor more forwarding references that then require SFINAE in order to stop the overload from being selected.

Resource Access Indicating Intent – RAII (Lite)

Imagine living inside a box with a food hole on top. Food might appear over the hole, and you can look at it all you want. To actually eat the food, you need to pull-in the food. You don’t know where the food comes from or if there’s more. You take a bite of the food and you may decide to push-out back the rest. You can still look at it, but you don’t know what happens to it. You may want to pull-in it back again.

Imagine owning some mystery pet in a box. You feed it some pet food by push-ining. Whether the pet eats the food or not, you don’t need it yourself. You may have some bread to share, so you attach some string to it and lower it into the hole. After some time, you pull-out what’s left. Maybe it just looked at it instead of eating it. You may have chocolate, which is deadly to your mystery pet. You can show it to them but not let them pull-in it.

Imagine living inside a box. You make food because you want to eat it, not just to look at. It’s like you pull-ined it from ingredients you had lying around. But you might push-out it in a transparent container to pull-in it later to eat.

You don’t have to know what others are doing, or tell them, inside and outside their box. This RAII is lighter and nimbler. With TOWAL RAII(L), you’ll always know where your TOWAL is.

I am the very module of a major programming language

(include 'tonal.tnl')
(func some-large-func (include 'slf.tnl'))
(let logo (include '8bit' 'logo.png'))
(let grammar (include 'utf8' 'grammar.txt'))

TONAL has no macros or token-pasting, and so does not use include files. TONAL includes are semantically checked as valid definitions. Tensors defined in modules will always refer to the same definition, and benefit from one-time compilation.

Present-tense definitions are supported. Modules can be used to define a complete tensor, or they can merely define the body of a tensor. Java definitions are all inline, and C++ separates declaration from definitions that can be compiled separately. TONAL combines them both, so that tensors can be defined in a separate include but still effectively defined inline.

TONAL does not read files present-tense, but can include binary and text files as always-immutable present-tense values or structures.

Dynamic module loading

Support for dynamic loading and unloading depends on the system. TONAL adds a few more capabilities to dynamic loading than just loading symbols. Extended checking of type access specifiers is done on load. Updating multiple dispatch tables is done on loading.

TONAL programs have well-defined initialization order of global objects and modules use the same system on loading and unloading.

Going forward, not backward; upward, not forward

We’ve explored the main features that are of interest to programmers from other languages – the high level, organizational and safety and productivity features – and how they compare to existing languages. TOWAL is a subject in its own right, but less complex than value categories.

You may want to look at the other Quick Tour for Beginners to get a feel of TONAL without the baggage of other languages. (goto 'Principles-of-Operations') for a deeper look at TONAL, from the ground up.