I can relate to many points. I made a lisp (a scheme) as a scripting language for my game dev purposes, and went through many of the same phases. I really wanted continuations in my scripts (super useful for sequencing things over time across multiple frames, which is mostly what games do). So I had to get comfortable with cps style. But oh, it blows up the stack. But what if I had my own stack?
What I have currently is very close to the stack interpreter described in the fantastic paper Three Implementation Models for Scheme, while initially it was built iteratively based on Lisp in Small Pieces. Though I don't use "real" continuations, just an "await", "race" etc. which is just syntactic sugar that is easy to implement once there is a cps conversion in place. There are some important optimizations that can't be done (or are really hard to prove) if there are real continuations. For example, who knows if a closure can be magically re-entered if the language basically supports goto? Better not put it on the stack, even if it seems like it won't escape the scope.
From what you write about macros, it hints at something like F-expressions and first class environments, though I am not sure if it actually does have it.
I find the Kernel programming language to be inspiring. It is based on first class environments and tries to go further in the direction of the philosophy of Scheme : "Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary."
Your post is also in this spirit.
A language like this would allow a much greater dynamism and a reflective capability, but compiling it is a research project. (There was also a minor scuffle here on HN, a decade ago, regarding the non-inclusion of eval in ClojureScript in order to enable better optimization.)
Edit: Brief explanation of F-expressions (vau expressions in Kernel)
For a function call, the argument expressions are evaluated and the function only receives the values.
For a macro, the argument expressions are accessible to the macro, but the macro cant access the environment, so it cant evaluate these expression, but rewrites code into some other code.
For a vau expression as in Kernel(analog of lambda), the expression receives not just the argument expressions but also the lexical environment. This means it can genuinely return a result and not just code. Unlike macros, they can be evaluated even at runtime. Lambda and classical macros can be easily built on top of vau expressions.
One evolution of (You're probably aware of re:axisofeval) Kernel is Wat (https://github.com/manuel/wat-js), which implements the Vau operator (hygenic F-exprs), full delimited continuations (as opposed to the usual Shift/reset functionality of most lisp-based langs) and dynamic variables (variables which can break out of the usual lexical ie. compile-time scope and take environmental ie. runtime values).
It's quite fast with all of that. The continuations allow you to define promises, threads, channels, exceptions, branching and so on.
Thing is, syntatically the continuations are required to carry quite a lot.
Algebraic effects layered on top would have a nicer syntax.
Similarly, I would suggest that dynamic variables and continuations are more understandable in a forth-like language where the return and data stacks are first class. What's more, forth-like languages use a concatenative (streaming) syntax which allows streams to be more first class.
Forsp (https://xorvoid.com/forsp.html) is such a language, that supports both lisp-like, and forth-like semantics
There's a lot more to say about GC, compilation steps, DSL towers, formal verification, meta-compilation and so on, suffice to recommend checking out the afore mentioned languages!
Oh that is a fantastic article. I wasn't even aware of Bel. I am currently working on my own Lisp dialect and some things in the spec are lovely. The idea of lit is pretty cool. I'll have to make a branch and toy around with it, it seems to make for a smaller compiler core.
The Bel spec though needs some love in a more modern formatting. I might post a formatted version later.
I can relate to many points. I made a lisp (a scheme) as a scripting language for my game dev purposes, and went through many of the same phases. I really wanted continuations in my scripts (super useful for sequencing things over time across multiple frames, which is mostly what games do). So I had to get comfortable with cps style. But oh, it blows up the stack. But what if I had my own stack? What I have currently is very close to the stack interpreter described in the fantastic paper Three Implementation Models for Scheme, while initially it was built iteratively based on Lisp in Small Pieces. Though I don't use "real" continuations, just an "await", "race" etc. which is just syntactic sugar that is easy to implement once there is a cps conversion in place. There are some important optimizations that can't be done (or are really hard to prove) if there are real continuations. For example, who knows if a closure can be magically re-entered if the language basically supports goto? Better not put it on the stack, even if it seems like it won't escape the scope.
From what you write about macros, it hints at something like F-expressions and first class environments, though I am not sure if it actually does have it.
I find the Kernel programming language to be inspiring. It is based on first class environments and tries to go further in the direction of the philosophy of Scheme : "Programming languages should be designed not by piling feature on top of feature, but by removing the weaknesses and restrictions that make additional features appear necessary."
Your post is also in this spirit.
A language like this would allow a much greater dynamism and a reflective capability, but compiling it is a research project. (There was also a minor scuffle here on HN, a decade ago, regarding the non-inclusion of eval in ClojureScript in order to enable better optimization.)
https://web.cs.wpi.edu/~jshutt/kernel.html
https://axisofeval.blogspot.com/2011/09/kernel-underground.h...
---
Edit: Brief explanation of F-expressions (vau expressions in Kernel)
For a function call, the argument expressions are evaluated and the function only receives the values.
For a macro, the argument expressions are accessible to the macro, but the macro cant access the environment, so it cant evaluate these expression, but rewrites code into some other code.
For a vau expression as in Kernel(analog of lambda), the expression receives not just the argument expressions but also the lexical environment. This means it can genuinely return a result and not just code. Unlike macros, they can be evaluated even at runtime. Lambda and classical macros can be easily built on top of vau expressions.
One evolution of (You're probably aware of re:axisofeval) Kernel is Wat (https://github.com/manuel/wat-js), which implements the Vau operator (hygenic F-exprs), full delimited continuations (as opposed to the usual Shift/reset functionality of most lisp-based langs) and dynamic variables (variables which can break out of the usual lexical ie. compile-time scope and take environmental ie. runtime values).
It's quite fast with all of that. The continuations allow you to define promises, threads, channels, exceptions, branching and so on. Thing is, syntatically the continuations are required to carry quite a lot. Algebraic effects layered on top would have a nicer syntax.
Similarly, I would suggest that dynamic variables and continuations are more understandable in a forth-like language where the return and data stacks are first class. What's more, forth-like languages use a concatenative (streaming) syntax which allows streams to be more first class. Forsp (https://xorvoid.com/forsp.html) is such a language, that supports both lisp-like, and forth-like semantics
Furthermore, forth-like languages give you explicit parsing to be able to handle quasiquote and special chars etc. This parsing can be layered, you can have a low-level parser along the lines of able-forth (https://github.com/ablevm/able-forth), and a higher-level (word/function level rather than char level) parser like rebol (https://en.wikipedia.org/wiki/Rebol) /ometa (https://wiki.squeak.org/squeak/3930).
There's a lot more to say about GC, compilation steps, DSL towers, formal verification, meta-compilation and so on, suffice to recommend checking out the afore mentioned languages!
Oh that is a fantastic article. I wasn't even aware of Bel. I am currently working on my own Lisp dialect and some things in the spec are lovely. The idea of lit is pretty cool. I'll have to make a branch and toy around with it, it seems to make for a smaller compiler core.
The Bel spec though needs some love in a more modern formatting. I might post a formatted version later.
And here it is: https://gist.github.com/Kintaro/2aaa3cb3cf1614883977993e86ce...
Nice, there are lots of interesting ideas/possibilities in Lisp that I feel deserve being further explored.
I've been working on a custom dialect with a focus on practical use as an embedded scripting language for quite a while now:
https://github.com/codr7/eli
[dead]