You're really very wrong on Nim's macros. It follows Lisp's defmacro tradition instead of Scheme syntax-rules/syntax-case, but that doesn't make it any less powerful (many would argue it's demonstrably more powerful). You are also dead wrong on syntax-rules/syntax-case capabilities, or maybe on what the syntax/AST is, if you think that there's anything they can do that Nim can't. Both systems deal with AST which means they both are unable to introduce new ways of parsing the code, only transform already parsed one. In (some) Scheme and Common Lisp you get access to readtable, which is the parser, but that's really a different thing. And even in Lisps it's not that popular: Clojure and Emacs Lisp disallow this for example.
Personally I favour pattern-based macros, like the ones implemented in Dylan, Elixir or Sweet.js (to show some non-sexp-based languages with such macros); but there is nothing "wrong" with procedural macros and they are not, in any way, less robust.
You don't have to be excited by Nim, but you should try to avoid spreading lies just because you aren't. Maybe a "lie" is too strong a word, but this statement: "Nim's macro system seems to be far less robust than that" is really very wrong and I wanted to stress this fact.
Lisp has reader macros that can alter lexical analysis and parsing; correct me if I am wrong, but I think that’s not possible in Nim. E.g. things like JSX are trivial to implement in Lisp.
Also, lisp macros let you e.g. write new control structures with multiple “body” parts - iirc, in nim only the last untyped macro arg can be a code body (you can put a block in parantheses, but that’s not as elegant)
I’m sure there’s other stuff that fexprs and other [a-z]exprs can do that nim can’t, but i’Ve never seen them in use (or used them myself)
Also, personally I think Nim’s model is more practical; lisp’s power essentially requires lispy or REBOLy syntax to be usable. Nim is pascalythonesque, and though complex is not complicated; much like Python, and unlike C++, you can make use of the ecosystem without being afraid of obscure details biting you - but it has all the capabilities when you need them.
First, can you talk a little about how Nim's macro system is more-powerful than Lisp's? That seems like a fairly broad claim, and unless it allows you to influence the compiler at compile time (and maybe even then), I am curious how that works out.
Second, one of the biggest issues with lisp is that macros end up sort of slow, especially when you're using them + lists to replicate ADTs with pattern matching (versus, say, ML, which can do very fast dispatch based on types). Doesn't Nim fall into that same trap?
Yes, but Nim's macro system, while very powerful, is, at present, the most clumsy and awkward thing I've seen. it's imperative, like Lisp macros. However, unlike Lisp macros, there are no templates, and the datastructure that Nim's macros manipulate is much closer to the actual AST, and thus far more complex. Because of this, Nim macros must clumsily plug together an AST, and do so in a manner that's so noisy that you have to squint to see what it's actually doing by the time you're done. Meanwhile, Lisp macros are not necessarily shorter, but it's far easier to see what's going on.
Fair enough I guess. If the Lisp macro system only sees parsed s-exps, in principle you still need to figure out for yourself what kind of construct you are dealing with on a higher level of abstraction. Nim's macro system seems to be operating on a higher level of abstraction in that regard.
> A lisp macro see a code block as a tree of symbols/primitives. It can do anything.
No, a regular macro still needs to be syntactically sensible. To handle arbitrary non-lisp syntax you need your lisp to support arbitrary reader macros (as in Common Lisp or — I believe — Racket) and that gets significantly more complex and involved and requires extensible/pluggable parsing.
Scheme does not have that for instance, SFRI-10 reader macros need to be wrapped in `#,()`, you can't just dump JSX or XML in Scheme source and expect it to work.
The Lisp language syntax is not motivated by 'ease of parsing'. It is motivated by the 'code as data' idea, which enables a lot of useful source transformations to be implemented AND used in a relatively easy way. One of the applications of that are macros.
Scheme got a lot wrong from a practical perspective. Use Lisp instead.
Type theories are over-hyped. 99.99% of all software is written in languages without advanced type systems.
Macros are nothing for 'excitement'. They are a tool. They enable the user to write source transformations, which has several applications in Lisp: control structures, compile-time computation, etc etc.
Of course they're powerful. They're true imperative macros, the very thing that all Smug Lisp Weenies worship (I don't, because I'm not an SLW, but they are Pretty Cool). The reason I think the macros are clumsy is pretty simple: Nim is really, really bad at manipulating its own AST: It shuffles around, munging data, and by the end, you have no idea what the result should look like. Unlike Lisp, there are no templates in macros, which make seeing what the result will be very easy.
This is how a debug macro, a very simple one, is implemented in Nim (taken from the tutorial):
macro debug(n: varargs[expr]): stmt =
result = newNimNode(nnkStmtList, n)
for i in 0..n.len-1:
result.add(newCall("write", newIdentNode("stdout"), toStrLit(n[i])))
result.add(newCall("write", newIdentNode("stdout"), newStrLitNode(": ")))
result.add(newCall("writeLine", newIdentNode("stdout"), n[i]))
What the heck is going on? This is the equivalent Lisp (specifically, Chicken Scheme, the dialect I am most familiar with, but it should be similar in most lisps with imperative macros):
(define-syntax debug
(ir-macro-transformer
(lambda (e i c)
(let ((stmts (cdr e)))
(map (lambda (stmt)
`(begin (write ,stmt)
(display ": ")
(newline))
stmts)))))
That's much easier to understand. It might be a bit less concise, but it more than makes up for it.
I don't blame Nim for being bad at manipulating its own AST. Lisp is pretty bad at it, too (no, lists are not the lisp AST, not in any lisp suitable for the Real World. I don't know who told you that, but it's a lie: Most lisp ASTs are just as complex as Nim's). What I do blame Nim for is not being potentially homoiconic so there's no easy datastructure to represent code in, or at least providing a datastructure that's easier to manipulate, or at the very least provide some mechanisms to make the existing structures easier to manipulate.
Some Lisp advocates may tell you that Lisp even does not have hygienic macros. Lisp dialects like Scheme have. Lisp usually has procedural macros, which transform source code (in the form of nested lists), not ASTs (like Nim).
That Nim has 'powerful hygienic macros' is fine, many languages have powerful macros.
That's definitely true in terms of defmacro and syntax-case. However, I do think with some effort the scripting-language community could come up with something almost as powerful as syntax-rules from R5RS, thus catching up to where the Scheme community was in 1991. With syntax-rules the person writing the macro does not have to explicitly deal with the parse tree.
> Lisp macros operate on a representation of the AST.
The representation of the AST is the AST in Lisp (Scheme has syntax objects).
> By overhead I mean the nim compiler can generate direct specific instatiations at compile time instead of doing runtime based duck-typing as in lisp.
SBCL has `deftransform`, which allows macro expansions to benefits from static type analysis.
Not a Lisp user, but a heavy Nim macro user. Nim macros are AST based, which means that after the parser part of the compiler have read the code and created a tree structure of it your macro is called on this tree. Then you can modify this tree to your hearts desire and return a new tree that the Nim compiler will splice back into the main tree before continuing. This means that you rewrite pretty much anything into pretty much anything else. You also have access to reading files on the machine, or doing pretty much anything else that Nim offers. For examples of what you can do with it you can have a look at my macro to create GUIs: https://peterme.net/cross-platform-guis-and-nim-macros.html, my article on how I use macros to create more read- and maintainable code: https://peterme.net/metaprogramming-and-read-and-maintainabi... or perhaps see how Protobuf was written as a string parsing macro which reads a Protobuf specification and spits out all the types and procedures you need to use it: https://github.com/PMunch/protobuf-nim
I love Lisp as much as the next neckbeard, but macros aren't unique to Lisp and they aren't the only kind of compile-time execution.
It's tough to come up with a better syntax than sexprs for writing macros in, that's clearly true. A number of recent languages have managed good-enough macros, Nim and Rust being two examples.
My point is only that unless you are using a hygienic macro system the idea that you are manipulating code in your macro is a (often white) lie. Code has semantics, a meaning, and unless the object you manipulate carries those semantics with it (that is, the syntax objects of eg `syntax-case`) you're just manipulating some data which has a necessarily superficial relationship with the code itself. Picolisp resolves this by simple "eliminating" lexical scope, which means that code really is trivially related to its denotation since the semantics of variable binding really are just "whatever is currently bound to this variable." Scheme resolves this by having syntax-transformations instead of macros: functions which genuinely manipulate syntax objects which carry along with them, among other things, information about their lexical context. Common Lisp accepts that most of the issues arising from the distinction between code itself and its nude denotation can be worked around and provides the tools to do that, but in Common Lisp one still transforms the denotation of the code, not the code itself. From my point of view, if one is purely interested in the aesthetics of the situation, the Scheme approach is much more satisfactory. From a practical point of view, it doesn't seem to be particularly onerous to program in, although the macros in scheme seem to lack the immediate intelligibility of the Common Lisp ones.
On the whole macro debate, whatever one thinks the default behavior should be with respect to variable capture, the syntax-case macro system in scheme works on a representation of program code which is at a higher resolution than what lisp macros works on.
The macro can find out for instance, if an identifier used in its input is a free-variable, or if two identifiers share the same binding.
Hence, it is possible to write a relatively short program to define defmacro in terms of define-syntax & syntax-case, but not the other way round. (a pretty short implementation & most of it is error checking - http://svn.plt-scheme.org/plt/trunk/collects/mzlib/defmacro....)
This also strikes me as a inconsistency in arc(albeit easily correctable), as it goes against the rule of not imposing restrictions on the user, by default. Even, if the default macro interface is something simple, the user should be able to write macros which work with syntax at a higher resolution if needed.
Also, if you use the most popular distribution, PLT scheme, you have all of reader macros, native hash tables, keyword arguments, a package repository. There is a really nice module system, along with other neat stuff like contracts, lazy modules, frp & of course the continuation based web-server. For OOP, there is Swindle, an analog of CLOS(not having used it, i don't know if it's just as good).
After R6Rs, there is a common module system and standard libraries for hash-tables even in the scheme spec.
The main flaw, it seems, is the relative lack of libraries as compared to other mainstream languages.
In particular, what i miss is something like wxPython for scheme.
EDIT : turns out i spoke too soon, defmacro does have an environment object(http://www.lispworks.com/documentation/HyperSpec/Body/03_aad...) as a parameter, so it has some access to lexical info of its input form. To implement define-syntax, one would also need some way to output a syntax object and not just a s-expr. Couldn't find any defmacro implementation of define-syntax on the net.
Personally I favour pattern-based macros, like the ones implemented in Dylan, Elixir or Sweet.js (to show some non-sexp-based languages with such macros); but there is nothing "wrong" with procedural macros and they are not, in any way, less robust.
You don't have to be excited by Nim, but you should try to avoid spreading lies just because you aren't. Maybe a "lie" is too strong a word, but this statement: "Nim's macro system seems to be far less robust than that" is really very wrong and I wanted to stress this fact.
reply