In this post I will summarize the lessons learned while implementing clocks. An opensourced test project to learn the in an outs of macros in clojure.
You can see the code at: http://www.github.com/boymaas/clocks
On macros
if you don't like something in the language, just adapt it.
This is a core design choice of lisps. It is called homoiconic. When code is data, data is changable by all the existing routines.
In practice this means when calling a macro you get the unevaluated code. You can than do all kinds of operations to this code, return it and the returned form will get evaluated.
This gives you the oppertunity to generate/transform/compile code which will get evaluated after the macroexpantion is done.
First expand then evalutate
The macro expander first expands all code recursively and then executes it. So try to keep this in mind when creating more complex macro's. Sometimes the two worlds get blurred. Such as I experienced while first generating functions in a macro and then storing references to these generated functions in expantion time when they are not yet evaluated.
It helps to think in two layers, macro-expantion time and evalutation time.
Pitfall quasiquote symbol resolving
Scheme has hygienic macros, which work well for me, Clojure resolves all symbols first in the namespace the macro is defined. (when quasiquote is used)
outermacro defined in ns outerns
body resolved in ns outns
innermacro defined in ns innerns
body gets resolved in innerns
This is a good thing in most cases since it avoids variable capture. But can be annoying when you just want to work with symbols.
When you want unresolved symbols, a problem with quasiquote
When defining a dsl using symbols not defined in the namespace this can pose
a problem since quasiquote will try to resolve every symbol. There is a way
to unresolve symbols. The name function:
(name resolved-symbol)
will give back the original symbol name as a string
(symbol (name resolved-symbol))
will convert the returned string back to the original symbol
(prewalk (fn [exp] (if (symbol? exp) (symbol (name exp)))) sexp)
We can combine this with a walker to unresolve all resolved symbols in a form. Combine this with a lookup on a symbol map and you will leave only specified symbols unresolved.
Prewalk? tools for transforming code, The walker
clojure.walk provides prewalk and postwalk, the walkers take a
function and replace the walked node by the return value of this
function.
Sequence conversion
While walking a piece of code you wish to convert, clojure has the tendency to convert the walked forms to sequences. When walking code you want vectors to stay vectors, etc. These have to be converted back.
As can be seen in the code the walk function as defined in clojure.walk does this for you.
(defn walk
"Traverses form, an arbitrary data structure. inner and outer are
functions. Applies inner to each element of form, building up a
data structure of the same type, then applies outer to the result.
Recognizes all Clojure data structures except sorted-map-by.
Consumes seqs as with doall."
{:added "1.1"}
[inner outer form]
(cond
(list? form) (outer (apply list (map inner form)))
(seq? form) (outer (doall (map inner form)))
(vector? form) (outer (vec (map inner form)))
(map? form) (outer (into (if (sorted? form) (sorted-map) {})
(map inner form)))
(set? form) (outer (into (if (sorted? form) (sorted-set) #{})
(map inner form)))
:else (outer form)))
Evaluation
What exactly happens when calling a macro with the namespaces and the variables.
(ns nsdefined)
(def rv 100)
(defmacro nstest [name]
(let [ns *ns*
rv rv
rvr (resolve rv)]
`(def name rv))
(ns nscalling)
(def rv 1000)
(nsdefined/nstest blah) # ns = nscalling, rv = 100, rvr = #'nscalling/rv
Conclusion: macro gets executed inside namespace its defined in. The ns is set to the calling ns, so resolve which uses ns to resolve will resolve to nscalling/rv.
(defmacro nstest [name]
(let [ns *ns*
rv rv
rvr (resolve rv)]
`(def name ~'rv))
(nsdefined/nstest blah) # ns = nscalling, rv = 100, rvr = #'nscalling/rv
In this case ~'rv will not be resolved by quasiquore and the symbol of nscalling will be used (symbol capture). Thus be bound to 1000.
Summarizing ns is the namespace of the caller, quasiquote will try to resolve symbols inside the namespace the macro was defined. Unresolved symbols can lead to variable capture in the calling namespace.
Conclusion
It was an interesting journey into the world of macro's inside clojure. A very different approach to the hygienic macro's in scheme. Who are more template based. Resolving the symbols is a very good feature for the simpler macro's but have to be compensated when designing a symbol dsl, It would be nice to have a special quasiquote which does not resolve.
