When to create syntax in Lisp?

From the gambit email list, I respond to this question about writing macros...

In your opinion, is it appropriate to use a macro to abstract away repetitive boiler-plate code? Or is this better done in a procedure?

This is almost always a procedural abstraction rather than syntax, especially for beginners with Lisp... better to spend a lot of time with just procedural abstraction, higher-order functions, etc.

Syntactical abstraction I use for controlling the order of evaluation and sometimes to put a pretty syntax around use of lambda.

As an example an old Lisp control structure is called PROG1. This structure takes a sequence of statements, evaluates each in order, and returns the result of the first statement after the last has been evaluated. This is can be expressed as a Gambit macro, but I'll call it begin1 to be more like Scheme's begin than the old Lisp's progn.

? (define-macro (begin1 first-statement . remaining-statements)
    (let ((result-var (gensym 'first-result)))
      `(let ((,result-var ,first-statement))
         , at remaining-statements
         ,result-var)))
? (begin1 1 2 3)
1
> (begin 1 2 3)
3
? (define result-var 5)
? (begin 1 2 result-var)
5
? (begin result-var 1 2 3)
3
? (begin1 result-var 1 2 3)
5
? (begin1 0 (display result-var) (newline) 1 2 3)
5
0

The begin1 is an example of control abstraction.

An example of the latter use of pretty syntax... consider a scenario where you are using a resource and you want some "before" and "after" actions. The base level way to implement this is with higher-order procedures. But who wants to write (lambda () ...) all the time? So on top of this build some pretty syntax. [Note that a better lambda notation like Smalltalk's block syntax would reduce the need for these situations. (pdf)] For example...

(define (call-when-ready procedure)
  (wait-until-ready time-out)
  (if (not (ready?))
      (call-when-ready procedure)))

Use it like this...

(call-when-ready 
  (lambda () 
    (display "I am glad this is finally ready!")
    (newline)
    (do-something)))

This is fine when someone else is generating the code for you. Normally you might want to abstract the procedure as a sequence of statements...

(when-ready
  (display "I am glad this is finally ready!")
  (newline)
  (do-something))

And so when-ready is defined as a macro that calls call-when-ready...

(define-macro (when-ready . body)
  `(call-when-ready
     (lambda ()
       ,@body)))

Will Farr added a neat example in the same thread...

Making little sub-languages for specialized processing e.g.

(with-vectors (v1 v2 v3) (v1 <- (+ v2 v3)))

for summing up the vectors v2 and v3 and storing it in v1. (I'm not going to put this macro up because it's long---the code is buried in this post: http://wmfarr.blogspot.com/2005/06/bigloo-macros-and-repl.html .)

[This] involves changing the evaluation rules for a piece of code (the vector assignment is evaluated once for each index with the corresponding variables bound to elements of a vector).Procedures would not work for either of them; you have to have a macro.