Why do list-tail raise-exception? Why do we haven't closure property with respect to cdr?

If you evaluate (list-tail '(1 2) 3) at guile scheme. You will get an exception. It would be smarter to have an '() as answer. Overall why do we haven't closure property with respect to cdr combinator? What complications may arise?

Examples to make my point clearer Now (cdr (cdr (cdr '(1 2))) -> raise-exception Should be (cdr (cdr (cdr ... (cdr '(1 2))...))) -> ()

Then we would automatically have properly working list-tail

(define (list-tail list n) 
  (if (= n 0)
      list
      (list-tail (cdr list) (- n 1)))

Group-by then could be written elegantly and exceptionless

(define (group-by list-arg n)
  (if (null? list-arg)
      '()
      (cons (list-head n) (list-tail n))))

Solution

The historic answer is that:

Originally, the Lisp 1 and 1.5 languages created by John MacCarthy did not allow (CDR NIL). The CDR function required a cons cell argument.
The idea that it would be convenient for (CDR NIL) to just return NIL came from a dialect called Interlisp (but may have been present elsewhere).
In the 1960's, there was another major dialect of Lisp called MacLisp (two decades before the Apple Mac, unrelated).

According to The Evolution of Lisp by Peter Gabriel and Guy Steele, some MacLisp people held a pow-wow with Interlisp people in 1974:

In 1974, about a dozen persons attended a meeting at MIT between the MacLisp and Interlisp implementors, including Warren Teitelman, Alice Hartley, Jon L White, Jeff Golden, and Guy Steele. There was some hope of finding substantial common ground, but the meeting actually served to illustrate the great chasm separating the two groups, in everything from implementation details to overall design philosophy. [...] In the end only a trivial exchange of features resulted from “the great MacLisp/Interlisp summit”: MacLisp adopted from Interlisp the behavior (CAR NIL) → NIL and (CDR NIL) → NIL, and Interlisp adopted the concept of a read table.

Both Interlisp and MacLisp are ancestral dialects to Common Lisp, which also has the forgiving car and cdr.

Further remarks are made in the above paper on this matter, begining with:

The adoption of the Interlisp treatment of NIL was not received with universal warmth.

You can see from this fifty, sixty years ago, Lisp people were already divided into camps, and didn't agree on everything. Whether the car of an empty list should just yield the empty list, or error out is a very old issue.

Ashwin Ram, presently director of AI at Google, put in his own opinion in favor of forgiving cdr in 1986, when he composed this poem.

It still remains a divisive issue that is a matter of opinion.

It is undeniable that the flexible car, cdr and their derivatives can help you "code golf" list processing code.

It's also true that such code-golfed code sometimes handles only happy cases without error checking, which can cause problems in some circumstances.

For instance, some list that is assumed to always have three items is subject to (caddr list) to get the third item. But, due to some bug, it has only two. Now the code just ran off with the nil value, which may cause a problem somewhwere else. For instance, suppose the value is expected to be a string, and in some totally different function elsewhere, nil is passed to some API that needs a string and blows up. Now you're hunting through the code to discover where this nil came from.

People who write Lisp interpreters or compilers that rely on the forgiving destructuring performed by car and cdr end up producing something that accepts bad syntax silently.

For instance

(defun interpret-if (form env)
  (let ((test (car form))
        (then (cadr form))
        (else (caddr form)))
   (if (interpret-expr test env)
     (interpret-expr then env)
     (interpret-expr else env))))

This is actually a very nice example for discussing both sides of the issue.

On the one hand, the code is succinct, and nicely supports the optional else clause: the user of this interpreter can do:

(if (> x y)
  (print "x is greater than y"))

In interpret-if, the else variable will pull out a nil, and that will get handed off to (eval expr else env) where it just evaluates to nil, and everything is cool; the optionality of else was obtained from free thanks to caddr not complaining.

On the other hand, the interpreter doesn't diagnose this:

(if) ;; no arguments at all

or this:

(if (> x y))  ;; spec says "then" is required, but no error!

However, all these issues have nice solutions and ways of working that don't require tightening up the list accessor functions, so that we can resort to the succinct coding when we need to. For instance, the interpreter could use pattern matching of some kind, like Common Lisp's rudimentary destructuring-bind:

(defun interpret-if (form env)
  (destructuring-bind (test then &optional else) form
     (if (interpret-expr test env)
       (interpret-expr then env)
       (interpret-expr else env))))

destructuring-bind has strict checking. It generates code with car, caddr and other functions under the hood, but also error checking code. The list (1 2 3) will not be destructured by the pattern (a b).

You have to look at the entire language and how it is used and what else is in it.

Introducing forgiving car and cdr into Scheme might give you less mileage than you think. There is another issue, which is that the only Boolean false value in Scheme is #f. The empty list () in Scheme is not false.

Therefore, even if car is forgiving, code like this cannot work.

Suppose the third element of a list is always a number, or else it doesn't exist. In Lisp, we can do this to default to zero:

(or (third list) 0)

for that to work in Scheme in the default 0 case, (third list) would have to return the Boolean false value #f.

A plausible approach might be to have different default values for car and cdr:

(car ()) -> #f
(cdr ()) -> ()

However, that is rather arbitrary: it works in some circumstances, but fails in situations like:

;; if list has more than two items ...
(if (cddr list) ...)

If cddr returns () by default, then that is always true, and so the test is useless. Different defaulting for car and cdr would probably be more error prone than common defaulting.

In Lisp, the forgiving list accessors work in a synergistic way with the empty list being false, which is why once upon a time I was quite surprised to learn that the forgiving list accessors came in fairly late into the game.

Early Scheme was implemented as a project written in Lisp, and therefore to interoperate smoothly with the host language, it used the same convention: () being NIL being the empty list and false. This was eventually changed, and so if you're wishing to have that back, you're asking for Scheme to revert a many-decades-old decision which is next to impossible now.

Object-oriented programming weighs in on this also. The fact that (car nil) does something instead of failing is an instance of the Null Object Pattern, which is something useful and good. We can express this in the Common Lisp object system, in which it practically disappears:

Suppose we had a car function which blows up on non-conses. We could write a generic function kar which doesn't, like this:

;; generic fun
(defgeneric kar (obj))

;; method specialization for cons class: delegate to car.
(defmethod kar ((obj cons))
  (car obj))

;; specialization for null class:
(defmethod kar ((obj null))) ;; return nil

;; catch all specialization for any type
(defmethod kar ((obj t))
  :oops)

Test:

[1]> (kar nil)
NIL
[2]> (kar '(a . b))
A
[3]> (kar "string")
:OOPS

In CLOS, the class null is that class whose only instance is the object nil. When a method parameter specializes to null, that method is only eligible when the argument for that parameter nil.

The class t is the superclass of everything: the top of the type spindle. (There is a bottom of the type spindle also, the class named nil, which contains no instances and is a subclass of everything.)

Specializing methods on null lets us catch method calls with nil parameters. Thanks to CLOS multiple dispatch, these can be thus handled in any parameter position. Because of that, and null being a class, the Null Object Pattern disappears in CLOS.

If you're debating Lisp with OOP people, you can present (car nil) can be spoken about as being the Null Object pattern.

The inconvenience of explicit null handling is recognized in numerous newer programming languages.

A common feature nowadays is to have null safe object access. For instance

foo.bar

might blow up if foo is null. So the given language provides

foo?.bar

or similar, which will only dereference .bar if foo isn't nil, otherwise the expression yields nil.

When languages add foo?.bar, they do not throw away foo.bar, or make foo.bar behave like foo?.bar. Sometimes you want the error (foo being nil is a programming error you want to catch in testing) and sometimes you want the default.

Sometimes you want the default, so you can collapse multiple levels of default and catch an error:

if (foo?.bar?.xyzzy?.fun() == nil) {
  // we coudn't have fun(); handle it
  // this was because either foo was nil, or else bar was nil,
  // or else xyzzy was nil, or else fun() returned nil.
} else {
  // happy case
}