Note 8, Programming Language Concepts (sestoft@dina.kvl.dk) 2002-04-02 *** ---------------------------------------------------------------------- Tail-calls and tail-recursive functions --------------------------------------- A recursive function is one that may call itself. For instance, the factorial function n! = 1 * 2 * ... * n may be implemented by a recursive function facr as follows: fun facr n = if n=0 then 1 else n * facr(n-1); A call to a function is a tail call if it is the last action of the calling function. For instance, the call from f to itself here is a tail call: fun f x = if x=0 then 17 else f(n-1) and the call from f to g here is a tail call: fun f x = if x=0 then g 8 else f(n-1) The recursive call from facr to itself (above) is not a tail call. When evaluating the else-branch n * facr(n-1) we must first compute facr(n-1), and when we are finished with that and have obtained a result v, then we must compute n * v and return to the caller. Thus the call facr(n-1) is not the last action of facr; after the call there is still some work to be done (namely the multiplication by n). The evaluation of facr 3 requires a certain amount of stack space to remember then outstanding multiplications by n: facr 3 ==> 3 * facr 2 ==> 3 * (2 * facr 1) ==> 3 * (2 * (1 * facr 0)) ==> 3 * (2 * (1 * 1)) ==> 3 * (2 * 1) ==> 3 * 2 ==> 6 Remembering the `work still to be done' after the call requires some space, and therefore a computation of facr(N) requires space proportional to N. On the other hand, consider this alternative definition of factorial: fun faci n r = if n=0 then r else faci (n-1) (r * n); An additional parameter r has been introduced to hold the result of the computation, with the intention that (faci n 1 = facr n) for all non-negative n. The recursive call faci (n-1) (r * n) to faci is a tail-call, and the function is said to be tail-recursive or iterative. There is no `work still to be done' after the recursive call, as shown by this computation of faci 3 1: faci 3 1 ==> faci 2 3 ==> faci 1 6 ==> faci 0 6 ==> 6 Indeed, most implementations of functional languages, including Moscow ML, execute tail-calls in constant space. Most implementations of imperative and object-oriented languages (C, C++, Java, C#, ...) do not care to implement tail calls in constant space. Thus the equivalent C or Java or C# method declaration: static int faci(int n, int r) { if (n == 0) return r; else return faci(n-1, r * n); } would most likely not execute in constant space. This could be seen clearly in the stack picture for the execution of recursive factorial (example imp/ex9.c) in micro-C in lecture 7. Imperative languages do not have to care as much about performing tail calls in constant space because they provide for- and while-loops to express iterative computations in a natural way. Thus the function faci would be expressed more naturally like this: static int faci(int n) { int r = 1; while (n != 0) { r = n * r; n = n - 1; } return r; } Which calls are tail calls? --------------------------- What does it mean that a call `is the last action' of the calling function? Let us consider a small eager (call-by-value) functional language, with the following forms of expressions: expr ::= i constant | x variable | let x = er in eb end let-binding | e1 + e2 base operator | if e1 then e2 else e3 conditional | let f x = er in eb end function binding | f e function call If we assume that the whole expression is in tail position, its subexpressions are resp. are not in tail position, as follows: Expression Status of subexpressions ------------------------------------------------------------ let x = er in eb end eb is in tail position, er is not e1 + e2 neither e1 nor e2 is in tail position if e1 then e2 else e3 e2 and e3 are in tail position, e1 is not let f x = er in eb end eb is in tail position, er is not f e e is not in tail position If an expression is not in tail position, then none of its subexpressions is in tail position. A tail call is a call in tail position. Thus all the calls to g below are tail calls, whereas those to h are not: g 1 g(h 1) h 1 + h 2 if 1=2 then g 3 else g(h 4) let x = h 1 in g x end let x = h 1 in if x=2 then g x else g 3 end let x = h 1 in g(if x=2 then h x else h 3) end let x = h 1 in let y = h 2 in g(x + y) end end Continuations and continuation-passing style (CPS) -------------------------------------------------- A continuation k is an explicit representation of `the rest of the computation', typically in the form of a function from the value of the current expression to the result of the entire computation. A function in continuation-passing style (CPS) takes an extra argument, the continuation k, which `decides what will happen to the result of the function'. To see a concrete function in continuation-passing style, consider again the recursive factorial function facr: fun facr n = if n=0 then 1 else n * facr(n-1); To write this function in continuation-passing style, we give it a continuation parameter k: fun facc n k = if n=0 then ?? else ?? Usually the then-branch would just return 1. In continuation-passing style, it should not return but instead give the result 1 to the continuation k, so it should be: fun facc n k = if n=0 then k 1 else ?? Now consider the else-branch: n * facr(n-1) The continuation for the else-branch is the same as that for the original function call (facc n k), that is, k. But what is the continuation for the recursive call facr(n-1)? It must be a function that accepts the result v of the recursive call, computes n * v, and then passes the result to the continuation of the entire else-branch. Thus the continuation of the recursive call can be expressed like this: fn v => k(n * v) so this is the factorial function in continuation-passing style: fun facc n k = if n=0 then k 1 else facc (n-1) (fn v => k(n * v)); If we define the identity function id : 'a -> 'a by fun id v = v; then it holds that (facr n = facc n id) for all non-negative n. Note that the resulting function facc is tail-recursive; in fact this will always be the case. This does not mean that the function now magically will run in constant space where previously it did not: the continuations will have to be created and stored until they are applied. But sometimes one can represent the action of the continuation very compactly, by a non-function, to obtain a constant-space tail-recursive function. In the case of facc, all a continuation ever does is to multiply its argument by some number, so why not simply represent it by that number? Then we get the iterative faci function! Things do not usually work out as neatly, though. Continuations were invented (or discovered?) independently by a number of people around 1970; see Reynolds's 1993 paper. The name is due to Christopher Wadsworth, a student of Christopher Strachey. CPS transformation ------------------ (Skip this if you like) There is a systematic transformation which can transform any expression or function into continuation-passing style (CPS). This transformation was discovered independently by Fischer (1972) and Plotkin (1975). The transformation (for eager or call-by-value languages) is easily expressed on the pure untyped lambda calculus because it has only three different syntactic constructs: variable x, function fn x => e, and function call e1 e2. Let [[e]] denote the CPS-transformation of the expression e, then: [[x]] is fn k => k x [[fn x => e]] is fn k => k (fn x => [[e]]) [[e1 e2]] is fn k => [[e1]] (fn m => [[e2]] (fn n => m n k)) It is somewhat more cumbersome to express the CPS transformation for SML or even for our higher-order functional example language. A lucid analysis and several improvements of the original CPS transformation can be found in Danvy and Filinski (1992). Interpreters in continuation-passing style ------------------------------------------ The interpreters we have considered in this course have been written as SML functions, and therefore they too can be rewritten in continuation-passing style. When an interpreter for a functional language is written in continuation-passing style, a continuation is a function from the value of an expression to the `answer' or `final result' of the entire computation of the interpreted program. When an interpreter for an imperative language is written in continuation-passing style, a continuation is a function from a store (created by the execution of a statement) to the `answer' (the `final store') produced by the entire computation. In itself, rewriting the interpreter (eval or exec function) in continuation-passing style achieves nothing. The big advantage is that by making `the rest of the computation' explicit as a continuation, interpreter is free to ignore the continuation and return a different kind of `answer'. This is useful for modelling the throwing of exceptions and similar abnormal termination. A continuation-passing interpreter for a functional language ------------------------------------------------------------ We now consider our simple functional language, extended with exceptions, an expression Raise exn that raises exception exn, and an expression Handle(e1, exn, e2) that evaluates e1 and returns its value if e1 does not raise any exception; if e1 raises exception exn, then it evaluates e2; and if e1 raises another exception exn', then the entire expression raises that exception. The expression Raise exn corresponds to the SML expression raise exn which is similar to the Java statement throw exn; The expression Handle(e1, exn, e2) corresponds to the SML expression e1 handle exn => e2 which is similar to the Java statement try { e1 } catch (exn) { e2 } The abstract syntax of our small functional language is extended as follows: datatype exn = Exn of string datatype expr = ... | Raise of exn | Handle of expr * exn * expr For now we consider only the raising of exceptions. In an interpreter for this language, a continuation may be a function k : int -> answer where: datatype answer = Success of int | Failure of string The continuation cont is called the normal (or success) continuation. It is passed to the evaluation function coEval1, which must apply the continuation to any normal result it produces. But when coEval1 evaluates (Raise exn) it may just ignore the continuation, and return (Failure s) where s is some message derived from exn. This way we can model abnormal termination of the interpreted object language program: fun coEval1 (e : expr) (env : vfenv) (cont : int -> answer) : answer = case e of CstI i => cont i | CstB b => cont (bool2int b) | Var x => (case lookup env x of Int i => cont i | _ => Failure "coEval1 Var") | ... | Raise (Exn s) => Failure s This allows the interpreted object language program to raise exceptions without using exceptions in the interpreter (the meta language). To allow object language programs to also handle exceptions, not only raise them, we add yet another continuation argument econt to the interpreter, called the abnormal (or failure) continuation. The failure continuation expects to receive an exception value, and will look at the value to decide what action to take: handle the exception, or pass it to an older failure continuation. More precisely, to evaluate Handle(e1, exn, e2) the interpreter will create a new error continuation econt1. If the evaluation of e1 does not throw an exception, then the success continuation will be called as usual and the error continuation will be ignored. However, if evaluation of e1 throws an exception exn1, then the new error continuation will be called and will look at exn1, and if it matches exn, then it will evaluate e2; otherwise it will pass exn1 to the outer error continuation econt, thus propagating the exception: fun coEval2 (e : expr) (env : vfenv) (cont : int -> answer) (econt : exn -> answer) : answer = case e of CstI i => cont i | CstB b => cont (bool2int b) | ... | Raise exn => econt exn | Handle(e1, exn, e2) => let fun econt1 exn1 = if exn1 = exn then coEval2 e2 env cont econt else econt exn1 in coEval2 e1 env cont econt1 end File cont/fun.sml gives all details of the two continuation-passing interpreters coEval1 and coEval2 for a functional language. The former implements a language where exceptions can be thrown but not handled, and the latter one implements a language where exceptions can be thrown as well as handled. Tail position and continuation-passing interpreters --------------------------------------------------- Note that expressions in tail positions are exactly those that are interpreted with the same continuations as the enclosing expression. Consider for instance let-binding: fun coEval1 (e : expr) (env : vfenv) (cont : int -> answer) : answer = case e of ... | Let(x, erhs, ebody) => coEval1 erhs env (fn xval => let val env1 = bind1 env (x, Int xval) in coEval1 ebody env1 cont end) where the let-body ebody is evaluated with the same continuation cont as the entire let-expression. This is no coincidence: a subexpression has the same continuation as the enclosing expression exactly when evaluation of that subexpression is the last action of the enclosing expression. A continuation-passing interpreter for an imperative language ------------------------------------------------------------- An imperative language with exceptions, a throw statement and a try-catch statement (as in C++, Java, and C#) can be modelled in precisely the same way. Let the abstract syntax be given by: datatype stmt = ... | Throw of exn | TryCatch of stmt * exn * stmt An interpreter that implements throw and try-catch must take a normal continuation cont as well as an error continuation econt. The error continuation must take two arguments: an exception and the store that exists when the exception is thrown. Usually the interpreter applies cont to the store resulting from some command, but when executing a throw statement it applies econt to the exception and the store. When executing a try-catch block the interpreter creates a new error continuation econt1; if called, that error continuation decides whether it will handle the exception exn1 given to it and execute the handler body stmt2, or pass the exception to the outer error continuation, thus propagating the exception: fun coExec2 stmt sto (cont : sto -> answer) (econt : exn * sto -> answer) : answer = case stmt of Asgn(x, e) => cont (set sto (x, eval e sto)) | If(e1, stmt1, stmt2) => if int2bool(eval e1 sto) then coExec2 stmt1 sto cont econt else coExec2 stmt2 sto cont econt | ... | Throw exn => econt(exn, sto) | TryCatch(stmt1, exn, stmt2) => let fun econt1 (exn1, sto1) = if exn1 = exn then coExec2 stmt2 sto1 cont econt else econt (exn1, sto1) in coExec2 stmt1 sto cont econt1 end In summary, the execution of a statement stmt by coExec2 coExec2 stmt sto cont econt can terminate in two ways * if the statement stmt executes normally, without throwing an exception, then its execution ends with calling the success continuation cont on a new store sto1: cont sto1 * otherwise, if the execution of stmt throws an exception exn1, then its execution ends with calling the error continuation econt on exn1 and a new store sto1: econt (exn1, sto1) Any handling of the exception is left to econt. File cont/imp.sml shows two continuation-passing interpreters for an imperative language, coExec1 and coExec2. The former implements exceptions using a single (success) continuation, and the latter implements throwable and catchable exceptions using two continuations: one for computations that terminate normally, and one for computations that throw an exception. Note that an expression cannot throw an exception in the imperative language modelled here. If it could, then the expression evaluator would have to be written in continuation-passing style too, and would have to take two continuation arguments: a normal continuation of type (value -> answer) and an error continuation of type (exn -> answer). Provided expressions have no side effects on the store, we can omit the store parameter to these expression continuations, because we can build the store into the continuation. The statement interpreter would have to pass suitable continuations to the expression interpreter, for instance: fun coExec2 stmt sto (cont : sto -> answer) (econt : exn * sto -> answer) : answer = case stmt of Asgn(x, e) => eval e sto (fn v => cont (set sto (x, v))) (fn exn => econt (exn, sto)) | ... The evaluation stack and continuations -------------------------------------- As shown in lecture 7, a micro-C program can be compiled to instructions that are subsequently executed by an abstract machine. In the abstract machine, the evaluation stack represents the (success) continuation. Namely, the return address in the stack frame says what instructions the continuation must execute, and the stack frame provides values for the variables used by those instructions (the continuation's free variables, actually). How could we represent an error continuation, for exception handling, in the stack machine? One approach is to store exception handler descriptions in the evaluation stack and introduce an additional exception handler register hr. The exception handler register hr is the index (in the stack) of the most recent exception handler description, or -1 if there is no exception handler. For this discussion, let us assume that an exception is represented simply by an integer (rather than an object as in Java or C#, or a value of the special type exn as in SML). An exception handler description (in the stack) has three parts: * the identity exn of the exception that this handler handles; * the address a of the associated handler block, that is, the code of the catch block; * a pointer to previous exception handler description (further down in the stack), or -1 if there is not previous exception handler. Thus the exception handler descriptions form a list with the most recent exception handler first, pointed to by hr. Older exception handler descriptions are found by following the pointer to the previous exception handler. This list can be thought of as a stack representing the error continuation; this stack is simply merged into the usual evaluation stack. A try-catch block try stmt1 catch (exn) stmt2 is compiled to the following code push exception handler description (exn, code address for stmt2, hr) code for stmt1 pop exception handler description L: The code for stmt2 must end with GOTO L where L is a label following the code for popping the exception handler description. The execution of throw exn must look through the chain of exception handlers in the stack until it finds one that will handle the thrown exception exn. If it does find such a handler (exn, a, h) it will pop the evaluation stack down to the point just below that handler, set hr to h, and set pc to a, thus executing the code for the associated exception handler (catch block) at address a. The popping of the evaluation stack may mean that many stack frames for unfinished function calls (above the exception handler) will be thrown away, and execution will continue in the function that declared the exception handler (the try-catch block), as desired. Thus we could implement exceptions in the stack machine by adding instructions PUSHHDLR, POPHDLR, and THROW for pushing a handler, for popping a handler, and for throwing an exception, respectively. The instructions for pushing and popping handlers should work as follows: PUSHHDLR exn a s ==> hr:a:exn:s Push old handler register hr, exception name exn, and handler address a; also set handler register hr to the address of hr in the stack. POPHDLR h:a:exn:s ==> s Pop exception handler description; also set handler register hr to h. The instruction THROW exn for executing the statement throw exn; should work as follows: while (hr != -1 && s[hr] != exn) hr = s[hr+2]; // Try the next exception handler if (hr != -1) { // Found a handler for exception exn pc = s[hr+1]; // execute the associated handler code hr = s[hr+2]; // with current handler being hr sp = hr-1; // after popping all frames above handler } else { print "uncaught exception exn"; stop machine; } Continuations and tail calls ---------------------------- If a function call is in tail position, then the continuation cont of the call is the same as the continuation of the entire enclosing function body. Moreover, the called function's body is evaluated in the same continuation as the func The continuation of a tail-called function is the same as that of the calling function... fun coEval1 (e : expr) (env : vfenv) (cont : int -> answer) : answer = ... | Call(f, earg) => (case lookup env f of fclosure as RClo (f, x, fbody, env1) => coEval1 earg env (fn argv => let val env2 = bind1 env1 (f, fclosure) val env3 = bind1 env2 (x, Int argv) in coEval1 fbody env3 cont end) | _ => Failure "coEval1 Call") In lecture 9 we shall see how one can compile micro-C tail calls so that a function can an arbitrary number of tail-recursive calls in constant space (example imp/ex12.c): int f(int n) { if (n) return f(n-1); else return 17; } The trick is to discard f's old stack frame, which contains the values of its local variables and parameters, such as n, and replace it by the called function's new stack frame. Only the return address and the old base pointer must be retained from f's old stack frame. It is admissible to throw away f's local variables and parameters because there is no way they could be used after the recursive call has returned (the call is the last action of f). This works also when one function f calls another function g by a tail call: then f's old stack frame is discarded (except for the return address and the old base pointer), and is replaced by g's new stack frame. Our stack machine has a special instruction for tail calls: TCALL m n a that discards n old variables from the stack frame, pushes m new arguments, and executes the code at address a. It does not push a return address or adjust the base pointer, so basically a TCALL is a specialized kind of jump (GOTO). Callcc: call with current continuation, and setjmp/longjmp ---------------------------------------------------------- (Skip this if you like) In some languages, notably Scheme and the New Jersey implementation of Standard ML (SML/NJ), one can capture the current continuation k using SML/NJ: callcc (fn k => ...) Scheme: (call-with-current-continuation (lambda (k) ...)) and then discard it, or apply to an argument e using SML/NJ: throw k e Scheme: (k e) This can be exploited in very powerful but often rather mysterious programming idioms. Example, in SML/NJ: open SMLofNJ.Cont; 1 + callcc (fn k => 2 + 5) evaluates to 1 + (2 + 5) 1 + callcc (fn k => 2 + throw k 5) evaluates to 1 + 5 In both cases, the continuation captured as k is the continuation that says `add 1 to my argument'. The corresponding examples in Scheme work precisely the same way: (+ 1 (call-with-current-continuation (lambda (k) (+ 2 5)))) (+ 1 (call-with-current-continuation (lambda (k) (+ 2 (k 5))))) In a sense, the setjmp function in C captures the current continuation (like callcc) and the corresponding function longjmp reactivates it (like throw). This is useful for implementing a kind of exception handling in C programs, but C's notion of continuation is much weaker than that of Scheme or SML/NJ. In fact, the C implementation of setjmp just stores the current machine registers, including the stack pointer, in a structure. Applying longjmp to that structure will restore the machine registers, including the stack pointer. The effect will that program execution continues at the point where setjmp was called -- exactly as in the SML/NJ and Scheme examples above. However, when the function that called setjmp returns, the stack will be truncated below the point at which the stored stack pointer points. Calling longjmp after this has happened may have strange effects (most likely the program crashes), since the restored stack pointer now points into a part of memory where there is no longer any stack, or where possibly a completely unrelated stack frame has been stored. Literature ---------- * Danvy and Filinski: Representing control. A study of the CPS transformation. In Mathematical Structures in Computer Science 2 (1992) 361-391. * Reynolds: Definitional interpreters for higher-order languages. 1972. Reprinted in Higher-order and symbolic computation 11, 4 (December 1998) 363--397. Shows how to use continuation-passing style to make the object language evaluation order independent of the meta-language evaluation order. * Strachey and Wadsworth: Continuations, a mathematical semantics for handling full jumps. Written 1974, reprinted in Higher-order and symbolic computation 13 (2000) 135-152. * Reynolds: The discoveries of continuations. Lisp and Symbolic Computation 6, 3/4 (1993) 233-247. * Guy Lewis Steele: LAMBDA: The Ultimate Declarative. MIT AI Lab memo AIM-379, November 1976. ftp://publications.ai.mit.edu/ai-publications/0-499/AIM-379.ps Shows that if functions (lambdas) and function calls are implemented properly, via continuation-passing style, then all other constructs can be implemented efficiently in terms of these. This idea was realized in the Rabbit Scheme compiler; see below. (Guy Steele is a co-designer of the Java programming language.) * Guy Lewis Steele: RABBIT: A Compiler for SCHEME (A Study in Compiler Optimization). MIT AI Lab technical report AITR-474, May 1978. ftp://publications.ai.mit.edu/ai-publications/0-499/AITR-474.ps Rabbit is the first compiler to use continuation-passing style; it was a breaktrhough in efficiency of functional language implementations. * Appel: Compiling with Continuations, Cambridge University Press 1992. Describes the design of the SML/NJ (Standard ML of New Jersey) compiler which initially transforms the entire program into continuation-passing style, as suggested by Steele.