In Racket, why does parameterize not work lexically with shift/reset?

Question

Consider the following piece of code:

(define p (make-parameter 'outer))
(reset
  (parameterize ([p 'inner])
    (displayln (p)) ; prints 'inner
    (shift k
      (begin
        (displayln (p)) ; prints 'outer (!!)
        (k 'done)))))

I would expect it to print out "inner" twice, because both calls to (displayln (p)) are inside the parameterize form. However, the output I got was actually

inner
outer
'done.

Why is this the case? Is there a way to get the expected behavior?

Answer 1

This is a tricky and subtle interaction between how parameters work, continuation frames and the implementation of reset/shift.

A parameter gets it value like so:

In a non-empty continuation, a parameter’s value is determined through a parameterization that is associated with the nearest enclosing continuation frame via a continuation mark (whose key is not directly accessible).

The body of the shift does not execute in the same continuation frame as the rest of the reset and the inner parameterize. Therefore, the inner parameter value does not apply in it.

A version using the low-level call-with-continuation-prompt and abort-current-continuation that are used to implement reset and shift might help clarify (this is a simplified version closer to control than shift but the principal is the same):

(define (demo)
  (define tag (default-continuation-prompt-tag))
  (define p (make-parameter 'outer))
  (call-with-continuation-prompt
   (thunk
    (parameterize ([p 'inner])
      (printf "inside c/cp: ~S~%" (p))
      (call-with-current-continuation
       (lambda (k)
         (abort-current-continuation
          tag
          (thunk
           (printf "inside call/cc: ~S~%" (p))
           (k 'done))))
       tag)
      (printf "after call/cc: ~S~%" (p))))
   tag
   ;; the default abort handler made explicit
   (lambda (abort-thunk) (call-with-continuation-prompt abort-thunk tag))))

Running it:

> (demo)
inside c/cp: inner
inside call/cc: outer
after call/cc: inner

The expressions in the shift are passed as a function of zero arguments to the abort handler, which calls it outside of the frame containing the inner parameterize, so the outer value is the one used. If the shift continuation is invoked, control jumps back inside that frame, and the inner value is once again applied.