upvar, TclOO, and next - explanation of (perhaps) unexpected behaviour

Question

I'm wondering if anyone can explain the behind-the-scene details of why I can chain upvars successfully in nested procs, but it does not work in nested TclOO methods (those methods overridden in child classes). (I've been told that calling [next] in a TclOO class method acts a bit like a "temporary tailcall", in that a new stack level is not created. Is it this the case? If so, what is the complete picture?)

For example, the following three approaches do not all give the same result:

proc addone {varname} {
  upvar $varname x;
  incr x;
}
proc addanotherone {varname} {
  upvar $varname xx;
  addone xx;
  incr xx;
}

oo::class create C1 {
  method addone {varname} {
    upvar $varname x;
    incr x;
  }
}
oo::class create S1 {
   superclass C1;
   method addone {varname} {
      upvar $varname xx;
      next xx;
      incr xx;
   }
 }
 oo::class create S2 {
   superclass C1;
   method addone {varname} {
     upvar $varname xx;
     next $varname;
     incr xx;
   }
 }
 set s1 [S1 new];
 set s2 [S2 new];
 set y 1;
 addanotherone y;
 set y; # First result gives 3, as expected;
 set y 1;
 $s1 addone y;
 set y; # gives 2, unexpected;
 set y 1;
 $s2 addone y; 
 set y;  #gives 3, unexpected, because original varname seems to be "two levels" deep.

If [next] somehow runs in the same stack level, can it create variables in the callers scope without "uplevel"?

If not, it's not really running at the same level, so is it more like a closure?

I'm interested in the real details on how it differs from, say, a tailcall, the use of uplevel, or whatever other concepts should be considered. Thank you!

Answer 1

The next command internally is a bit like uplevel (specifically uplevel 1) in that it temporarily removes the stack frame of the method calling next while running the superclass implementation, restoring the stack frame when next returns (of course).

This means that you can override methods from superclasses without those superclasses needing to be especially prepared for it. That was a major problem with some other older object systems for Tcl, where you needed a special call to get the depth argument for upvar and uplevel, and it was ever so easy to forget that, so I changed things for TclOO. However, a direct consequence of that change means that what you're doing in S1 » addone won't work; it creates/overwrites an additional variable, xx, in the calling scope. S2 » addone is what I'd consider idiomatic.

If you're passing an internal variable around between a method and the method it overrides — which by definition requires the two to cooperate — use a variable in the object's state namespace; your classes have complete control over that. Or invoke a method via my or [self]; that's a standard method call (with all that implies).