libcwal  Tech-note 4b560dde33

i've been looking almost two years for this...

Today i found my "holy grail" algo which can find and destroy values not reachable from script code. It was deceptively simple to do. It relies very much on cwal's "upscope on reference to extend lifetime" of values (not variables), and just fudges the scope IDs (higher is newer) to force some upscoping...

To copy/paste the cwal_engine_vacuum() API docs:

       DO NOT USE! This is an experiment.

       This function cleans up all values owned by the current scope
       by determining whether or not they refer to, or are referred to
       by, scope-level properties (variables). The mechanism is
       relatively simple:

       1) Push a new scope onto the stack with the same parent as e's
       current scope, but fake its level to (s->level-1) so that it
       looks like an older scope. We'll call the current scope s1 and
       this new scope s2.

       2) cwal_props_copy() all scope properties from s1 to
       s2. Because s2 looks like an older scope, this will transfer
       any values in s1 which are variables or reachable via them,
       leaving any orphaned values in s1 but not in s2.

       3) clean all values remaining in s1.

       4) re-set s2's parent to be s1 and fake s2's level to
       (s1->level+1) so that s2 looks like a newer scope.

       5) cwal_props_copy() all scope properties from s2 to
       s1. Because of step 4, s1 now looks like a higher/older scope
       to the copy process, which will move the variables, and values
       referenced by them, back into s1. Note that we cannot simply
       move the value lists from s2 to s1 because we need to ensure
       that the value->scope pointers all point to where they need to,
       and the underlying engine does that for us if we just copy the
       values back again.

       6) Clean up scope s2, as if it had been popped from the stack.

       The end result is that after this call (on success), only
       variabes, and values reachable via variable references, will be
       in the scope, all other (presumably script-unreachable) values
       having been cleaned up.

So... this troublesome object will be cleaned up by that operation:

  var i = 3 // refcount = 1, just to have some other value here
  var o = object{} // refcount = 1
  o.(o) = o // refcount = 3 (var, key, val)
  unset o // refcount == 2 (key, val) and it's no longer reachable via any vars. Orphaned!

Vacuum will (indirectly) find and destroy that Object for us without destroying i's value :). Without this algo, only destruction of its owning scope will clean up the orphaned 'o' object.

So far the tests (and think-throughs) look good, but i've got more tests to run, and then to plug it into th1ish and see if it works in real code. In theory it can simply replace th1ish's use of the existing sweep functionality (the "little brother" of vacuum which only cleans up temporaries).

And it required no API changes except to fix two (now) incorrect assertions :-D.

Happy Hacking!