libcwal  Tech-note 45c756c351

As mentioned in a post a couple days ago, the Holy Grail cleanup algorithm cwal has been searching for since almost two years was validated today :-D. The original algorithm, as described in the prior post, is in place and (with only a small bit of pointer fudgery) cleans up all script-unreachable values. This operation is more time-expensive than a sweep-up (potentially by orders of magnitude), but it is "complete," in that it cleans up any values in the current scope which a script cannot see. (Sidebar: that could be bad for C-side bindings if they're not careful to ensure that sure their Value instances are reachable via a variable reference. FIXME: we need a safe place to stash such values outside of a script's visibility. That said, all the current native plugins react just fine to this change, but i can conceive code which would be ill-behaved.)

Anyway...

Tonight i replaced th1ish's auto-sweep (which runs every few expressions, when it is safe to do so) with the vacuum algorithm and, after some relatively small amount of debuggering, it's working beautifully. All existing unit tests still run, and even valgrind is happy (which is the real test, as the engine is a huge nest of pointers doing all sorts of tricky things).

Interestingly, only about 1 in 15 vacuums actually cleans up more values than a sweep operation does (peaking at just over 2 times as many more in some of the existing unit tests). Because sweeping is much cheaper than vacuuming, this implies that further optimization is in order to use sweep most of the time and run a vacuum only occasionally (say once in 5-10 sweep-ups or so). The peak memory usage is, for these test scripts, practically unchanged (+/- 500 bytes, depending on the script), arguing for sweep over vacuum (since sweeping is much cheaper), but vacuuming can catch all those pesky cycles. (That said, for the types of scripts th1ish is really intended for (library test script bindings), vacuuming should never be necessary. But it's fun to work on nonetheless.)

However, those optimizations are for tomorrow evening. Tonight, i celebrate!

Happy Hacking!