[Core] Prune references in running block to reduce memory footprint after fork

[vsergeev]

Blocks that have forked for running still indirectly hold references to the other blocks in the flow graph, which includes any variables or memory allocated in their instantiate() and initialize() methods.

The memory footprint of a running block can be reduced by pruning these unneeded references.

[vsergeev]

This is a bit difficult to address generally, because references to the other blocks may exist in local variables up the call stack, and there isn't an easy way in Lua to spawn a new function with an empty call stack -- you can create a new thread with coroutine.create(), but I'm not aware of a way to discontinue the main thread to release the references and switch to running the new one. Changing the function environment to a sandboxed one with setfenv() also does not affect the call stack.

It is possible to use the debug library to forcibly nil out any local variables up the call stack, and then prune any references to other blocks through the block's Pipe objects after the fork, like so:

diff --git a/radio/core/composite.lua b/radio/core/composite.lua
index 834b769..55d28b2 100644
--- a/radio/core/composite.lua
+++ b/radio/core/composite.lua
@@ -565,9 +565,32 @@ function CompositeBlock:start(multiprocess)
             end
 
             if pid == 0 then
+                (function (block)
                 -- Create a set of file descriptors to save
                 local save_fds = {}
 
+                local i = 2
+                while true do
+                    local g = _G.debug.getinfo(i)
+                    if not g then
+                        break
+                    end
+
+                    local j = 1
+                    while true do
+                        local name, _ = _G.debug.getlocal(i, j)
+                        if not name then
+                            break
+                        end
+
+                        print("clearing local", i, j, name)
+                        _G.debug.setlocal(i, j, nil)
+                        j = j + 1
+                    end
+                    i = i + 1
+                end
+                print()
+
                 -- Ignore SIGPIPE, handle with error from write()
                 ffi.C.signal(ffi.C.SIGPIPE, ffi.cast("sighandler_t", ffi.C.SIG_IGN))
 
@@ -576,6 +599,7 @@ function CompositeBlock:start(multiprocess)
                     for _, fd in pairs(block.inputs[i]:filenos()) do
                         save_fds[fd] = true
                     end
+                    block.inputs[i].pipe.output = nil
                 end
 
                 -- Save output pipe fds
@@ -583,8 +607,13 @@ function CompositeBlock:start(multiprocess)
                     for _, fd in pairs(block.outputs[i]:filenos()) do
                         save_fds[fd] = true
                     end
+                    for j = 1, #block.outputs[i].pipes do
+                        block.outputs[i].pipes[j].input = nil
+                    end
                 end
 
+                collectgarbage()
+
                 -- Save open file fds
                 for file, _ in pairs(block.files) do
                     local fd = (type(file) == "number") and file or ffi.C.fileno(file)
@@ -620,6 +649,8 @@ function CompositeBlock:start(multiprocess)
 
                 -- Exit
                 os.exit(0)
+
+                end)(block)
             else
                 self._pids[block] = pid
             end

The running function could be further isolated in an sandboxed environment with setfenv(), in case there are any references in _G. This achieves the objective, but using debug.setlocal() in this way is definitely a hack and may have unforeseen consequences.

The best solution is probably creating a new Lua thread or Lua state and finding a way to terminate the main one.