Erlang's slave module: code and IO distribution

Question

We use Erlang with the slave module in an application that is spawning slave nodes on different machines that report to and are coordinated by a master node on another machine. The slave nodes are opening ports on their machines and running some external programs (essentially, the erlang slave nodes (we call them workers) are just fancy wrappers around the external programs).

However, we encountered some unexpected problems for which we have not found good solutions.

1. Code distribution. Currently our Makefile rsyncs the compiled erlang code (the ebin folder) to the machines running the worker nodes and we load it via the -pa argument on worker node startup. There really should be some way of automatically distributing the code at runtime via Erlang but we are not sure how to do it.

2. Logging. The slave module documentation says "All TTY output produced at the slave will be sent back to the master node". However, when we run lager (the basho logger) on our slave (worker) nodes, its output does not get redirected to the master node's tty (there's only the log output from the master node). Currently we have a process running on the master node that logs (via lager) messages it gets from the slave nodes. So to log something on the slave nodes, we send messages to the master node.

We're starting the worker nodes like this:

slave:start(IP, NodeName, NodeArgs)

where NodeArgs is

 -setcookie thecookie -pa /home/dau/mapro/deps/nicedecimal/ebin/ /home/dau/mapro/deps/meck/ebin/ /home/dau/mapro/deps/lager/ebin/ /home/dau/mapro/deps/jsx/ebin/ /home/dau/mapro/apps/worker/ebin/ /home/dau/mapro/ebin/ -s worker_app

where all given paths are absolute paths on the machines running the worker nodes.

Answer 1

You're doing the right thing as far as item #1: you are responsible for installation of the Erlang source modules or precompiled beam files on the remote host machines, just as you are for installation of OTP/Erlang itself. Once they are in the code path on the remote machines you can load and unload and hotswap to you heart's content, though :)

wrt item #2: have you verified that your slave module code can output naive error messages (i.e. erlang:display()) to your master node? in which case the problem is lager configuration - try setting error_logger_redirect to false as a quick test, perhaps... apologies in advance if none of this is useful!