Problems with Erlang NIF and threads

Question

I have a little problem with threads in Erlang NIFs. You can view my code here: http://pastebin.com/HMCj24Jp. The problem is that when I starting the thread it takes some arguments and starts the generate_binary function. This is okay but when I'm trying to read the arguments everything crashes.

It's perhaps not the most complex problem, but I could not find any documentation about this so I hope some of you might know the answer.

Answer 1

Your generate_buffer() NIF is creating a thread to call generate_binary() but the calling NIF doesn't wait for the newly-created thread to finish. The thread just gets created and likely is still running by the time the NIF returns, though this will be nondeterministic, as threads are in general. You're probably crashing the Erlang BEAM emulator because generate_binary() is off trying to call into the Erlang run-time system after generate_buffer() has returned, confusing the poor thing horribly.

Now, even assuming you fix this to make it do what you wanted, I don't think you should be using explicit native threads here at all.

First, Erlang NIFs are supposed to look like regular Erlang functions, differing only in that they happen to be written in a different language. Erlang functions don't spawn separate threads of execution, then return, leaving that thread running. Excepting those that deal with I/O and persistent data storage, Erlang functions are deterministic and referentially transparent. Your NIF is neither. So, even if it worked, it's still "wrong" in the sense that it violates an experienced Erlang programmer's expectations.

Second, if you need multiprocessing, Erlang already provides the idea of processes. If your NIF will really do so much work that it can benefit from multiprocessing, why not rework your NIF so it can work on a subrange of the data, then call it multiple times, once each from a number of Erlang processes? Then you don't need explicit native threads; the BEAM emulator will create the optimal number of threads for you, transparently.

Third, thread creation overhead is going to kill performance if the lifetime of the thread only extends over the course of a single Erlang NIF call, as it seems you actually intended. This is another reason Erlang processes will be more efficient here.