How to make perl thread without copying all variables?

Question

I have one perl program, where using some form of parallelism would really be helpful.

However, I have quite a lot of data in variables, that I don't need at all at that part of the program.

If I use perl threads, it copies all variables every time I create a new thread. In my case, that hurts a lot.

What should I use to make a new thread without the copying? Or are there some better thread implementations, that don't copy everything?

Answer 1

Like the syntax an ease of threads but not all the fat? Use the amazing forks module! It implements the threads interface using fork and IPC making it easy to share data between child processes.

Answer 2

Really, you just have to avoid ithreads. They're horrible, and unlike every other form of threads on the planet they're more expensive than regular heavyweight processes. My preferred solution is to use an event-based framework like POE or AnyEvent (I use POE) and break out any tasks that can't be made nonblocking into subprocesses using POE::Wheel::Run (or fork_call for AnyEvent). It does take more up-front design work to write an app in that manner, but done right, it will give you some efficient code. From time to time I've also written code that simply uses fork and pipe (or open '-|') and IO::Select and waitpid directly within its own event loop, but you should probably consider that a symptom of my having learned C before perl, and not a recommendation. :)

A word to the wise, though: if you're running on Windows, then this approach might be almost as bad as using ithreads directly, since Perl makes up for win32's lack of fork() by using ithreads, so you'll pay that same ithread-creation cost (in CPU and memory) on every fork. There isn't really a good solution to that one.

Answer 3

Use the fork(2) system call to take advantage of Copy-on-write.