Prevent overwriting modules in Julia parallelization

Question

I've written a Julia module with various functions which I call to analyze data. Several of these functions are dependent on packages, which are included at the start of the file "NeuroTools.jl."

module NeuroTools

using MAT, PyPlot, PyCall;

function getHists(channels::Array{Int8,2}...

Many of the functions I have are useful to run in parallel, so I wrote a driver script to map functions to different threads using remotecall/fetch. To load the functions on each thread, I launch Julia with the -L option to load my module on each worker.

julia -p 16 -L NeuroTools.jl parallelize.jl

To bring the loaded functions into scope, the "parallelize.jl" script has the line

@everywhere using NeuroTools

My parallel function works and executes properly, but each worker thread spits out a bunch of warnings from the modules being overwritten.

WARNING: replacing module MAT
WARNING: Method definition read(Union{HDF5.HDF5Dataset, HDF5.HDF5Datatype, HDF5.HDF5Group}, Type{Bool}) in module MAT_HDF5...
(contniues for many lines)

Is there a way to load the module differently or change the scope to prevent all these warnings? The documentation does not seem entirely clear on this issue.

Answer 1

Coincidentally I was looking for the same thing this morning

(rd,wr) = redirect_stdout()

So you'd need to call

remotecall_fetch(worker_id, redirect_stdout)

If you want to completely turn it off, this will work

If you want to turn it back on, you could

out = STDOUT
(a,b) = redirect_stdout()
#then to turn it back on, do:
redirect_stdout(out)