I am wondering what this error might mean:
PicklingError: Can't pickle <type 'function'>: attribute lookup __builtin__.function failed
I understand that it has something to do with using multiple cores. I am running my program on a cluster and using 15 threads in this line of my code:
gauss2 = PTSampler(ntemps, renwalkers, rendim, lnlike, lnprior, threads=15)
The sampler in question is the one documented for the Parallel Tempering sampler at http://dan.iel.fm/emcee/current/user/pt/
Any idea what this error might mean?
With pickle protocol v1, you cannot pickle open file objects, network connections, or database connections.
Pickle is able to serialize and deserialize Python objects into bytestream. It does work well on most cases — with reservations. When multiprocessing spawns a process, Pickle is called by default to handle data transfer. A simple example of how this is done follows: import pickleclass Foo(object):
To solve this type of error we have to declare that variable as global variables. We can't pickle local objects so that we are declaring that variable result as global. Here we have given only one print statement. If it is declared then we can say that the program has run successfully.
multiprocessing is a package that supports spawning processes using an API similar to the threading module. The multiprocessing package offers both local and remote concurrency, effectively side-stepping the Global Interpreter Lock by using subprocesses instead of threads.
The error means you are trying to pickle a builtin FunctionType
… not the function itself. It's likely do to a coding error somewhere picking up the class of the function instead of the function itself.
>>> import sys
>>> import pickle
>>> import types
>>> types.FunctionType
<type 'function'>
>>> try:
... pickle.dumps(types.FunctionType)
... except:
... print sys.exc_info()[1]
...
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> def foo(x):
... return x
...
>>> try:
... pickle.dumps(type(foo))
... except:
... print sys.exc_info()[1]
...
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> try:
... pickle.dumps(foo.__class__)
... except:
... print sys.exc_info()[1]
...
Can't pickle <type 'function'>: it's not found as __builtin__.function
>>> pickle.dumps(foo)
'c__main__\nfoo\np0\n.'
>>> pickle.dumps(foo, -1)
'\x80\x02c__main__\nfoo\nq\x00.'
If you have a FunctionType
object, then all you need to do is get one of the instances of that class -- i.e. a function like foo
.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With