dreaded "not the same object error" pickling a queryset.query object

Question

I have a queryset that I need to pickle lazily and I am having some serious troubles. cPickle.dumps(queryset.query) throws the following error:

Can't pickle <class 'myproject.myapp.models.myfile.QuerySet'>: it's not the same object as myproject.myapp.models.myfile.QuerySet

Strangely (or perhaps not so strangely), I only get that error when I call cPcikle from another method or a view, but not when I call it from the command line.

I made the method below after reading PicklingError: Can't pickle <class 'decimal.Decimal'>: it's not the same object as decimal.Decimal and Django mod_wsgi PicklingError while saving object:

def dump_queryset(queryset, model):
    from segment.segmentengine.models.segment import QuerySet
    memo = {}
    new_queryset = deepcopy(queryset, memo)
    memo = {}
    new_query = deepcopy(new_queryset.query, memo)
    queryset = QuerySet(model=model, query=new_query)    
    return cPickle.dumps(queryset.query)

As you can see, I am getting extremely desperate -- that method still yields the same error. Is there a known, non-hacky solution to this problem?

EDIT: Tried using --noreload running on the django development server, but to no avail.

EDIT2: I had a typo in the error I displayed above -- it was models.QuerySet, not models.mymodel.QuerySet that it was complaining about. There is another nuance here, which is that my models file is broken out into multiple modules, so the error is ACTUALLY:

 Can't pickle <class 'myproject.myapp.models.myfile.QuerySet'>: it's not the same object as myproject.myapp.models.myfile.QuerySet

Where myfile is one of the modules under models. I have an __ini__.py in models with the following line:

from myfile import *

I wonder if this is contributing to my issue. Is there some way to change my init to protect myself against this? Are there any other tests to try?

EDIT3: Here is a little more background on my use case: I have a model called Context that I use to populate a UI element with instances of mymodel. The user can add/remove/manipulate the objects on the UI side, changing their context, and when they return, they can keep their changes, because the context serialized everything. A context has a generic foreign key to different types of filters/ways the user can manipulate the object, all of which must implement a few methods that the context uses to figure out what it should display. One such filter takes a queryset that can be passed in and displays all of the objects in that queryset. This provides a way to pass in arbitrary querysets that are produced elsewhere and have them displayed in the UI element. The model that uses the Context is hierarchical (using mptt for this), and the UI element makes a request to get children each time the user clicks around, we can then take the children and determine if they should be displayed based on whether or not they are included in the Context. Hope that helps!

EDIT4: I am able to dump an empty queryset, but as soon as I add anything of value, it fails.

EDIT4: I am on Django 1.2.3

Answer 1

This may not be the case for everyone, but I was using Ipython notebook and having a similar issue pickling my own class. The problem turned out to be from a reload call

from dir.my_module import my_class    
reload(dir.my_module)

Removing the reload call and then re-running the import and the cell where the instance of that object was created then allowed it to be pickled.

Answer 2

not so elegant but perhaps it works: add the directory of the myfile -module to os.sys.path and use only import myfile in each module where you use myfile. (remove any from segment.segmentengine.models.segment import, anywhere in your project)