how does pickle know which to pick?

Question

I have my pickle function working properly

    with open(self._prepared_data_location_scalar, 'wb') as output:
        # company1 = Company('banana', 40)
        pickle.dump(X_scaler, output, pickle.HIGHEST_PROTOCOL)
        pickle.dump(Y_scaler, output, pickle.HIGHEST_PROTOCOL)


    with open(self._prepared_data_location_scalar, 'rb') as input_f:
        X_scaler = pickle.load(input_f)
        Y_scaler = pickle.load(input_f)

However, I am very curious how does pickle know which to load? Does it mean that everything has to be in the same sequence?

Answer 1

What you have is fine. It's a documented feature of pickle:

It is possible to make multiple calls to the dump() method of the same Pickler instance. These must then be matched to the same number of calls to the load() method of the corresponding Unpickler instance.

There is no magic here, pickle is a really simple stack-based language that serializes python objects into bytestrings. The pickle format knows about object boundaries: by design, pickle.dumps('x') + pickle.dumps('y') is not the same bytestring as pickle.dumps('xy').

If you're interested to learn some background on the implementation, this article is an easy read to shed some light on the python pickler.

Answer 2

wow I did not even know you could do this ... and I have been using python for a very long time... so thats totally awesome in my book, however you really should not do this it will be very hard to work with later(especially if it isnt you working on it)

I would recommend just doing

pickle.dump({"X":X_scalar,"Y":Y_scalar},output)

 ...
data = pickle.load(fp)
print "Y_scalar:",data['Y']
print "X_scalar:",data['X']

unless you have a very compelling reason to save and load the data like you were in your question ...

edit to answer the actual question...

it loads from the start of the file to the end (ie it loads them in the same order they were dumped)