Does keras.backend.clear_session() deletes sessions in a process or globally?

Question

I create up to 100 keras models in separated script an save them localy with model.save(). For Training them, I use multiprocessing.pool. In those processes I load each model separately. Because of occuring Memory Errors I used keras.backend.clear_session(). This seems to work but I have also read that it deletes the weights of models.

So to come back to my question, if I import "from keras import backend as K" in each process of the pool and at the end, after I saved the models, I use K.clear_session(), do I clear important data of parallel running processes or just data of this process?

If it deletes important data of parallel running processes. Is there any possibility of creating a local tensorflow session inside the process. Then assign the needed model to this session and then clear_session() this local one?

I´m thankful for any input.

In adition it would be helpful if anyone knows the exact functionality of clear_session(). The explanation of this function is not very informative especially for beginners like me.

Thank you :)

Answer 1

I faced similar kind of issue but I am not running models in parallel but alternatively i;e; either of the models (in different folders but same model file names) will run.

When I run the models directly without clear_session it was conflicting with the previously loaded model and cannot switch to other model. After including clear_session at the beginning of statements (which loads the model) it was working, however it was also deleting global variables declared at the beginning of the program which are necessary for prediction activity.

lesson learnt: clear_session will not only "Destroys the current TF graph and creates a new one." as mentioned in the documentation but also deletes global variables defined in the program.

So I defined the global variables just after the clear_session statement

** feedback appreciated