Does Scikit-learn support transfer learning?

Question

Does Scikit-learn support transfer learning? Please check the following code.

model clf is gotten by fit(X,y)

Can model clf2 learn on the base of clf and transfer learn by fit(X2,y2) ?

---

>>> from sklearn import svm
>>> from sklearn import datasets
>>> clf = svm.SVC()
>>> X, y= ....
>>> clf.fit(X, y)
SVC()

>>> import pickle
>>> s = pickle.dumps(clf)
>>> clf2 = pickle.loads(s)
>>> clf2.fit(X2,y2)

>>> clf2.predict(X[0:1])

Answer 1

In the context of scikit-learn there's no transfer learning as such, there is incremental learning or continuous learning or online learning.

By looking at your code, whatever you're intending to do won't work the way you're thinking here. From this scikit-learn documentation:

Calling fit() more than once will overwrite what was learned by any previous fit()

Which means using fit() more than once on the same model will simply overwrite all the previously fitted coefficients, weights, intercept (bias), etc.

However if you want to fit a portion of your data set and then improve your model by fitting a new data, what you can do is look for estimators that include partial_fit API implementation.

If we call partial_fit() multiple times, framework will update the existing weights instead of re-initialising them.

Another way to do incremental learning with scikit-learn is to look for algorithms that support the warm_start parameter.

From this doc:

warm_start: bool, default=False

When set to True, reuse the solution of the previous call to fit() as initialization, otherwise, just erase the previous solution. Useless for liblinear solver.

Another example is Random forrest regressor.