Imbalanced data: undersampling or oversampling? [closed]

Question

I have binary classification problem where one class represented 99.1% of all observations (210 000). As a strategy to deal with the imbalanced data, I choose sampling techniques. But I don't know what to do: undersampling my majority class or oversampling the less represented class. If anybody have an advise?

Thank you.

P.s. I use random forest algorithm from sklearn.

Answer 1

Undersampling:
Undersampling is typically performed when we have billions (lots) of data points and we don’t have sufficient compute or memory(RAM) resources to process the data. Undersampling may lead to worse performance as compared to training the data on full data or on oversampled data in some cases. In other cases, we may not have a significant loss in performance due to undersampling.

Undersampling is mainly performed to make the training of models more manageable and feasible when working within a limited compute, memory and/or storage constraints.

Oversampling:

oversampling tends to work well as there is no loss of information in oversampling unlike undersampling.

Answer 2

I think that there is a typo in the accepted answer above. You should not "undersample the minority" and "oversample the majority"; rather, you should undersample the majority and oversample the minority.

If you're familiar with Weka, you can experiment using different data imbalance techniques and different classifiers easily to investigate which method works best. For undersampling in Weka, see this post: combination of smote and undersampling on weka.

For oversampling in Weka, you can try the SMOTE algorithm (some information is available here: http://weka.sourceforge.net/doc.packages/SMOTE/weka/filters/supervised/instance/SMOTE.html). Of course, creating 20,811 synthetic minority data (i.e., if you're looking for balanced data) is more computationally expensive than undersampling because: (1) there is a computational cost associated with creating the synthetic data; and (2) there is a greater computational cost associated with training on 42,000 samples (including the 20,811 synthetic samples created for the minority class) as opposed to 21,000 samples.

In my experience, both data imbalance approaches you've mentioned work well, but I typically experiment first with undersampling because I feel that it is a little cheaper from a resource perspective.

There are Python packages for undersampling and oversampling here:

1. Undersampling: http://glemaitre.github.io/imbalanced-learn/auto_examples/ensemble/plot_easy_ensemble.html
2. Oversampling: http://contrib.scikit-learn.org/imbalanced-learn/stable/auto_examples/over-sampling/plot_smote.html

You can also investigate cost-sensitive classification techniques to penalize misclassifications of minority class via a cost matrix.

1. Here is a link to a nice Weka package: https://weka.wikispaces.com/CostSensitiveClassifier
2. Here is a link to a Python package: https://wwwen.uni.lu/snt/research/sigcom/computer_vision_lab/costcla_a_cost_sensitive_classification_library_in_python