Simplest feature selection algorithm

Question

I am trying to create my own and simple feature selection algorithm. The data set that I am going to work with is here (very famous data set). Can someone give me a pointer on how to do so?

I am planning to write a feature rank algorithm for a text classification. This is for a sentiment analysis of movie reviews, classifying them as either positive or negative.

So my question is on how to write a simple feature selection for a text data set.

Answer 1

Feature selection methods are a big topic. You can start with following:

1. Chi square

2. Mutual information

3. Term frequency

etc. Read this paper if you have time: Comparative study on feature selection in text categorization this will help you lot.

The actual implementation depends on how you pre-process the data. Basically its keeping the counts, be it hash table or a database.

Answer 2

Random features work well, when you are then building ensembles. It's known as feature bagging.