Preprocessing machine learning data [closed]

Question

This may be a stupid question, but I am new to ML and can't seem to find a clear answer.

I have implemented a ML algorithm on a Python web app.

Right now I am storing the data that the algorithm uses in an offline CSV file, and every time the algorithm is run, it analyzes all of the data (one new piece of data gets added each time the algorithm is used).

Apologies if I am being too vague, but I am wondering how one should generally go about implementing the data and algorithm properly so that:

1. The data isn't stored in a CSV (Do I simply store it in a database like I would with any other type of data?)

2. Some form of preprocessing is used so that the ML algorithm doesn't have to analyze the same data repeatedly each time it is used (or does it have to given that one new piece of data is added every time the algorithm is used?).

Answer 1

The data isn't stored in a CSV (Do I simply store it in a database like I would with any other type of data?)

You can store in whatever format you like.

Some form of preprocessing is used so that the ML algorithm doesn't have to analyze the same data repeatedly each time it is used (or does it have to given that one new piece of data is added every time the algorithm is used?).

This depends very much on what algorithm you use. Some algorithms can easily be implemented to learn in an incremental manner. For example, Linear/Logistic Regression implemented with Stochastic Gradient Descent could easily just run a quick update on every new instance as it gets added. For other algorithms, full re-trains are the only option (though you could of course elect not to always do them over and over again for every new instance; you could, for example, simply re-train once per day at a set point in time).