Menu
I am having a problem in one of the teams that I am working in. One of the guys is a bit SQL happy in my opinion and wants to store the log information generated by a small python FTP downloader into a database, instead of just a nice formatted text file. Now its always been my opinion that a database should only be used if it speeds things up, or provides a more reliable interface to the data. What are your opinions?
Thanks!
Edit: In this particular instance, the data will grow about 100 lines per day and be processed once and thrown away. Although this case is of immediate concern, I am more interested in a general answer.
Edit 2: Thanks for all of your responses! I have marked the answer with the most up votes as the answer because I feel that it succinctly states most of the points you all have made, but I will watch and see if something else comes up.
351
They Require Fewer Hardware and Software Components. There's a good reason many operating systems, such as Windows, Macintosh, and Linux, use flat file databases. Though the databases can become bogged down with several thousand records, they are usually more efficient than other options.
Flat-file databases are simple and are essentially “free” but limit data access to manual processes and/or structured programs. Relational databases are generally more complex with varying costs but provide advanced capabilities and more efficient access options.
The primary advantage of a database vs a text file or an single data file (aka a flat file) is that a database allows you to capture relationships across single data files.
Databases support good data access because: Large volumes of data can be stored in one place. Multiple users can read and modify the data at the same time. Databases are searchable and sortable, so the data you need can be found quick and easily.
If you want to run reports on the data, or ask it questions later, a database is a logical choice, especially if you are storing multiple runs in the same database file to look for trends.
If you are only writing the logs from individual runs, and don't care about the data after you review it, then a database probably doesn't make sense.
157
Look, a lot of the "think of future needs" arguments are blantant over-engineering. KISS.
The only thing you need to do to address future needs in this respect is to simply write your logging routines in such a way that it is easy to totally redirect it later to something else. DIY text, syslog-type services, or a DB. Keep that concept in mind, but DON'T write anything but what you need right now.
From what you described, it absolutely sounds like you should just use a simple text file.
42
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
