Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Store static data in an array, or in a database?

Tags:

database

We always have some static data which can be stored in a file as an array or stored in a database table in our web based project. So which one should be preferred?

In my opinion, arrays have some advantages:

  1. More flexible (it can be any structure, which specifies a really complex relation)
  2. Better performance (it will be loaded in memory, which will have better read/write performance compared with a database's I/O operations)

But my colleague argued that he preferred DB approach, since it can keep a uniform data persistence interface, and be more flexible.

So which should be preferred? Or how can we choose? Or we should prefer one in some scenario and another in other scenarios? what are the scenarios?

EDIT:

Let me clarify something. Truly just as Benjamin made the change to the title, the data we want to store in an array(file) won't change so frequently, which means the code won't change the value of the array in the runtime. If the data change very frequently I will use DB undoubtedly. That's why I made such a post.

And sometimes it's hard to store some really complex relations like:

Task = {
  "1" : {
    "name" : "xx",
    "requirement" : {
          "level" : 5,
          "money" : 100,
     }
   ...
 }

Just like the above code sample(a python dict or you can think it as an array), the requirement field is hard to store in DB(store a structure like pickled object directly in DB? not so good I think). So in such condition, I will prefer arrays.

So what's your idea? In such scenario, we should prefer arrays to DB, right?

Regards.

like image 894
Zhu Tao Avatar asked Nov 05 '09 12:11

Zhu Tao


People also ask

What is static data in database?

Static data, also known as reference or lookup data, is data that is necessary for the first deployment so that the application which is built upon that database can work properly. This data can be any set of predefined values that are rarely changed, e.g. postal codes, lists of states e.g. NY, etc.

What is static data store?

Static data refers to data that doesn't change frequently. This is often the data that defines the configuration and structure of the system.

Is array static data type?

A static datatype is one which is having fixed size in memory. As we declare the array size in advance, that much bytes or space is reserved in memory and cannot be increased later. So this way an array is a Static datatype.

Can databases store arrays?

Array Databases are a class of No-SQL databases that store, manage, and analyze data whose natural structures are arrays. With the growth of large volumes of spatial data (i.e., satellite imagery) there is a pressing need to have new ways to store and manipulate array data.


2 Answers

Lets be pragmatic/objetive:

  • Do you write to your data on runtime? Yes: Db, No: File
  • Do you update your data more than once per week? Yes: Db, No: File
  • It's a pain to release an updated data file? Yes: Db, No: File,
  • Do you read that data often? Yes: File/Cache, No: Db
  • It is a pain to update that data file and you need extra tools? Yes: db, No: File

For sure I've forgotten other points, but I guess the basics are there.

like image 190
graffic Avatar answered Sep 19 '22 00:09

graffic


The "flexiable" array in a file is fraught with a zillion issues already delt with by using a DB. Unless you can prove that the DB is really going to way slower than using the other approach use a DB. Move on and start solving business problems.

Edit

Comment from OP asks what the issues with using a file might be, here are a handful (pause to take a deep breath).

  • Concurrency: You have to manage the situation where multiple requests may be trying to write back to the file. Not too hard but it becomes a bottleneck.
  • Performance: Yes modifying an in-memory array is quicker but how do you determine how much and when the array needs to be persisted to a file. Note that using a DB doesn't pre-clude the use of an appropriate in-memory cache. Writing a file back each time a small modification is made isn't going to perform that well.
  • Scalability: Really a function of the first two. In order to acheive any scalable goals you need to be able to quickly modify small bits of the data that is persisted. IWO if you don't use a DB you would end up writing one. If you find you need more than one webserver to support growing demand where are you going to store the file(s)? Now you've got file I/O over a network (ableit likely a very quick one).
  • Structure: Your code will be responsible for managing the structure of data, querying it etc if you use an array. How will you do that in way which acheives greater "flexibility" than using a DB? All manner of choices and complexity are needed here.
  • Reliability: You need to ensure the integrity of your persisted data. In the event of some failure your array/file code would need to ensure that data is at least not so corrupt that the application can continue.
like image 33
AnthonyWJones Avatar answered Sep 23 '22 00:09

AnthonyWJones