Architecture for managing JSON files in a javascript project: Internal database vs.git

Question

In my company, developers manage some resources manually (by manipulating a JSON file):

• Configuration: language strings.
• Templates: page templates.
• Menu: menu structure.

I am thinking about the advantages and disadvantages of moving towards an internal database that will generate the JSON files during the build process.

As i see it, there is (as always :-)) a tradeoff:

Benefits & disadvantages

The benefits of the manual approach (JSON manipulation):

• Low resource footprint: no servers, databases or logins needed.
• Uses existing infrastructure: git is used for versioning.
• Easy rollback: artifacts can be easily rolled back.

The disadvantages of the manual approach:

• Easily breakable: no database means data integrity is questionable. locks? validations?
• No APIs: it is harder to offer internal APIs based on file system data.
• Huge files: files are becoming bigger and bigger as time goes by, to the point they may even freeze the IDE.

Does anybody have experience with managing JSON resources? Is manipulating a JSON file and using GIT-based versions a logical approach?

Thanks,

Answer 1

I love managing such data within GIT. There are a lot of people that favor the database approach, but I think that the simplicity of the file based approach is so much more worth than data integrity.

You can assure that with tasks during the deployment, build or test process. This would also eliminate the problem of large files. Just split them up into logical parts and concat it for your deployment.

We even build a process on top of simple CSV files that used pull-requests for project managers to change configuration parts in the application. We used automated tests to make sure that new changes did not break anything.

The database driven approach has the big downside of syncing. You need to make sure that latest data get's synced to development, staging and live systems. Changes are hard to track and you need to make sure that they don't break your whole platform when something goes wrong.

Don't underestimate the effort it takes to sync stuff. Syncing is super hard to get right.

Answer 2

This is more of a requirement analysis. Depending on if your company cares putting data centralized, Or distributed, you may have to decide. The things you need to take care:

Do you think this approach (GIT) will work for 50+ developers?

Sure. In fact, there will be more visibility, everyone will have ability to do their own experiment. You better follow RAC (Review and commit), so that no errors will be committed as you would apply and see if everything is working fine before committing to master.

Huge files: files are becoming bigger and bigger as time goes by, to the point they may even freeze the IDE.

I don't see any reason (understand) why would they pile up. The versions (branches) each developer work would have their own editions. And you only commit a diff when you do to master.

Easily breakable: no database means data integrity is questionable. locks? validations?

I don't think so. git approach is not easily breakable, although it does not provide any access security mechanism. It looks vague and ad-hoc, but you probably (most probably) can trust it. You know, you will only commit after a git apply and feel comfortable. Since every developer checks out a master, you can think of those many database copies existing for the original version. If one machine crashes, you will find them elsewhere. And that is the flexibility git is providing at no cost at all. (developers keep those files as part of their regular development).