Rails - recovering database from Production.log

Question

We recently lost a database and I want to recover the data from de Production.log.

Every request is logged like this:

Processing ChamadosController#create (for XXX.XXX.XXX.40 at 2008-07-30 11:07:30) [POST] Session ID: 74c865cefa0fdd96b4e4422497b828f9 Parameters: {"commit"=>"Gravar", "action"=>"create", "funcionario"=>"6" ... (all other parameters go here).

But some stuff to post on de database were in the session. In the request I have the Session ID, and I also have all the session files from the server.

Is there anyway I can, from this Session ID, open de session file and get it's contents?

Answer 1

It's probably best to load the session file into a hash -- using the session-id as the key -- and then go through all the log files in chronological order, and parse out the relevant info for each session, and modify your database with it.

• I guess you're starting out with an old database backup? Make sure to do this in a separate Rails environment -- e.g. don't do this in production; create and use a separate "recovery" environment / DB.

• think about some sanity checks you can run on the database afterwards, to make sure that the state of the records makes sense

Going forward:

• make sure that you do regular backups going forward (e.g. with mysqldump if you use MySQL).

• make sure to set up your database for master/slave replication

hope this helps -- good luck!

Answer 2

Have you tried using Marshal#load? I'm not sure how you're generating those session files, but it's quite possible Rails just uses Marshal.

Answer 3

A client exactly had the same problem a few weeks ago. I came up with the following solution:

1. play back the latest backup you have (in our case it was one year old)
2. write a small parser that moves all the requests from production in a temporary database (i chose mongodb for that): i used a rake task and "eval" to create the hash.
3. play back the data in the following order
   1. play in the first create of an object, if it does not already exist.
   2. find the last update (by date) and play it back.

here is the regex for scanning the production.log:

file = File.open("location_of_your_production.log", "rb")
contents = file.read
contents.scan(/(Started POST \"(.*?)\" for (.*?) at (.*?)\n.*?Parameters: \{(.*?)\}\n.*?Completed (.*?) in (.*?)ms)/m).each do |x|
  # now you can collect all the important data.
  # do the same for GET requests as well, if you need it.
end

In my case, the temporary database speeded up the process of the logfile parsing, so the above noted steps could be taken. Of course, everything that was not sent over production.log will be lost. Also, updates of the objects would send the whole information, it might be different in your case. I could also recreate the image uploads, since the images were sent base64 encoded in the production.log.

good luck!