heroku db:push throws "Taps Server Error: undefined method `symbolize_keys' for nil:NilClass"

Question

I'm in the process of switching to Heroku hosting. I have on my local machine a copy of the current production database (>100 M) and I cannot push it to heroku.

I'm getting the same error over and over again:

Taps Server Error: undefined method `symbolize_keys' for nil:NilClass

This seems to be related to some NULL values in the database (not all). Any ideas on how to fix this? Thanks!

Here is the full output:

!!! Caught Server Exception
HTTP CODE: 500
Taps Server Error: undefined method `symbolize_keys' for nil:NilClass
/app/lib/taps/data_stream.rb:183:in `parse_json'/app/lib/taps/server.rb:69:in 
POST /sessions/:key/push/verify_stream /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:863:in
call/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:863:in 
route /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:521:in
instance_eval /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:521:in
route_eval /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:500:in 
route!/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:497:in 
catch/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:497:in 
route!/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:476:in 
each/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:476:in 
route!/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:601:in 
dispatch! /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:411:in
call! /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in 
instance_eval /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in
invoke/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in 
catch /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:566:in
invoke /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:411:in
call!/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:399:in 
call/app/.bundle/gems/ruby/1.8/gems/rack-1.2.1/lib/rack/auth/basic.rb:25:in 
call /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:979:in 
call /app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:1005:in 
synchronize/app/.bundle/gems/ruby/1.8/gems/sinatra-1.0/lib/sinatra/base.rb:979:in
call/home/heroku_rack/lib/static_assets.rb:9:in 
call'/home/heroku_rack /lib/last_access.rb:15:in 
call'/app/.bundle/gems/ruby/1.8/gems/rack-1.2.1/lib/rack/urlmap.rb:47:in 
call'/app/.bundle/gems/ruby/1.8/gems/rack-1.2.1/lib/rack/urlmap.rb:41:in 
each'/app/.bundle/gems/ruby/1.8/gems/rack-1.2.1/lib/rack/urlmap.rb:41:in 
call'/home/heroku_rack/lib/date_header.rb:14:in 
call'/app/.bundle/gems/ruby/1.8/gems/rack-1.2.1/lib/rack/builder.rb:77:in 
call'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/connection.rb:76:in         
pre_process'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/connection.rb:74:in 
catch'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/connection.rb:74:in       
pre_process'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/connection.rb:57:in
process'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/connection.rb:42:in     
receive_data'/app/.bundle/gems/ruby/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in 
run_machine'/app/.bundle/gems/ruby/1.8/gems/eventmachine-0.12.10/lib/eventmachine.rb:256:in 
run'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/backends/base.rb:57:in 
start'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/server.rb:156:in 
start'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/controllers/controller.rb:80:in 
start'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/runner.rb:177:in 
send'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/runner.rb:177:in 
run_command'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/lib/thin/runner.rb:143:in 
run!'/app/.bundle/gems/ruby/1.8/gems/thin-1.2.7/bin/thin:6/usr/ruby1.8./bin/thin:19:in  
load'/usr/ruby1.8.7/bin/thin:19

Answer 1

Heroku blog has a good discussion on strategies for pushing big data to heroku using taps

1. To break up the job, push only specific tables
   heroku db:pull --tables logs,tags

2. Resume transfer using the crash logfile
   heroku db:pull --resume-filename session_file.dat

For more info see: http://blog.heroku.com/archives/2010/4/21/supporting_big_data_part_1/