Existing data serialized as hash produces error when upgrading to Rails 5

Question

I am currently upgrading a Ruby on Rails app from 4.2 to 5.0 and am running into a roadblock concerning fields that store data as a serialized hash. For instance, I have

class Club
  serialize :social_media, Hash
end

When creating new clubs and inputting the social media everything works fine, but for the existing social media data I'm getting:

ActiveRecord::SerializationTypeMismatch: Attribute was supposed to be a Hash, but was a ActionController::Parameters.

How can I convert all of the existing data from ActionController::Parameter objects to simple hashes? Database is mysql.

Answer 1

From the fine manual:

serialize(attr_name, class_name_or_coder = Object)

[...] If class_name is specified, the serialized object must be of that class on assignment and retrieval. Otherwise SerializationTypeMismatch will be raised.

So when you say this:

serialize :social_media, Hash

ActiveRecord will require the unserialized social_media to be a Hash. However, as noted by vnbrs, ActionController::Parameters no longer subclasses Hash like it used to and you have a table full of serialized ActionController::Parameters instances. If you look at the raw YAML data in your social_media column, you'll see a bunch of strings like:

--- !ruby/object:ActionController::Parameters...

rather than Hashes like this:

---\n:key: value...

You should fix up all your existing data to have YAMLized Hashes in social_media rather than ActionController::Parameters and whatever else is in there. This process will be somewhat unpleasant:

1. Pull each social_media out of the table as a string.
2. Unpack that YAML string into a Ruby object: obj = YAML.load(str).
3. Convert that object to a Hash: h = obj.to_unsafe_h.
4. Write that Hash back to a YAML string: str = h.to_yaml.
5. Put that string back into the database to replace the old one from (1).

Note the to_unsafe_h call in (3). Just calling to_h (or to_hash for that matter) on an ActionController::Parameters instance will give you an exception in Rails5, you have to include a permit call to filter the parameters first:

h = params.to_h                   # Exception!
h = params.permit(:whatever).to_h # Indifferent access hash with one entry

If you use to_unsafe_h (or to_unsafe_hash) then you get the whole thing in a HashWithIndifferentAccess. Of course, if you really want a plain old Hash then you'd say:

h = obj.to_unsafe_h.to_h

to unwrap the indifferent access wrapper as well. This also assumes that you only have ActionController::Parameters in social_media so you might need to include an obj.respond_to?(:to_unsafe_hash) check to see how you unpack your social_media values.

You could do the above data migration through direct database access in a Rails migration. This could be really cumbersome depending on how nice the low level MySQL interface is. Alternatively, you could create a simplified model class in your migration, something sort of like this:

class YourMigration < ...
  class ModelHack < ApplicationRecord
    self.table_name = 'clubs'
    serialize :social_media
  end

  def up
    ModelHack.all.each do |m|
      # Update this to match your real data and what you want `h` to be.
      h = m.social_media.to_unsafe_h.to_h
      m.social_media = h
      m.save!
    end
  end

  def down
    raise ActiveRecord::IrreversibleMigration
  end
end

You'd want to use find_in_batches or in_batches_of instead all if you have a lot of Clubs of course.

---

If your MySQL supports json columns and ActiveRecord works with MySQL's json columns (sorry, PostgreSQL guy here), then this might be a good time to change the column to json and run far away from serialize.

Answer 2

Extending on short's reply - a solution that does not require a database migration:

class Serializer
  def self.load(value)
    obj = YAML.load(value || "{}")
    if obj.respond_to?(:to_unsafe_h)
      obj.to_unsafe_h
    else
      obj
    end
  end
  def self.dump(value)
    value = if value.respond_to?(:to_unsafe_h)
      value.to_unsafe_h
    else
      value
    end
    YAML.dump(value)
  end
end

serialize :social_media, Serializer

Now club.social_media will work whether it was created on Rails 4 or on Rails 5.

Answer 3

The reply by @schor was a life-saver, but I kept getting no implicit conversion of nil into String errors when doing the YAML.load(value).

What worked for me was:

class Foo < ApplicationRecord
  class NewSerializer
    def self.load(value)
      return {} if !value #### THIS NEW LINE
      obj = YAML.load(value)
      if obj.respond_to?(:to_unsafe_h)
        obj.to_unsafe_h
      else
        obj
      end
    end
    def self.dump(value)
      if value.respond_to?(:to_unsafe_h)
        YAML.dump(value.to_unsafe_h)
      else
        YAML.dump(value)
      end
    end
  end

  serialize :some_hash_field, NewSerializer
end

I gotta admin the Rails team totally blindsided me on this one, a most unwelcome breaking change that doesn't even let an app fetch the "old" data.