Why does dumping and loading a Hash using Marshal in Ruby throw a FormatError?

Question

I'm running Ruby installed from RubyInstaller. Here's the version:

C:\Users\Sathya>ruby -v
ruby 1.9.2p290 (2011-07-09) [i386-mingw32]

Here's the exact code, which is throwing the error:

hashtime = Hash.new(Time.mktime('1970'))
hashtime[1]  = Time.now
=> 2011-10-04 19:26:53 +0530
print hashtime
{1=>2011-10-04 19:26:53 +0530}=> nil
hashtime[1]  = Time.now
=> 2011-10-04 19:27:20 +0530
print hashtime
{1=>2011-10-04 19:27:20 +0530}=> nil
File.open('timehash','w') do |f|
  f.write Marshal.dump(hashtime)
end
=> 56  

Now, trying to load it.

Marshal.load (File.read('timehash'))

Gives the error:

ArgumentError: dump format error for symbol(0x42)
        from (irb):10:in `load'
        from (irb):10
        from C:/Ruby192/bin/irb:12:in `<main>'

Why is this throwing an error? Am I doing something wrong, or is this a bug?

I'm running on Windows 7 Ultimate, 64-bit

---

Here's the results from the edited debug code you had mentioned:

hashtime = Hash.new
=> {}
hashtime[1] = Time.now
=> 2011-10-04 20:49:52 +0530
hashdump = Marshal.dump(hashtime)
=> "\x04\b{\x06i\x06Iu:\tTime\r\x8F\xE4\e\x80<\xADGO\x06:\voffseti\x02XM"
hashtime = Marshal.load (hashdump)
=> {1=>2011-10-04 20:49:52 +0530}
print hashtime
{1=>2011-10-04 20:49:52 +0530}=> nil  

---

Results to Edit 2:

hashtime = Hash.new
=> {}
hashtime[1] = Time.now
=> 2011-10-04 21:04:24 +0530
hashdump = Marshal.dump(hashtime)
=> "\x04\b{\x06i\x06Iu:\tTime\r\x8F\xE4\e\x80\x92o\x8C\x89\x06:\voffseti\x02XM"
print "hashdump: #{hashdump}"
ÅS?ÇÆoîë?:?offseti?XM=> nile
File.open('timehash','w') do |f|
 f.write hashdump
end
=> 36
hashdump2 = File.read('timehash')
=> "\x04\b{\x06i\x06Iu:\tTime\n\x8F\xE4\e\x80\x92o\x8C\x89\x06:\voffseti\x02XM"
print "hashdump2: #{hashdump2}"
hashdump2:{?i?Iu:       Time
ÅS?ÇÆoîë?:?offseti?XM=> nil
hashtime2 = Marshal.load (hashdump2)
ArgumentError: dump format error for symbol(0x8c)
        from (irb):73:in `load'
        from (irb):73
        from C:/Ruby192/bin/irb:12:in `<main>'  

Some of the characters didn't come out, here's a screenshot:

---

Now I'm getting a time format different error

hashtime = Hash.new
=> {}
hashtime[1] = Time.now
=> 2011-10-04 21:23:15 +0530
hashdump = Marshal.dump(hashtime)
=> "\x04\b{\x06i\x06Iu:\tTime\r\x8F\xE4\e\x80\xB9\xE1\xFB\xD4\x06:\voffseti\x02X
M"
print "hashdump: #{hashdump}"
ÅΣ←Ç╣ß√╘♠:♂offseti☻XM=> nile
File.open('timehash','wb') do |f|
 f.write hashdump
end
=> 36
hashdump2 = File.read('timehash')
=> "\x04\b{\x06i\x06Iu:\tTime\n\x8F\xE4\e\x80\xB9\xE1\xFB\xD4\x06:\voffseti\x02X
M"
print "hashdump2: #{hashdump2}"
hashdump2:{♠i♠Iu:       Time
ÅΣ←Ç╣ß√╘♠:♂offseti☻XM=> nil
hashtime2 = Marshal.load (hashdump2)
TypeError: marshaled time format differ
        from (irb):10:in `_load'
        from (irb):10:in `load'
        from (irb):10
        from C:/Ruby192/bin/irb:12:in `<main>'

Answer 1

You need to write to the file in binary mode by appending a b to the file mode:

File.open('timehash','wb') do |f|
  f.write Marshal.dump(hashtime)
end

You can see this is the issue by comparing the strings (from our debugging) before writing to disk vs after reading back in:

=> "\x04\b{\x06i\x06Iu:\tTime\r\x8F\xE4\e\x80\x92o\x8C\x89\x06:\voffseti\x02XM"
=> "\x04\b{\x06i\x06Iu:\tTime\n\x8F\xE4\e\x80\x92o\x8C\x89\x06:\voffseti\x02XM"
                             ^^

a \r (carriage return) is being changed to a \n (newline)

However, it seems like even with the binary modifier your system is not obeying you and is changing \r to \n... So let's try encoding the data to base64:

File.open('timehash','w') do |f|
  hashtime_marshal = Marshal.dump(hashtime)
  f.write [hashtime_marshal].pack("m")
end

hashtime_encoded = File.read('timehash')
hashtime = Marshal.load( hashtime_encoded.unpack("m")[0] )

Let me know if that works?

---

Old Information:

Don't pass anything to Hash.new:

>> hashtime = Hash.new
=> {}
>> hashtime[1] = Time.now
=> Tue Oct 04 10:57:49 -0400 2011
>> hashtime
=> {1=>Tue Oct 04 10:57:49 -0400 2011}
>> File.open('timehash','w') do |f|
?>   f.write Marshal.dump(hashtime)
>> end
=> 22
>> Marshal.load (File.read('timehash'))
(irb):10: warning: don't put space before argument parentheses
=> {1=>Tue Oct 04 10:57:49 -0400 2011}

The documentation states that the obj parameter to Hash.new is the default value... it should work as you have it... I don't know why it doesn't... but in your case nil is an acceptable default, just check to see if values are nil and if so, use a Time.mktime('1970') for them instead.

EDIT: That solved the problem for me, however, I'm on OS X not Windows. So, let's try a bit of debugging. What happens when you run the following code?

hashtime = Hash.new
hashtime[1] = Time.now
hashdump = Marshal.dump(hashtime)
hashtime = Marshal.load (hashdump)
print hashtime

EDIT #2: OK. So Marshal.dump and Marshal.load seem to work. Looks like it's something with the file I/O... Please post the results of the following code...

hashtime = Hash.new
hashtime[1] = Time.now
hashdump = Marshal.dump(hashtime)
print "hashdump: #{hashdump}"
File.open('timehash','w') do |f|
  f.write hashdump
end
hashdump2 = File.read('timehash')
print "hashdump2: #{hashdump2}"
hashtime2 = Marshal.load (hashdump2)
print hashtime2

Answer 2

Instead of reading with File.read try File.binread or File.open('timehash', 'rb')