How to transfer basic objects between Ruby and Python?

Question

Currently I'm using JSON as a serialization format to transfer a simple hash containing strings, numbers and an array from Ruby into a Python script:

IO.popen('./convert.py', 'w') do |w|
    w.write({ :height => height, :width => width, :id => job_id, :data => pix }.to_json)
    w.write "\n"
    w.close_write
end

In this case height, width and job_id are all numbers and pix is an array of arrays of integers.

The python script this is running is:

#!/usr/bin/env python

from PIL import Image
import json
import sys

output = json.load(sys.stdin)

width = output['width']
height = output['height']
name = 'images/' + str(output['id']) + '/image.bmp'
data = [ tuple(datum) for datum in output['data'] ]

img = Image.new("RGB", (width, height))

img.putdata(data)

img.save(name)

A bit of quick testing utilising an array with 3.9 million values (probably around 1/4 of the size that will normally be used) revealed the script to take around 105 seconds, and 90 seconds with all lines below output = ... commented out. Obviously it would be good if the serialisation didn't take 85% of the processing time for such a simple script.

The only method to speed this up that I can think of is to find some form of binary serialisation/marshalling that could be utilised to transfer the data from Ruby to Python. Unfortunately I have been unable to find any such system, only RMarshal which appears to be able to work the other way.

Answer 1

Maybe MessagePack is the way to go then. Bindings for several languages exist, including Ruby and Python.