Using JRuby/Jython for Ruby/Python interoperability?

Question

Quite-probably a silly question, as I don't know much about Java/Jython/JRuby/bytecode, but..

I stumbled across _why's unholy again today.. It allows you to output Python bytecode from Ruby code.. Basically allowing them to produce the same bytecode..

Jython outputs Java bytecode, as does JRuby.. Since these both compile to the same bytecode, does this mean you could potentially use any Python library from Ruby, and Ruby libraries from Python?

Answer 1

No, that won't work. At least not the way you think it would.

Interoperability between Jython and JRuby works the same way as between CPython and YARV: they both run on the same platform, so they can communicate with each other using that platform.

In the case of CPython and YARV, that platform is C/POSIX, so they can communicate with each other using C structs, ints, char*s and C function calls. In the case of Jython and JRuby, that platform is the JVM, so they can communicate with each other using JVM objects, JVM classes, JVM interfaces, JVM types and JVM methods.

In both cases, those platform primitives look nothing like Python or Ruby objects.

To JRuby, Jython is just yet another Java program. To Jython, JRuby is just another Java program.

For example: in Ruby, you can add, remove and redefine methods dynamically at any moment. On the JVM, the smallest unit of code that can be dynamically added and removed is a class. So, a Ruby method is actually not represented as a Java method. It is represented as a Java class. And logically, a Ruby object with a couple of methods is represented as a Java object with no methods, just a Dictionary<String, RubyMethod> field. IOW: it's totally unusable from Java, and, since from JRuby's point of view Jython is just Java, it's also unusable from Jython.

Now, there are ways to make this a little bit better. You could use actual Java types to communicate between the two – both implementations have great interoperability with Java. So, instead of passing a Ruby hash to Python or a Python dictionary to Ruby, you would use a Java Map from both Ruby and Python. But note that this requires that both your Ruby and Python code are specifically written to work on the JVM. IOW: you cannot just use any Python or Ruby library you find on the web, which is what you are asking about.

Another possibility is the one mentioned by @duncan in his answer: embed Jython or JRuby as a scripting engine into your Ruby or Python application. But again, this doesn't really answer your question about using arbitrary Python libraries from Ruby or vice versa.

So, what is the problem here?

The problem is that in order for the two runtimes to communicate, they need to speak the same "language". And in this particular case, the only language that the two runtimes have in common, is Java, or rather a severely crippled subset of Java.

So, we need to find a common language. One way to define such a language would be for both runtimes to understand each other's Meta-Object Protocol (MOP).

A MOP is basically an object model for the language's object model. Um, that's confusing because we use the word "object model" to mean two different things. Let me rephrase that:

A MOP is basically a domain model for the language's object system. Just like a domain model for a banking system contains objects that represent real-world customers, accounts, balances, ledgers and so on, and methods that represent real-world actions like money transfers, withdrawals and so on, a MOP contains objects that represent language classes, methods, variables, objects and methods that represent language actions like looking up a variable, calling a method, inheriting from a class, constructing an instance of a class.

Normally, every runtime keeps its MOP private, and every runtime has its own MOP.

If JRuby and Jython exposed their MOPs to each other and understood each other's MOPs (or, even better yet: they exposed their MOPs to the JVM and both used the same MOP), then you could pass one of those crazy JRuby method bags to Jython, and it would know how to find the methods that belong to that object and how to call them, because it can just ask JRuby's MOP how to do it.

There is actually a project to create just such a MOP for the JVM: the dynalang MOP is a project for a shared, standardized MOP for dynamic languages running on the JVM. It was created by Attila Szegedi, the maintainer of the Mozilla Rhino ECMAScript engine. At the moment, none of the big language implementations uses it, but there is collaboration going on between at least Rhino, JRuby, Jython and Groovy to make sure that dynalang is generic enough that it can support all of the different language's object models.

If you want a sneak peek at what a world with such a shared MOP would look like, you can take a look at Microsoft's Dynamic Language Runtime (DLR). The DLR contains just such a MOP and all runtimes which support the DLR (which, in addition to the usual suspects such as IronRuby, IronPython, IronJS and IronScheme now also includes C# 4 and Visual Basic.NET 10) can almost seamlessly interoperate with each other.

Another similar platform is the Parrot Virtual Machine, which was specifically designed to allow multiple dynamic languages to interoperate on the same runtime platform. There are implementations of Python (Pynie) and Ruby (Cardinal) available, but especially Cardinal is still very far from being even a remotely complete Ruby implementation.

Answer 2

There are two ways to do it. Both offer the ability to statically compile code and produce a real Java class from the script. Jython AFAIK in this case generates Java source code and then calls javac, via an jythonc script. But this requires compilation.

For both interpreters, you can call Java code from scripts, and you can embed the interpreter in a Java application.

For example, to call Java from Python:

>>> from java.util import Random
>>> r = Random()
>>> r.nextInt()
501203849

To embed JRuby interpreter in Java, you can do (note, there is a JSR223 based way too, this is the core one):

package vanilla;

import org.jruby.embed.ScriptingContainer;

public class HelloWorld {

    private HelloWorld() {
        ScriptingContainer container = new ScriptingContainer();
        container.runScriptlet("puts Hello world");
    }

    public static void main(String[] args) {
        new HelloWorld();
    }

You could do the same from Jyton (I guess you would need to give the jruby paths correctly):

import org.jruby.embed.ScriptingContainer
container = ScriptingContainer()
container.runScriptlet("puts Hello world")

The same can be done the other way around.

You won't get the whole ruby stdlib exported to the python interpreter by doing an import. You would need to precompile ruby's stdlib to bytecode in advance.

However with the technique described above, and adding a couple of helper scripts and defined interfaces, you can bridge specific functionality from one language to the other.