Elixir/Erlang: Communication with external process

Question

Say I have a simple python script which executes an elixir/erlang script using the subprocess module.

Say the OS PID of the python script is P1 and that of the spawned elixir/erlang script running is P2.

I want to know if communication between P1 and P2 is possible. More specifically, P1 writes something to the stdin of P2, and P2 reads the received input from P1 and writes some corresponding output to its own stdout and P1 reads from the stdout of P2 and again writes something to the stdin of P2 and so on.

I know the other way is possible, i.e., spawning external process from inside elixir/erlang and then communicating with the process. Any help appreciated, thanks.

Answer 1

Yep, this sort of cross-language IPC is entirely possible. The vast majority of the documentation and blog posts and such (and the responses so far here on StackOverflow!) assume the opposite of what you seem to be asking - that is, they assume that Erlang/Elixir is spawning the Python subprocess, rather than Python spawning an Erlang/Elixir subprocess. If that's okay (i.e. you're okay with your Erlang or Elixir app spinning up the Python process), then great! Badu's answer will help you do exactly that, and you could also have a gander at the documentation for Elixir's Port module for an extra reference.

But that doesn't seem to be the answer you seek, and that's less fun. The world needs more documentation on how to go the other way around, so let's dive into the wonderful world of running Erlang as a subprocess of a Python script!

First, our Python script (eip.py):

#!/usr/bin/env python
from subprocess import Popen, PIPE

erl = Popen(['escript', 'eip.escript'],
            stdin=PIPE, stdout=PIPE, stderr=PIPE)
ping = input('Ping: ')

outs, errs = erl.communicate(input=ping.encode('utf-8'),
                                 timeout=15)

print(outs.decode('utf-8'))

On the Erlang side (as you might've noticed in that Python code), a really easy way to go about this is to use the escript program, which allows us to write more-or-less self-contained Erlang scripts, like this here eip.escript:

#!/usr/bin/env escript

main(_Args) ->
  Ping = io:get_line(""),
  io:format("Pong: ~ts", [Ping]).

Now, when you run python3 eip.py and enter asdf at the Ping: prompt, you should get back Pong: asdf.

---

Doing the same thing with Elixir is only slightly more complicated: we need to create a Mix project with a bit of extra configuration and such to tell Mix to put together an escript file. So let's start with the project:

$ mix new eip
* creating README.md
* creating .formatter.exs
* creating .gitignore
* creating mix.exs
* creating lib
* creating lib/eip.ex
* creating test
* creating test/test_helper.exs
* creating test/eip_test.exs

Your Mix project was created successfully.
You can use "mix" to compile it, test it, and more:

    cd eip
    mix test

Run "mix help" for more commands.

(It's probably overkill to even use Mix for this simple example, but I'm assuming you'll eventually want to do something more advanced than this example)

Next, you'll want to add an escript option to your mix.exs, like so:

defmodule Eip.MixProject do
  use Mix.Project

  def project, do: [
    app: :eip,
    version: "0.1.0",
    elixir: "~> 1.9",
    start_permanent: Mix.env() == :prod,
    deps: deps(),
    escript: escript()
  ]

  def application, do: [extra_applications: [:logger]]

  defp deps, do: []
  defp escript, do: [main_module: Eip]
end

And finally, your lib/eip.ex module:

defmodule Eip do
  def main(_argv) do
    ping = IO.gets("")
    IO.puts("Pong: #{ping}")
  end
end

And now we just need to build it:

$ mix escript.build
Compiling 1 file (.ex)
Generated eip app
Generated escript eip with MIX_ENV=dev

eip.py will need a slight adjustment to point to this new Elixirified ping/pong IPC thingamabob:

#!/usr/bin/env python
from subprocess import Popen, PIPE, TimeoutExpired

erl = Popen(['escript', 'eip/eip'],
            stdin=PIPE, stdout=PIPE, stderr=PIPE)
ping = input('Ping: ')

outs, errs = erl.communicate(input=ping.encode('utf-8'))

print(outs.decode('utf-8'))

Unfortunately, this doesn't entirely work:

$ python3 eip.py
Ping: asdf
Pong: eof

The same results happen even when using a more direct port of the Erlang version (i.e. replacing IO.gets("") with :io.get_line("") and IO.puts("Pong: #{ping}") with :io.fwrite("Pong: ~ts", [ping]), which means something specific to Elixir's STDIN handling in general is causing it to prematurely believe it's reached end-of-file. But hey, at least one direction works!