Any readline binding support for elixir?

Question

I'm trying to create a program which accepts user input as similar way as iex or erl does (ex. when pressing allow keys to navigate previous histories).

If the standard IO.gets is used as follows,

IO.gets "user> "

The console ends up with the following when up-allow is pressed.

user> ^[[A^[[A

Is there any function/library to have readline capability which can be used inside elixir code?

---

What I have investigated so far is,

• Some languages have binding support for readline libraries, but I haven't been able to find corresponding capability for elixir.
• iex implementation seems to delegate this capability to erl (/lib/iex/history.ex seems to just managing list of history), but I haven't been able to find out corresponding functionality in erlang side.
• I've tried erl_ddll.load, but failing to go further from the following.

on iex,

iex(4)> :erl_ddll.load('/usr/lib', 'libreadline')
{:error, {:open_error, -10}}
iex(5)> :erl_ddll.format_error({:open_error, -10})
'dlopen(/usr/lib/libreadline.so, 2): image not found'

I'm on OSX and installed libreadline through homebrew, and I can find libreadline.dylib in /usr/lib.

---

[Additional notes about purpose]

I was experimenting on the following (mal) with elixir, which is a lisp repl implemented in various languages (but not with elixir/erlang).

https://github.com/kanaka/mal

The part of the step is implementing repl with history, and some languages are using readline binding libraries if there's not native one.

[A little more update - 2015/3/22]

I was trying with NIF approach (as similar as encurses) to use readline library. I could make it somewhat work on erlang (erl), but stuck on elixir side. When reading inputs from C libraries (readline or just plain scanf), "mix run -e" or "iex" seems to behave a little weird (skips or ignores some inputs), but couldn't find out the reason. encurses seems behaving similar.

The following was my trials.

https://github.com/parroty/ereadline

https://github.com/parroty/readline

I may be going with more general approach like rlwrap.

Answer 1

Disclaimer: I'm by no means an expert on the subject of jury-rigging Erlang's shell code to do one's bidding. Corrections or clarifications to this answer are very welcome. This was a learning process for me, too.

tl;dr: erl and iex rely on cooperation between the edlin module and the tty_sl port to implement their Readline-like features. You can use these, too, though official documentation for doing so is lacking, to say the least.

---

Erlang's shell (upon which Elixir's is built) is quite a bit more complex than a typical REPL. Unlike a typical REPL, it's not just looping over input and evaluating it; it's actually built much like a typical OTP application, with supervision trees all the way down.

This article by the author of Learn You Some Erlang for Great Good! goes into further detail on the architecture of the whole Erlang shell. To summarize:

• A platform-specific TTY driver passes user input to user_drv
• user_drv either drops into shell management mode (if it receives ^G or ^C) or passes input to the currently-selected group (there can be multiple groups and therefore shells; when you press ^G, you have the option of creating more groups, switching to existing groups, etc.)
• group works with edlin to put together a line of code for evaluation; once it has a line, it sends it to shell
• shell does the actual evaluation, then sends the result to group, which sends the result to user_drv
• If the group sending things to user_drv is the active group, then user_drv passes it on to the TTY driver (and thus to the user); else, it's muted

The part of this process that's relevant to the question is edlin, which is an Erlang implementation of Readline-like functionality. edlin is, unfortunately, not documented particularly well as far as I can tell, but the gist of using it in Elixir (based on what I'm able to glean out of lib/kernel/src/group.erl) is the following:

• Call :edlin.init in the process that's using edlin (this sets up a "kill buffer", in the Emacs sense of a "kill ring")
• When you're ready to read a line, call {:more_chars,continuation,requests} = :edlin.start(prompt), where prompt is a charlist representing - you guessed it - your shell's command-line prompt
• Handle requests, which should be in the form of [{:put_chars,:unicode,prompt}]; in the Erlang shell, "handle" means "send to user_drv to be printed", but in your case this might be different

At this point, the looping starts. In each iteration, you'll be calling :edlin.edit_line(characters,continuation) (where characters is a character list, i.e. from user input). Each call will give you one of the following tuples:

• {:done,line,rest,requests}: edlin encountered a newline and is done processing your line; at this point, you can do whatever you want with line (your line) and rest (all the characters following your line)
• {:more_chars,continuation,requests}: edlin needs more characters; call :edlin.edit_line(characters,continuation)
• {:blink,continuation,requests}: I'm not 100% sure here, but I think this has to do with edlin highlighting characters (like when the cursor jumps to a matching ( if you type ))
• {:undefined,character,rest,continuation,requests}: Not 100% sure here, either, but I think it has to do with handling things like command history

In all cases, requests will be a list of tuples corresponding to instructions for user_drv, usually for things like writing characters, moving the cursor, etc.

---

Next is the question of dealing with the TTY. user_drv.erl does this with something called tty_sl, which is an Erlang port (that is, an external program designed to behave like an Erlang process) with different versions for Windows and Unix. The basic procedure (again, Elixirified):

• Define the following (we'll need it later):

  def put_int16(num, tail) do  # we need this in a bit
    use Bitwise  # because macros
    [num |> bsr(8) |> band(255), num |> band(255) | tail]
  end
  

• Call port = Port.open {:spawn,'tty_sl -c -e'} (-e for "echo", -c for whatever "canon" means); there's a bit more error-checking in this step for user_drv, apparently so that it can start the older user instead (which - per the above-linked article - appears to be an older version of the Erlang shell)

• Spin up the edlin-using process described above and store it in, say, shell (user_drv does a bunch more stuff here in order to setup multiple groups)
• Start looping to handle requests from your shell

Then, in the loop:

• Receive a request (really a list of requests per above)

• Convert each request to something that tty_sl understands:

  command = case request do
    {:put_chars,:unicode,chars} ->            # OP_PUTC
      {:command, [0|:unicode.characters_to_binary(chars,:utf8)]}
    {:move_rel,count} ->                      # OP_MOVE
      {:command, [1|put_int16(count, [])]}
    {:insert_chars,:unicode,chars} ->         # OP_INSC
      {:command, [2|:unicode.characters_to_binary(chars,:utf8)]}
    {:delete_chars,count} ->                  # OP_DELC
      {:command, [3|put_int16(count, [])]}
    :beep ->                                  # OP_BEEP
      {:command, [4]}
    {:put_chars_sync,:unicode,chars,reply} -> # OP_PUTC_SYNC
      {{:command, [5|:unicode.characters_to_binary(chars,:utf8)]}, reply}
    else ->
      else
  end
  

• Send command to the TTY:

  result = case command do
    {:requests,requests} ->
      # Handle more requests
    {:command,_} = command ->
      send port, command
      :ok
    {command,reply} ->
      send port, command
      reply
    _ ->
      :ok
  end
  

You'll also need to receive stuff from the TTY. In the case of user_drv, the TTY sends messages to the same user_drv process as the group processes. In any case, you'll want to handle some additional messages besides the requests sent through group by edlin:

• {port,{:data,bytes}}: Convert bytes to characters and send to your shell. Since we're in Elixir-land, we might not even need to do the conversion.
• {port,:eof}: Similar deal; send the :eof to your shell
• {port,:ok}: Not 100% sure about this from user_drv, but I believe it has to do with the :put_chars_sync command, given that the code in user_drv to handle this message deals with a Reply variable and that the only tty_sl command involving a reply is :put_chars_sync

The rest of the messages handled in user_drv pertain to the supervision tree (namely: handling process exits for both tty_sl and the various groups).

---

Of course, there's probably a much simpler answer to all of this: just use user_drv and create yourself a new shell. This can be done (I think; not 100% sure here) with something along the lines of user_drv_pid = :user_drv.start('tty_sl -c -e', {MyShell,:start}). This seems to be how iex works (see IEx.CLI.start/0).