How to simulate Ruby's until loop in Elixir?

Question

I'm converting a Ruby project to Elixir. How does Ruby's until loop translate to Elixir?

until scanner.eos? do
  tokens << scan(line + 1)
end

Here's the full Ruby method:

def tokenize
  @tokens = []
  @lines.each_with_index do |text, line|
    @scanner = StringScanner.new(text)
    until @scanner.eos? do
      @tokens << scan(line + 1)
    end
  end
  @tokens
end

@lines is just a text file split by new lines. @lines = text.split("\n")

In Elixir, I've already converted the string scanner which looks like this: StringScanner.eos?(scanner):

@spec eos?(pid) :: boolean
def eos?(pid) when is_pid(pid) do

Also, in Elixir, tokens are tuples: @type token :: {:atom, any, {integer, integer}}. Where the {integer, integer} tuple is the line and position of the token.

This is the Elixir psuedo-code which doesn't quite work.

@spec scan(String.t, integer) :: token
def scan(text, line) when is_binary(text) and is_integer(line) do
  string_scanner = StringScanner.new(text)
  until StringScanner.eos?(string_scanner) do
    result = Enum.find_value(@scanner_tokenizers, fn {scanner, tokenizer} ->
      match = scanner.(string_scanner)
      if match do
        tokenizer.(string_scanner, match, line)
      end
    end)
    IO.inspect result
  end
  StringScanner.stop(string_scanner)
  result
end

Someone on the slack channel suggested using recursion, however they didn't elaborate with an example. I've seen recursion examples for summing / reducing which use accumulators etc. However, I don't see how that applies when evaluating a boolean.

Can anyone provide a working example which uses StringScanner.eos?(scanner)? Thanks.

Answer 1

It may be something like

def tokens(scanner) do
  tokens(scanner, [])
end

defp tokens(scanner, acc) do
  if StringScanner.eos?(scanner) do
    acc
  else
    tokens(scanner, add_to_acc(scan_stuff(), acc))
  end
end

At least this can be the general idea. As you'll see I kept a couple of functions very generic (scan_stuff/0 and add_to_acc/2) as I don't know how you mean to implement those; the first one is meant to do what scan(line + 1) does in the Ruby code, while the second one is meant to do what << does in the Ruby code (e.g., it could add the scanned stuff to the list of tokens or something similar).