How to capture each line from System.cmd output?

Question

How to capture each line written in console when run commands from System.cmd or by other approach?

I want even capture final results but what is displayed in console in this case it will be similar to: Cloning into 'myrepo'... remote: Counting objects: 3271, done. and send each line via channels:

case System.cmd("git", ["clone", "[email protected]:#{vault}/#{repo}.git"], cd: repo) do
  {results, 0} ->
    Myapp.Endpoint.broadcast("app:setup", "new:line", results)
  {_, code} ->
    raise RuntimeError, "`git clone` failed with code #{code}"
end

I haven't found solution, there was similar issues but without clear answer: question or question

Answer 1

So, there are a couple approaches here, and I'm going to try to sum up and have you be able to make sense of the previous answers in the other questions.

First of all, you should probably know that handling this stuff in detail, you need to learn how to use :erlang.open_port/2. You can pass {:line, max_length} to the options to get 1 message per line. The output from git that you are seeing is stuff being written to stderr, and you can pass :stderr_to_stdout to redirect that so they come in 1 line per message. You can loop using receive until you receive an eof message, and you can see the docs for more details on when that eof message will be emitted.

bitwalker's answer in your second link will get you what you want with some modification:

defmodule Shell do
  def exec(exe, args, opts \\ [:stream]) when is_list(args) do
    port = Port.open({:spawn_executable, exe}, opts ++ [{:args, args}, :binary, :exit_status, :hide, :use_stdio, :stderr_to_stdout])
    handle_output(port)
  end

  def handle_output(port) do
    receive do
      {^port, {:data, data}} ->
        IO.inspect(data) # Replace this with the appropriate broadcast
        handle_output(port)
      {^port, {:exit_status, status}} ->
        status
    end
  end
end

Now, while we can redirect stderr to stdout, one of the problems here is that git will detect a redirected stream and adjust how much it streams accordingly. I want you to compare the output of these calls:

gitcmd = System.find_executable("git")
Shell.exec(gitcmd, ["clone","--progress",url], [{:line, 4096}])

This prints out 1 message for each "\n" it finds in the stream. Notice all that junk with \r? That's how the progress comes out. You can remove --progress from the args, and you'll just get 1 or 2 boring lines. If you want to get everything, you need to stream the results, and then split by line-ish output yourself.

Shell.exec(gitcmd, ["clone","--progress",url], [:stream])

You can see from using :stream instead of :line we'll get everything as it's being flush'd from the other side. You'll have to clean up the dangling \r and \n yourself, and you may want to plan for receiving part of a line, but I think this should get you well onto your journey.

This was all about redirecting stderr to stdout, but if you need to actually retain the difference between stderr and stdout, and if you want the ability to kill the process, rather than depend on it closing after you close stdin, you're going to have to rely on something like porcelain which manages a well-behaved process that will "manage" the other process for you.