Golang equivalent of creating a subprocess in Python

Question

I am attempting to convert a Python script to Golang, just really to see performance differences and to help me learn Golang more.

In Python, I have 2 scripts. One is a script that runs an infinite loop and sleeps for a minute before running again. The code checks an endpoint on my server and reads the output, then determines if it needs to do anything. If it does, it handles the output and starts a new subprocess. The subprocess is another Python script that does a lot of calculations and creates hundreds of threads. There can be multiple subprocesses running at any given time and they're all different tasks for different users.

I've got my Golang code reading from the API, and it's ready to start a new subprocess. But I'm not really sure how I go about it.

I know when I've created the new subprocess (or whatever it is that is the Go equivalent) I can create a bunch of Go routines, but really I'm just stuck on the "subprocess" bit.

I have tried using Go routines as a substitute for the subprocess but I don't think this is the way to go?

As requested for visualisation purposes, I have added an example of the code.

api.py:

while True:
    someparameter = 'randomIDfromdatabase'
    subprocess.Popen(["python3", "mycode.py", someparameter])
    time.sleep(60)

mycode.py

parameter = sys.argv[1]
for i in range(0, 100):
    thread.append(MyClass(parameter))
    thread.start()

I basically need the Golang equivalent of "subprocess.Popen".

Answer 1

You can use the Go os/exec package for subprocess-like behavior. For example, here's a trivial program that runs the date program in a subprocess and reports its stdout:

package main

import (
    "fmt"
    "log"
    "os/exec"
)

func main() {
    out, err := exec.Command("date").Output()
    if err != nil {
        log.Fatal(err)
    }
    fmt.Printf("The date is %s\n", out)
}

A more interesting example from gobyexample that shows how to interact with the stdio/stdout of launched processes:

package main

import "fmt"
import "io/ioutil"
import "os/exec"

func main() {

    // We'll start with a simple command that takes no
    // arguments or input and just prints something to
    // stdout. The `exec.Command` helper creates an object
    // to represent this external process.
    dateCmd := exec.Command("date")

    // `.Output` is another helper that handles the common
    // case of running a command, waiting for it to finish,
    // and collecting its output. If there were no errors,
    // `dateOut` will hold bytes with the date info.
    dateOut, err := dateCmd.Output()
    if err != nil {
        panic(err)
    }
    fmt.Println("> date")
    fmt.Println(string(dateOut))

    // Next we'll look at a slightly more involved case
    // where we pipe data to the external process on its
    // `stdin` and collect the results from its `stdout`.
    grepCmd := exec.Command("grep", "hello")

    // Here we explicitly grab input/output pipes, start
    // the process, write some input to it, read the
    // resulting output, and finally wait for the process
    // to exit.
    grepIn, _ := grepCmd.StdinPipe()
    grepOut, _ := grepCmd.StdoutPipe()
    grepCmd.Start()
    grepIn.Write([]byte("hello grep\ngoodbye grep"))
    grepIn.Close()
    grepBytes, _ := ioutil.ReadAll(grepOut)
    grepCmd.Wait()

    // We ommited error checks in the above example, but
    // you could use the usual `if err != nil` pattern for
    // all of them. We also only collect the `StdoutPipe`
    // results, but you could collect the `StderrPipe` in
    // exactly the same way.
    fmt.Println("> grep hello")
    fmt.Println(string(grepBytes))

    // Note that when spawning commands we need to
    // provide an explicitly delineated command and
    // argument array, vs. being able to just pass in one
    // command-line string. If you want to spawn a full
    // command with a string, you can use `bash`'s `-c`
    // option:
    lsCmd := exec.Command("bash", "-c", "ls -a -l -h")
    lsOut, err := lsCmd.Output()
    if err != nil {
        panic(err)
    }
    fmt.Println("> ls -a -l -h")
    fmt.Println(string(lsOut))
}

---

Note that goroutines have little to do with subprocesses. Goroutines are a way to do multiple things concurrently in a single Go process. That said, when interacting with subprocesses, goroutines often come in handy because they help waiting for subprocesses to finish while also doing other things in the launching (main) program. But the details of this are very specific to your application.