Whether go uses shared memory or distributed computing

Question

Go has the slogan "Do not communicate by sharing memory; instead, share memory by communicating". I was wondering whether Go uses shared memory or distributed computing approach. For example, for MPI it is clearly distributed, OpenMP is clearly shared memory; but I was not sure about Go, which is unique.

I have seen many posts, such as Shared memory vs. Go channel communication, effective Go document etc., but could not clarify. Thanks in advance.

Answer 1

Go does not prevent you from sharing memory between goroutines/threads. What they mean by communicating, is that you send a chunk of data, or a pointer to said chunk, across a channel. This effectively transfers 'ownership' of the data to the target reader of the channel. Mind you, this transfer of ownership is not enforced by the language or the runtime, it is just by convention.

You are still perfectly capable of writing to the same memory from two goroutines, if you so choose. In other words: Go does not prevent you from shooting yourself in the foot, it just provides language semantics which make these mistakes easier to detect.

If a value is passed into a channel, the programmer must then assume that value is no longer his to write to in the same goroutine.

func F(c chan *T) {
    // Create/load some data.
    data := getSomeData()

    // Send data into the channel.
    c <- data

    // 'data' should now be considered out-of-bounds for the remainder of
    // this function. This is purely by convention, and is not enforced
    // anywhere. For example, the following is still valid Go code, but will
    // lead to problems.
    data.Field = 123
}

Answer 2

The question assumes that shared memory and distributed computing are opposites. That's a bit like asking: Are RAM and LAN opposites? It would be clearer to differentiate between shared memory concurrency within a CPU/memory node and between CPU/memory nodes.

This is part of a bigger picture of parallel processing research. There have been many research projects, including:

• developing non-Von-Neumann computers that have multiple CPUs sharing a single memory, joined by some form of switching fabric (often a Clos network). OpenMP would be a good fit for these.

• developing parallel computers that consist of a collection of CPUs, each with their own separate memory, and with some communications fabric between the nodes. This is typically the home of MPI, amongst others.

The first case is specialised in the High Performance Computing fraternity. It is the latter case that is familiar to most of us. In this case, usually these days the comms is simply via Ethernet, but various faster lower-latency alternatives have been (successfully) developed for certain niches (eg IEEE1355 SpaceWire, which emerged from the Transputer serial links).

For many years, the dominant view was that efficient parallelism would only be possible if the memory was shared, because the cost of communication by passing messages was (naively) assumed to be prohibitive. With shared-memory concurrency, the difficulty is in the software: because everything is interdependent, designing the concurrency gets combinatorially harder and harder as systems get larger. Hard-core expertise is needed.

For the rest of us, Go follows Erlang, Limbo and of course Occam in promoting the passing of messages as the means to choreograph the work to be done. This arises from the algebra of Communicating Sequential Processes, which provides the basis for creating parallel systems of any size. CSP designs are composable: each subsystem can itself be a component of a larger system, without a theoretical limit.

Your question mentioned OpenMP (shared-memory) and MPI (distributed memory message passing), which can be used together. Go could be considered to be approximately equivalent of MPI in that it promotes message passing. It does however also allow locks and shared memory. Go is different from both MPI and OpenMP because it is not explicitly concerned with multi-processor systems. To progress into the world of parallel processing using Go, a network message passing framework would be needed, such as OpenCL, for which someone is working on a Go API.