How to learn internals of the Go Programming Language? For noob

Question

Recently I've participated several Go job interviews. The first one asked me How is channel implemented?, then the second one asked How is goroutine implemented?. Well as you can guess, the next one asked How is a Go interface implemented?.

I've been using Go for six months, but to be honest I never did care or know these Go internals.

I tried to learn these by reading the source code of Go, but can't really understand the quintessence.

So the question is, for a noob in Go, how do I learn the Go internals?

Answer 1

The most organized collection of internal resources links is probably this:

Golang Internals Resources

Other than that, answers to these questions aren't collected in once place, but they are scattered in different blog posts.

Slice internals: The Go Blog: usage and internals

String internals: The Go Blog: Strings, bytes, runes and characters in Go

Constants internals: The Go Blog: Constants

Reflection insight: The Go Blog: The Laws of Reflection

Interface internals: RSC: Go Data Structures: Interfaces

Channel implementation: Overview on SO: How are Go channels implemented?

Channel internals: Go channel on steroids

Map implementation: Overview on SO: Golang map internal implementation - how does it search the map for a key?; also related: Go's maps under the hood

Map internals: Macro View of Map Internals In Go

Answer 2

Let me warn you that you may be missing the real point of the interviewers.

(Disclaimer: I do job interviews of Go programmers from time to time, for a somewhat demanding project, so all of the below is my personal world view. Still, it is shared by my cow-orkers ;-)).

Most of the time, it's rather worthless for an employee to know precisely how this or that bit of the runtime (or the compiler) is implemented—in part because this may change in any future release, and in part because there do exist at least two up-to-date implementations of Go ("the gc suite", and a part of GCC), and they all are free to implement a particular feature in any way they wish.

What a (sensible) interviewer should really be interested in is whether you understand "the why" of a particular core feature.

Say, when they ask you to explain how a channel is implemented, what they should be interested to hear from you is that a channel provides synchronization and may also provide buffering. So you may tell them that a channel is like a variable protected by a mutex—for the case of an unbuffered channel,—or like a slice protected by a mutex. And then add that an upshot of using a channel instead of a hand-crafted solution involving a mutex is that operations on channels can be easily combined using the select statement, while implementing a matching functionality without channels is possible (and by that time they will probably would like to hear from you about sync.Cond) but is really cumbersome and error-prone.

It's not actually completely worthless to know nitty-gritty details of channels—say to know that their implementation tries hard to lower the price paid for the synchronization in the "happy case" (there is no contention at the moment a goroutine accesses a channel), and that it is also clever about not jumping straight into the kernel to sleep on a lock in the "unhappy case", but I see no point in knowng these details by heart.

The same applies to goroutines. You should maintain a clear picture in what are the differences between an OS process and a thread running in it, and what context belongs to a thread—that is, what is needed to be saved and restored when switching between the threads. And then the differences between an OS thread and a "green thread" which a goroutine mostly is. It's okay to just know who schedules OS threads and who schedules goroutines, and why the latter is faster. And what are the benefits of having goroutines (the main one is network poller integrated into the scheduler (see this), the second is dynamic stacks, the third is low context switching overhead most of the time).

My recommendation is to read through the list presented by @icza.

And in addition to what you've asked about, I'd present the following list of what a good candidate should be familiar with—in the order from easiest to hardest (to grok):

• The mechanics of slices and append. You should know arrays exist and how they are different from slices.

• How interfaces are implemented.

• The dualistic nature of Go strings (given s contains a string, what is the difference between iterating over its contents via for i := 0; i < len(s); i++ { c := s[i] } over for i, c := range s {}).

  Also: what kinds of data strings may contain—you should know that it's perfectly okay to contain arbitrary binary data in them, UTF-8 is not a requirement.

  The differences between string and []byte.

• How blocking I/O is implemented (for the network; that's about the netpoller integrated into the runtime).

  Knowing about the difference in handing non-network blocking I/O and syscalls in general is a bonus.

• How the scheduler is implemented (those Ps running Gs on Ms).