Golang net.Listen binds to port that's already in use

Question

Port 8888 is already bound on my (OS X 10.13.5) system, by a process running inside a docker container:

$ netstat -an | grep 8888
tcp6       0      0  ::1.8888               *.*                    LISTEN
tcp4       0      0  *.8888                 *.*                    LISTEN

A python program which tries to bind to that port (using as close to the socket options of golang as I can manage), fails in the way I expect:

import socket
import fcntl
import os


def main():
    sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
    flag = fcntl.fcntl(sock.fileno(), fcntl.F_GETFL)
    fcntl.fcntl(sock.fileno(), fcntl.F_SETFL, flag | os.O_NONBLOCK)
    sock.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
    sock.bind(("0.0.0.0", 8888))
    sock.listen(5)


main()

fails with:

$ python test.py
Traceback (most recent call last):
  File "test.py", line 15, in <module>
    main()
  File "test.py", line 11, in main
    sock.bind(("0.0.0.0", 8888))
OSError: [Errno 48] Address already in use

But a go program creating a connection via net.Listen does not fail, as I expect it to:

package main

import (
    "fmt"
    "net"
)

func main() {
    _, err := net.Listen("tcp", "0.0.0.0:8888")
    if err != nil {
        fmt.Printf("Connection error: %s\n", err)
    } else {
        fmt.Println("Listening")
    }
}

Succeeds with:

$ go run test.go
Listening

A coworker reports that with the same setup, his Ubuntu system correctly fails the go program.

Why does this succeed on a Mac, and how can I get the net.Listen to show an error in binding to port 8888?

edit: If I occupy port 8888 with a simple go program like:

package main

import (
    "log"
    "net/http"
)

func main() {
    log.Fatal(http.ListenAndServe("0.0.0.0:8888", nil))
}

Then test.go correctly fails to bind to the port. However the docker process (which is running basically that ^^^) does not cause it to fail.

edit 2: If I specify "tcp4", then the program does indeed fail as I expect. If I specify "tcp6", it succeeds but netstat says it binds to * instead of ::1:

$ netstat -an | grep 8888
tcp6       0      0  *.8888                 *.*                    LISTEN
tcp6       0      0  ::1.8888               *.*                    LISTEN
tcp4       0      0  *.8888                 *.*                    LISTEN

So, specifying "tcp4" will solve my actual problem, but I really want to understand what the heck is going on with the "tcp46" connection type, and I can't find any documentation. Help!

Answer 1

OK, I think I have a story to tell about why this happens:

1. Docker on mac, when mapping a port, binds to IPv4 0.0.0.0:<port> and IPv6 [::1]:<port>. Note that on IPv6 it maps to the equivalent of localhost rather than 0.0.0.0, which would be [::]!
2. Golang, when opening a socket to listen on, by default opens an IPv6 socket that is mapped in some way to also listen to IPv4. (I still don't completely understand this tcp46 type, so if you have good docs please point me to them!).
3. So my golang program was opening an IPv6 socket on [::]:8888, the equivalent of 0.0.0.0:8888 in IPv6. This succeeded because docker was listening on [::1] (the equivalent of 127.0.0.1), not [::]
4. That's it! So the golang program succeeded even though it will only get connected to by a client connecting on IPv6 from a non-loopback address (I think, I got too tired to test this, give me a break)

My coworker reports that on Ubuntu, docker listens on [::], which is why he was unable to reproduce the problem I was seeing. This seems like the sensible behavior! And I have no idea why it doesn't do so on mac.

I also think it's surprising and possibly a bit wrong that Go happily succeeds in this instance, even though it's creating a socket that's very difficult to actually access? But I can't say that it's definitely a bug, and I definitely don't feel like trying to report it as such to the go project.

Answer 2

Regarding the tcp46 output from netstat, I can't find any documentation but I did find the relevant source.

From network_cmds-543/netstat.proj/inet.c:

void
protopr(uint32_t proto,     /* for sysctl version we pass proto # */
        char *name, int af)
{

...

    struct xinpcb_n *inp = NULL;

...

            const char *vchar;

#ifdef INET6
            if ((inp->inp_vflag & INP_IPV6) != 0)
                vchar = ((inp->inp_vflag & INP_IPV4) != 0)
                ? "46" : "6 ";
            else
#endif
                vchar = ((inp->inp_vflag & INP_IPV4) != 0)
                ? "4 " : "  ";

xinpcb_n is defined in bsd/netinet/in_pcb.h.

 * struct inpcb captures the network layer state for TCP, UDP and raw IPv6
 * and IPv6 sockets.

Elsewhere in that file, the inp_vflag is documented as /* INP_IPV4 or INP_IPV6 */. Those are in turn defined as:

#define INP_IPV4    0x1
#define INP_IPV6    0x2

So basically, when a socket has both v4 and v6 bits set, it will display 46 in the protocol column.

---

Regarding Go, there's this socket() function in the net package:

func socket(ctx context.Context, net string, family, sotype, proto int, ipv6only bool, laddr, raddr sockaddr, ctrlFn func(string, string, syscall.RawConn) error) (fd *netFD, err error) {

...

if err = setDefaultSockopts(s, family, sotype, ipv6only); err != nil {

setDefaultSockopts() has per platform definitions, here's an excerpt from the BSD variant:

func setDefaultSockopts(s, family, sotype int, ipv6only bool) error {

...

    if supportsIPv4map() && family == syscall.AF_INET6 && sotype != syscall.SOCK_RAW {
        // Allow both IP versions even if the OS default
        // is otherwise. Note that some operating systems
        // never admit this option.
        syscall.SetsockoptInt(s, syscall.IPPROTO_IPV6, syscall.IPV6_V6ONLY, boolint(ipv6only))

So, allowing of both IP versions on a socket is driven by the ipv6Only boolean. After a little more digging, I found where this is decided and includes a detailed explanation of the logic:

// favoriteAddrFamily returns the appropriate address family for the
// given network, laddr, raddr and mode.
//
// If mode indicates "listen" and laddr is a wildcard, we assume that
// the user wants to make a passive-open connection with a wildcard
// address family, both AF_INET and AF_INET6, and a wildcard address
// like the following:
//
//  - A listen for a wildcard communication domain, "tcp" or
//    "udp", with a wildcard address: If the platform supports
//    both IPv6 and IPv4-mapped IPv6 communication capabilities,
//    or does not support IPv4, we use a dual stack, AF_INET6 and
//    IPV6_V6ONLY=0, wildcard address listen. The dual stack
//    wildcard address listen may fall back to an IPv6-only,
//    AF_INET6 and IPV6_V6ONLY=1, wildcard address listen.
//    Otherwise we prefer an IPv4-only, AF_INET, wildcard address
//    listen.
//
//  - A listen for a wildcard communication domain, "tcp" or
//    "udp", with an IPv4 wildcard address: same as above.
//
//  - A listen for a wildcard communication domain, "tcp" or
//    "udp", with an IPv6 wildcard address: same as above.
//
//  - A listen for an IPv4 communication domain, "tcp4" or "udp4",
//    with an IPv4 wildcard address: We use an IPv4-only, AF_INET,
//    wildcard address listen.
//
//  - A listen for an IPv6 communication domain, "tcp6" or "udp6",
//    with an IPv6 wildcard address: We use an IPv6-only, AF_INET6
//    and IPV6_V6ONLY=1, wildcard address listen.
//
// Otherwise guess: If the addresses are IPv4 then returns AF_INET,
// or else returns AF_INET6. It also returns a boolean value what
// designates IPV6_V6ONLY option.
//
// Note that the latest DragonFly BSD and OpenBSD kernels allow
// neither "net.inet6.ip6.v6only=1" change nor IPPROTO_IPV6 level
// IPV6_V6ONLY socket option setting.