golang TCPConn.SetWriteDeadline doesn't seem to work as expected

Question

I'm trying to detect sending failures by inspecting the error returned by golang TCPConn.Write, but it's nil. I also tried using TCPConn.SetWriteDeadline without success.

That's how things happen:

1. the server starts
2. a client connects
3. the server sends a message and the client receives it
4. the client shuts down
5. the server sends one more message: no error
6. the server sends the third message: only now the error appears

Question: why only the second message to a non-existing client results in an error? How should the case be handled properly?

The code follows:

package main

import (
    "net"
    "os"
    "bufio"
    "fmt"
    "time"
)

func AcceptConnections(listener net.Listener, console <- chan string) {

    msg := ""

    for {

        conn, err := listener.Accept()

        if err != nil {
            panic(err)
        }

        fmt.Printf("client connected\n")

        for {

            if msg == "" {
                msg = <- console
                fmt.Printf("read from console: %s", msg)
            }

            err = conn.SetWriteDeadline(time.Now().Add(time.Second))

            if err != nil {
                fmt.Printf("SetWriteDeadline failed: %v\n", err)
            }

            _, err = conn.Write([]byte(msg))

            if err != nil {
                // expecting an error after sending a message
                // to a non-existing client endpoint
                fmt.Printf("failed sending a message to network: %v\n", err)
                break
            } else {
                fmt.Printf("msg sent: %s", msg)
                msg = ""
            }
        }
    }
}

func ReadConsole(network chan <- string) {

    console := bufio.NewReader(os.Stdin)

    for {

        line, err := console.ReadString('\n')

        if err != nil {

            panic(err)

        } else {

            network <- line
        }
    }
}

func main() {

    listener, err := net.Listen("tcp", "localhost:6666")

    if err != nil {
        panic(err)
    }

    println("listening on " + listener.Addr().String())

    consoleToNetwork := make(chan string)

    go AcceptConnections(listener, consoleToNetwork)

    ReadConsole(consoleToNetwork)
}

The server console looks like this:

listening on 127.0.0.1:6666
client connected
hi there!
read from console: hi there!
msg sent: hi there!
this one should fail
read from console: this one should fail
msg sent: this one should fail
this one actually fails
read from console: this one actually fails
failed sending a message to network: write tcp 127.0.0.1:51194: broken pipe

The client looks like this:

package main

import (
    "net"
    "os"
    "io"
    //"bufio"
    //"fmt"
)

func cp(dst io.Writer, src io.Reader, errc chan<- error) {

    // -reads from src and writes to dst
    // -blocks until EOF
    // -EOF is not an error
    _, err :=  io.Copy(dst, src)

    // push err to the channel when io.Copy returns
    errc <- err
}

func StartCommunication(conn net.Conn) {

    //create a channel for errors
    errc := make(chan error)

    //read connection and print to console
    go cp(os.Stdout, conn, errc)

    //read user input and write to connection
    go cp(conn, os.Stdin, errc)

    //wait until nil or an error arrives
    err := <- errc

    if err != nil {
        println("cp error: ", err.Error())
    }
}

func main() {

    servAddr := "localhost:6666"

    tcpAddr, err := net.ResolveTCPAddr("tcp", servAddr)

    if err != nil {
        println("ResolveTCPAddr failed:", err.Error())
        os.Exit(1)
    }

    conn, err := net.DialTCP("tcp", nil, tcpAddr)

    if err != nil {
        println("net.DialTCP failed:", err.Error())
        os.Exit(1)
    }

    defer conn.Close()

    StartCommunication(conn)

}

EDIT: Following JimB's suggestion I came up with a working example. Messages don't get lost any more and are re-sent in a new connection. I'm not quite sure though how safe is it to use a shared variable (connWrap.IsFaulted) between different go routines.

package main

import (
    "net"
    "os"
    "bufio"
    "fmt"
)

type Connection struct {
    IsFaulted bool
    Conn net.Conn
}

func StartWritingToNetwork(connWrap * Connection, errChannel chan <- error, msgStack chan string) {

    for {

        msg := <- msgStack

        if connWrap.IsFaulted {

            //put it back for another connection
            msgStack <- msg

            return
        }

        _, err := connWrap.Conn.Write([]byte(msg))

        if err != nil {

            fmt.Printf("failed sending a message to network: %v\n", err)

            connWrap.IsFaulted = true

            msgStack <- msg

            errChannel <- err

            return

        } else {

            fmt.Printf("msg sent: %s", msg)
        }
    }
}

func StartReadingFromNetwork(connWrap * Connection, errChannel chan <- error){

    network := bufio.NewReader(connWrap.Conn)

    for (!connWrap.IsFaulted) {

        line, err := network.ReadString('\n')

        if err != nil {

            fmt.Printf("failed reading from network: %v\n", err)

            connWrap.IsFaulted = true

            errChannel <- err

        } else {

            fmt.Printf("%s", line)
        }
    }
}

func AcceptConnections(listener net.Listener, console chan string) {

    errChannel := make(chan error)

    for {

        conn, err := listener.Accept()

        if err != nil {
            panic(err)
        }

        fmt.Printf("client connected\n")

        connWrap := Connection{false, conn}

        go StartReadingFromNetwork(&connWrap, errChannel)

        go StartWritingToNetwork(&connWrap, errChannel, console)

        //block until an error occurs
        <- errChannel
    }
}

func ReadConsole(network chan <- string) {

    console := bufio.NewReader(os.Stdin)

    for {

        line, err := console.ReadString('\n')

        if err != nil {

            panic(err)

        } else {

            network <- line
        }
    }
}

func main() {

    listener, err := net.Listen("tcp", "localhost:6666")

    if err != nil {
        panic(err)
    }

    println("listening on " + listener.Addr().String())

    consoleToNetwork := make(chan string)

    go AcceptConnections(listener, consoleToNetwork)

    ReadConsole(consoleToNetwork)
}

Answer 1

This isn't Go specific, and is a artifact of the underlying TCP socket showing through.

A decent diagram of the TCP termination steps is at the bottom of this page: http://www.tcpipguide.com/free/t_TCPConnectionTermination-2.htm

The simple version is that when the client closes its socket, it sends a FIN, and receives an ACK from the server. It then waits for the server to do the same. Instead of sending a FIN though, you're sending more data, which is discarded, and the client socket now assumes that any more data coming from you is invalid, so the next time you send you get an RST, which is what bubbles up into the error you see.

Going back to your program, you need to handle this somehow. Generally you can think of whomever is in charge of initiating a send, is also in charge of initiating termination, hence your server should assume that it can continue to send until it closes the connection, or encounters an error. If you need to more reliably detect the client closing, you need to have some sort of client response in the protocol. That way recv can be called on the socket and return 0, which alerts you to the closed connection.

In go, this will return an EOF error from the connection's Read method (or from within the Copy in your case). SetWriteDeadline doesn't work because a small write will go though and get dropped silently, or the client will eventually respond with an RST, giving you an error.