Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

golang force net/http client to use IPv4 / IPv6

Tags:

go

web-crawler

I' using go1.11 net/http and want to decect if a domain is ipv6-only.

What did you do?

I create my own DialContext because want I to detect if a domain is ipv6-only. code below

package main
import (
    "errors"
    "fmt"
    "net"
    "net/http"
    "syscall"
    "time"
)
func ModifiedTransport() {
    var MyTransport = &http.Transport{
        DialContext: (&net.Dialer{
            Timeout:   30 * time.Second,
            KeepAlive: 30 * time.Second,
            DualStack: false,
            Control: func(network, address string, c syscall.RawConn) error {
                if network == "ipv4" {
                    // I want to  cancel connection here client.Get("http://myexample.com") return a non-nil err.
                    return errors.New("you should not use ipv4")
                }
                return nil
            },
        }).DialContext,
        MaxIdleConns:          100,
        IdleConnTimeout:       90 * time.Second,
        TLSHandshakeTimeout:   10 * time.Second,
        ExpectContinueTimeout: 1 * time.Second,
    }
    var myClient = http.Client{Transport: MyTransport}
    resp, myerr := myClient.Get("http://www.github.com")
    if myerr != nil {
        fmt.Println("request error")
        return 
    }
    var buffer = make([]byte, 1000)
    resp.Body.Read(buffer)
    fmt.Println(string(buffer))
}
func main(){
    ModifiedTransport();
}

I do not now how to close the request even when I can get into network == "ipv4".

addition

Python can solve the question via Force requests to use IPv4 / IPv6. I do not know how to do it in golang. Could someone one help me? Thanks so much!

like image 675
fang jinxu Avatar asked Oct 30 '18 00:10

fang jinxu


3 Answers

The network passed to the Control function is either tcp4 for an IPv4 connection or tcp6 for an IPv6 connection, if you are making an outgoing TCP connection.

From the comments on type Dialer:

        // Network and address parameters passed to Control method are not
        // necessarily the ones passed to Dial. For example, passing "tcp" to Dial
        // will cause the Control function to be called with "tcp4" or "tcp6".

(In case of non-TCP connection, other strings are possible.)

Known networks are "tcp", "tcp4" (IPv4-only), "tcp6" (IPv6-only), "udp", "udp4" (IPv4-only), "udp6" (IPv6-only), "ip", "ip4" (IPv4-only), "ip6" (IPv6-only), "unix", "unixgram" and "unixpacket".

like image 64
Michael Hampton Avatar answered Oct 20 '22 04:10

Michael Hampton


Oh. I fixed the problem myself.

We could not configure force ipv6 connection because it is hard coded

...
if cm.scheme() == "https" && t.DialTLS != nil {
        var err error
        pconn.conn, err = t.DialTLS("tcp", cm.addr())
        if err != nil {
            return nil, wrapErr(err)
        }
...

( code here.)

I add a ipv6only flag for transport.go, a getTcpString() and it works.

(diff here)

like image 2
fang jinxu Avatar answered Oct 20 '22 03:10

fang jinxu


This works for me with go1.17.1:

var (
    zeroDialer net.Dialer
    httpClient = &http.Client{
        Timeout: 10 * time.Second,
    }
)

func init() {
    transport := http.DefaultTransport.(*http.Transport).Clone()
    transport.DialContext = func(ctx context.Context, network, addr string) (net.Conn, error) {
        return zeroDialer.DialContext(ctx, "tcp4", addr)
    }
    httpClient.Transport = transport
}
like image 1
tmm1 Avatar answered Oct 20 '22 04:10

tmm1