I am new to go and trying to make a simple webcrawler. I keep getting a "panic: runtime error: invalid memory address or nil pointer dereference" and have no idea how to fix the issue. I have an "advancedFetcher" function and a "basicFetcher" function and I get the same error under either one. This answer suggests checking each error (which I have I think) but I am still getting an error. Thanks!
package main
import (
"crypto/tls"
"fmt"
"io/ioutil"
"net"
"net/http"
"time"
)
var tr = &http.Transport{
TLSClientConfig: &tls.Config{InsecureSkipVerify: true},
MaxIdleConnsPerHost: 25,
DisableKeepAlives: true,
Proxy: http.ProxyFromEnvironment,
Dial: (&net.Dialer{
Timeout: 10 * time.Second,
KeepAlive: 10 * time.Second,
}).Dial,
TLSHandshakeTimeout: 5 * time.Second,
}
var client = &http.Client{Transport: tr}
func makeGetRequest(uri string) *http.Response {
req, err := http.NewRequest("GET", uri, nil)
if err != nil {
fmt.Println(err)
}
req.Header.Add("User-Agent", "Hi it's me again")
resp, err := client.Do(req)
if err != nil {
fmt.Println(err)
}
return resp
}
func advancedFetcher(uri string, c chan int) {
fmt.Println("Getting: ", uri)
resp := makeGetRequest(uri)
defer resp.Body.Close()
c <- resp.StatusCode
}
func basicFetcher(uri string, c chan int) {
fmt.Println("Getting: ", uri)
resp, err := http.Get(uri)
if err != nil {
fmt.Println(err)
}
defer resp.Body.Close()
_, err = ioutil.ReadAll(resp.Body)
c <- resp.StatusCode
}
func main() {
c := make(chan int)
rows := []string{"https://www.google.com", "http://www.fdicconnect.gov"}
for _, row := range rows {
//go basicFetcher(row, c)
go advancedFetcher(row, c)
}
for _ = range rows {
select {
case result := <-c:
fmt.Println(result)
}
}
fmt.Println("All Done")
}
EDIT #1:
Getting: https://www.google.com
Getting: http://www.fdicconnect.gov
200
Get http://www.fdicconnect.gov: dial tcp 167.176.6.86:80: i/o timeout
panic: runtime error: invalid memory address or nil pointer dereference
[signal 0xb code=0x1 addr=0x0 pc=0x400ff2]
goroutine 21 [running]:
runtime.panic(0x6594c0, 0x7e9b53)
/usr/local/go/src/pkg/runtime/panic.c:279 +0xf5
main.advancedFetcher(0x6c3b70, 0x1a, 0xc208004180)
/home/me/go/src/testfolder/tester/main.go:41 +0x1c2
created by main.main
/home/me/go/src/testfolder/tester/main.go:61 +0x123
goroutine 16 [chan receive]:
main.main()
/home/me/go/src/testfolder/tester/main.go:65 +0x198
goroutine 19 [finalizer wait]:
runtime.park(0x413bd0, 0x7ee660, 0x7ec8c9)
/usr/local/go/src/pkg/runtime/proc.c:1369 +0x89
runtime.parkunlock(0x7ee660, 0x7ec8c9)
/usr/local/go/src/pkg/runtime/proc.c:1385 +0x3b
runfinq()
/usr/local/go/src/pkg/runtime/mgc0.c:2644 +0xcf
runtime.goexit()
/usr/local/go/src/pkg/runtime/proc.c:1445
goroutine 17 [syscall]:
runtime.goexit()
/usr/local/go/src/pkg/runtime/proc.c:1445
Is your program printing an error before panicking? If memory serves, if http.Get
returns an error, then resp.Body == nil
, which would cause your error because you can't call resp.Body.Close()
on nil
.
As twotwotwo suggests, you should write your function so that it returns a (result, error) pair so you can terminate it gracefully before running into problems.
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With