: first path segment in URL cannot contain colon

Question

Here's my code ( part of it ) :

type SitemapIndex struct {
    // Locations []Location `xml:"sitemap"`
    Locations []string `xml:"sitemap>loc"`
}

~~~ SNIP ~~~
func main(){
    var s SitemapIndex
    resp, _ := http.Get("https://www.washingtonpost.com/news-sitemaps/index.xml")
    bytes, _ := ioutil.ReadAll(resp.Body)
    xml.Unmarshal(bytes, &s)
    for _, Location := range s.Locations {
        fmt.Printf("%s\n", Location)
        resp, err := http.Get(Location)
        if err != nil {
            log.Fatal(err)
        } else {
            bytes, _ := ioutil.ReadAll(resp.Body)
            xml.Unmarshal(bytes, &n)
            for idx := range n.Titles {
                newsMap[n.Titles[idx]] = NewsMap{n.Keywords[idx], n.Locations[idx]}
            }
        }
        for idx, data := range newsMap {
            fmt.Println("\n\n\n", idx)
            fmt.Println("\n", data.Keyword)
            fmt.Println("\n", data.Location)
        }
    }

Now, when I run this code I get this output :

https://www.washingtonpost.com/news-sitemaps/politics.xml

2019/01/28 02:37:13 parse 
https://www.washingtonpost.com/news-sitemaps/politics.xml
: first path segment in URL cannot contain colon
exit status 1

I read a few posts and did some experiment myself, like I made another file with the following code

package main

import ("fmt"
    "net/url")

func main(){
    fmt.Println(url.Parse("https://www.washingtonpost.com/news-sitemaps/politics.xml"))
}

And it didn't throw any error, so I understand the error is not with the url .

Now, I just started learning Go using sentdex's tutorials , a few hours ago and so don't have much idea as of now. Here's the video link

Thanks and regards. Temporarya

Answer 1

The problem here is that Location has whitespace prefix and suffix so string is not valid URL. Unfortunately, error message does not help to see that.

How to detect:

I typically use %q fmt helper that wraps string into parentheses:

fmt.Printf("%q", Location) 

Will be printed as "\nhttps://www.washingtonpost.com/news-sitemaps/politics.xml\n"

How to fix:

add this line before using Location in code:

Location = strings.TrimSpace(Location)

Answer 2

Another reason to get this error is when you use an IP address without specifying the protocol before it.

Example for cases you'll get this error:

parsedUrl, err := url.Parse("127.0.0.1:3213")

How to fix it:

parsedUrl, err := url.Parse("http://127.0.0.1:3213")

Poor documentation, unfortunately.