Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

http.FileServer caching files and serving old versions after edit

Tags:

http

go

Having issues with the http package in the core of go. It appears that the file contents is cached although the Content-Length in the response body is correct. To demonstrate here is a simplified version of the application I am writing.

package main

import (
    "fmt"
    "net/http"
)

func main() {
    http.Handle("/", http.FileServer(http.Dir("./www/")))
    err := http.ListenAndServe(":8080", nil)
    if err != nil {
        fmt.Println(err)
    }
}

Now suppose we have a very simple html page:

<!doctype html>
<html>
<body>
    <p>Hello there</p>
</body>
</html>

I execute the go program and access http://localhost:8080 in the browser to be presented with:

Hello there

Checking the response headers I can see the following:

Status Code:200 OK
Accept-Ranges:bytes
Content-Length:68
Content-Type:text/html; charset=utf-8
Date:Fri, 20 Dec 2013 10:04:03 GMT
Last-Modified:Fri, 20 Dec 2013 10:03:32 GMT

Now I edit the html file so the <p> tag contains Hello there everyone and reload the page. I get the following:

Hello there

Again looking at the response headers I get

Status Code:200 OK
Accept-Ranges:bytes
Content-Length:77
Content-Type:text/html; charset=utf-8
Date:Fri, 20 Dec 2013 10:04:34 GMT
Last-Modified:Fri, 20 Dec 2013 10:04:14 GMT

So the Content-Length has changed as well as last modified but not the actual file content delivered by the http.FileServer handler. This issue happens even after closing the program down and doing go run src/.../main.go. The only way I have found so far to clear the apparently cached version of the file is to reboot the machine the program is running on.

I have tried the following:

  • Executing program on win / ubuntu / osx 10.8.5
  • Going through the chain of functions / interfaces on golang.org/src to see if the served file is cached on disk anywhere

Any help with this would be very much appreciated.

like image 743
onmylemon Avatar asked Dec 20 '13 11:12

onmylemon


3 Answers

Ok, so after a few weeks of ignoring the problem and moving on I have finally worked out what the issue is.

In order to leave my main computer fairly uncustomised I use Vagrant to develop applications using golang, nodejs and php. It appears that running a go application on a Virtual Box share with all the html files stored on that share causes this issue.

To prove this I span up a Vagrant box and copied the files from the /vagrant shared directory to /home/vagrant/testing/, then replicated all the previously listed actions. This made the problem disappear.

So in other words, don't use a Virtual Box shared folder to host files intended for use in a http.FileServer method.

like image 94
onmylemon Avatar answered Nov 07 '22 08:11

onmylemon


Until VirtualBox has fixed the issue I made a go file that can be dropped into a project to disable sendfile support for the current process, go http package will then fallback to io.Copy. Also works with boot2docker with some small docker config changes.

https://github.com/wader/disable_sendfile_vbox_linux

With newer versions of firejail you can do the same thing using:

firejail --seccomp.enotsup=sendfile ./program
like image 2
Mattias Wadman Avatar answered Nov 07 '22 08:11

Mattias Wadman


If you use some kind of an proxy, that would be the problem. Some proxies cache frequently used files (usually only .js, .css etc., but usually not .html) and ip addresses. If the server is on your local computer, try to use localhost or 127.0.0.1 instead of an ip address, so the request doesn't go trough the proxy. If it's not you have to configure or disable the proxy to access the newest version of the website. I don't know how common this is but, it would be the problem.

like image 1
dogamak Avatar answered Nov 07 '22 07:11

dogamak