How can I use a sub-packages with Go on Google Cloud Functions?

Question

I'd like to use a helper package from Go Cloud Function. The package has some helper logic that can be shared between multiple functions. But, what is the right way to structure the packages so they all work? The package should be in the same project - not published and public as a completely separate package.

I work at Google. The goal for this question is to proactively answer common questions and help developers starting off with Go on GCF.

Answer 1

You can use subpackages with Go modules. Go modules are Go's new dependency management solution - they let you work outside of GOPATH and let you manage the exact versions of each dependency you have.

Modules also let you define a group of Go packages with the same import path prefix. When you're writing a function, this lets you import other packages in your module.

The function you're deploying needs to be at the root of your module.

Here is an example file structure and how packages would be imported:

.
├── cmd
│   └── main.go # Useful for testing. Can import and setup your function.
├── function.go # Can import example.com/foo/helperpackage
├── function_test.go
├── go.mod # module example.com/foo
└── helperpackage
    └── helper.go

This setup has your function in function.go and tested by function_test.go. They are in a module named example.com/foo. helperpackage can be imported by function.go using example.com/foo/helperpackage.

This also has a cmd directory, which may be helpful for local testing. You can import example.com/foo and start an HTTP server which registers your function as an HTTP handler. For example:

package main

import (
    "log"
    "net/http"

    "example.com/foo"
)

func main() {
    http.Handle("/HelloHTTP", foo.HelloHTTP)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Note: You could use a vendor directory to achieve the same result. But, all of the packages your function imports would need to be in the vendor directory (with the full import path), which works, but is cumbersome to maintain. It's uncommon to copy sub-packages into your vendor directory, so I wouldn't recommend this.

Answer 2

We also ran into this issue. The go.mod approach provided by @TylerBui-Palsulich (https://stackoverflow.com/a/54255486/2307346) didn't work as we also had to download dependencies from a private repository.

From the Google documentation:

If your function's dependencies are hosted in a repository that is not publicly accessible, you must use a vendor directory to fetch your dependencies before deploying your function (https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)

Given the following package structure:

cloudfunctions.go
sharedpackage
-> shared.go
go.mod

In our go.mod file we defined the following namespace: module some/namespace The cloudfunctions.go file has the following package definition and imports

package foobar

import (
    "bitbucket.org/some/private/dependency"
    "some/namespace/sharedpackage"
)

Because of the private dependency we cannot use the go.mod file. Instead we provide a vendor directory.

Warning: If you have both a go.mod file and a vendor directory at the root of your project, the contents of the vendor directory will be ignored when your function is built in the cloud. To ensure that your vendor directory is used, you must exclude the go.mod file from your project's source code prior to deployment. If you are using the gcloud command-line tool, you can ensure that go.mod is not uploaded by using .gcloudignore. (https://cloud.google.com/functions/docs/writing/specifying-dependencies-go#using_a_vendor_directory)

gcloud deploy will ignore the vendor directory if it finds go.mod, we fix this with a .gloudignore file with the following content:

go.mod
go.sum

At the end we have the following file structure:

cloudfunctions.go
sharedpackage
-> shared.go
go.mod
vendor
.gcloudignore

The last step, Fixing the cannot find package error

When running the deploy step you will still run into an error similar to:

 (gcloud.functions.deploy) OperationError: code=3, message=Build failed: /tmp/sgb/gopath/src/serverlessapp/vendor/some/namespace/cloud_functions.go:5:2: cannot find package "some/namespace/sharedpackage" in any of:...

This is because the go.mod file, containing the module name, is ignored. now the Go compiler no longer knows that some/namespace/sharedpackage refers to the local sharedpackage directory.

We managed to get it working by changing the module name to match the package name:

Change module name in go.mod to module foobar Change the imports in cloudfunctions.go to: "foobar/sharedpackage":

package foobar

import (
    "bitbucket.org/some/private/dependency"
    "foobar/sharedpackage"
)

Now the Go compiler is able to detect that foobar/sharedpackage is a subpackage of the foobar package.