Clean and generic project structure for GO applications and mongodb [closed]

Question

I want to build an API based application using GO and MongoDB. I'm from Asp.net MVC background. Probably if I make an architecture with MVC web application things to be consider are

1. Separation of concerns(SoC)

   • DataModel
   • BusinessEntities
   • BusinessServices
   • Controllers

2. Dependeny Injection and Unity of Work

3. Unit Testing
   • MoQ or nUnit
4. Integration with UI frameworks
   • Angularjs or others
5. RESTful urls that enables SEO

Below architecture could be a solution for my need in MVC based appications

There are resources around the web to build Asp.Net or Java based applications, but I have not find solution to Golang application architecture.

Yes GO is different to C# or Java, but still there are Structs, Interfaces to create reusable code and a generic application architecture. Consider above points in mind, how we can make a clean and reusable project structure in GO applications and a generic repositories for DB(Mongodb) transactions. Any web resources also a great point to start.

Answer 1

It depends on your own style and rules, in my company, we develop our projects this way:

• Configuration is determined by environment variables, so we have a company/envs/project.sh file which has to be evaluated before service (outside the project in the image).
• We add a zscripts folder that contains all extra scripts, like adding users or publishing a post. Intended to be used only for debug proposes.
• Data models (entities) are in a package called project/models.
• All controllers and views (HTML templates) are categorized into "apps" or "modules". We use the REST path as main group delimiter, so path /dogs goes to package project/apps/dogs and /cats to project/apps/cats.
• Managers are in their separated package at project's root project/manager.
• Static files (.css, .png, .js, etc.) are located at project/static/[app/]. Sometimes is required to have the optional [app/] folder, but it only happens when two apps have dashboards or conflicting file names. Most of cases you won't need to use [app/] for static resources.

Managers

We call a manager, a package that contains pure functions which helps apps to perform its task, for example, databases, cache, S3 storage, etc. We initialize each manager calling package.Startup() before we start to listen, and finalize calling package.Finalize() when program is interrupted.

An example of a manager could be project/cache/cache.go:

type Config struct {
    RedisURL string `envconfig:"redis_url"`
}

var config Config
var client *redis.Client

func Startup(c Config) error {
   config = c
   client, err := redis.Dial(c.RedisURL)
   return err
}

func Set(k,v string) error {
   return client.Set(k, v)
}

in main.go (or your_thing_test.go):

var spec cache.Config
envconfig.Process("project", &spec)

cache.Startup(spec)

And in a app (or module):

func SetCacheHandler(_ http.ResponseWriter, _ *http.Request){
   cache.Set("this", "rocks")
}

Modules

A module is a container of views and controllers that are isolated from other modules, using our configuration I would recommend to not create dependencies between modules. Modules are also called apps.

Each module configures its routes using a router, sub-router or what your framework provides, for example (file project/apps/dogs/configure.go):

func Configure(e *echo.Echo) {
    e.Get("/dogs", List)
}

Then, all handlers live in project/apps/dogs/handlers.go:

// List outputs a dog list of all stored specimen.
func List(c *echo.Context) error {
    // Note the use of models.Xyz
    var res := make([]models.Dog, 0) // A little trick to not return nil.
    err := store.FindAll("dogs", nil, &res) // Call manager to find all dogs.
    // handle error ...

    return c.JSON(200, res) // Output the dogs.
}

Finally you configure the app in main (or in a test):

e := echo.New()
dogs.Configure(e)
// more apps

e.Run(":8080")

Note: for views, you can add them to project/apps/<name>/views folder and configure them the using the same function.

Other

Sometimes we also add a project/constants and a project/utils package.

Here is what it looks like:

Note that in above sample, templates are separated from apps, thats because its a placeholder, directory is empty.

Hope it was useful. Greetings from México :D.

Answer 2

I've also struggled about how to structure my Go web APIs in the past and don't know any web resources that tell you exactly how to write a Go web API.

What I did was just check out other projects on Github and try out how they structured their code, for example, the Docker repo has very idomatic Go code on it's API.

Also, Beego is a RESTful framework that generates the project structure for you in a MVC way and according to their docs, it can also be used for APIs.

Answer 3

I've been building a web APIs in golang for a little while now.

You'll have to do some research but I can give you some starting points:

1. Building Web Apps with Go -- ebook
2. github.com/julienschmidt/httprouter -- for routing addresses
3. github.com/unrolled/render/ -- for rendering various forms of responses(JSON, HTML, etc..)
4. github.com/dgrijalva/jwt-go -- JSON Web Tokens
5. www.gorillatoolkit.org/pkg/sessions -- Session Management

And for reference on how some things work together in the end:

Go Web API Repo -- personal project

Answer 4

1. Separation of concerns (SoC)

I haven't worked with SoC directly, but I have my own pattern. You can adapt to whatever pattern (MVC, your own, etc.).

In my code, I separate my code into different packages:

myprojectname (package main)      — Holds the very basic setup and configuration/project consts
  * handlers   (package handlers) — Holds the code that does the raw HTTP work
  * models     (package models)   — Holds the models
  * apis       (NOT a package)
    - redis    (package redis)    — Holds the code that wraps a `sync.Pool`
    - twilio   (package twilio)   — Example of layer to deal with external API

Notes:

1. In each package other than main, I have a Setup() function (with relevant arguments) that is called by the main package.
2. For the packages under the apis folder, they are often just to initialize external Go libraries. You also can directly import existing libraries into the handlers/models without an apis package.

3. I setup my mux as an exported global in the handlers package like this...

   Router := mux.NewRouter()
   

   ...and then create a file for each URL (different methods with the same URL are in the same file). In each file, I use Go's init() function, which is ran after global variables are initialized (so it's safe to use the router) but before main() is ran (so it's safe for main to assume everything has been setup). The great thing about init() is that you can have as many of those methods as you want in a single package, so they automatically get ran when the package is imported.

   Main then imports myprojectname/handlers and then serves the handlers.Router in main.

2. Dependency Injection and Unity of Work

I haven't worked with Unity of Work, so I have no idea of possible Go implementations.

For DI, I build an interface that both the real object and the mock objects will implement.

In the package, I add this to the root:

var DatabaseController DatabaseControllerInterface = DefaultController

Then, the first line of each test I can change DatabaseController to whatever that test needs. When not testing, the unit tests should not be ran, and it defaults to DefaultController.

3. Unit Testing

Go provides a built in testing with the go test package command. You can use go test --cover to also emit a coverage percent. You can even have coverage displayed in your browser, highlighting the parts that are/aren't covered.

I use the testify/assert package to help me with testing where the standard library falls short:

// something_test.go
//
// The _test signifies it should only be compiled into a test
// Name the file whatever you want, but if it's testing code
// in a single file, I like to do filename_test.go.

package main

import (
    "testing"

    "github.com/stretchr/testify/assert"
)

func TestMath(t *testing.T) {
    assert.Equal(t, 3+1, 4)
}

4. Integration with UI frameworks

I haven't seen any for Angular. Although I haven't used it, Go has a good template engine built into the standard lib.

5. RESTful URLs that enables SEO

Again, I can't help you here. It's up to you: send proper status codes (don't send a 200 with a 404 page as you'll get docked for duplicate pages), don't duplicate pages (pay attention to google.com/something vs google.com/something/; hopefully your framework will not mess this up), don't try to trick the search engine, and so on.