Deployed a Next.js application to App Engine Standard [Nodejs] and got a 500 error

Question

I deployed to app engine with nodejs8 runtime and got a 500. Im deploying a next.js application, and upon reviewing StackDriver I get. It appears .next might be getting ignored. The error is as follows:

throw new Error("Could not find a valid build in the '".concat(this.distDir, "' directory! Try building your app with 'next build' before starting the server."));

Error: Could not find a valid build in the '/srv/build' directory! Try building your app with 'next build' before starting the server. at Server.readBuildId (/srv/node_modules/next/dist/server/next-server.js:753:15) at new Server (/srv/node_modules/next/dist/server/next-server.js:80:25) at module.exports (/srv/node_modules/next/dist/server/next.js:6:10) at Object.<anonymous> (/srv/server.js:10:13) at Module._compile (module.js:653:30) at Object.Module._extensions..js (module.js:664:10) at Module.load (module.js:566:32) at tryModuleLoad (module.js:506:12) at Function.Module._load (module.js:498:3) at Function.Module.runMain (module.js:694:10)

My package.json file looks like:

   {
  "name": "emails",
  "private": true,
  "version": "1.0.0",
  "main": "server.js",
  "scripts": {
    "dev": "NODE_ENV=development node server.js",
    "build": "next build",
    "lint": "standard",
    "prestart": "next build", 
    "start": "NODE_ENV=production node server.js",
    "appspot-deploy": "gcloud app deploy --project=email-app-219521",
    "deploy": "gcloud app deploy"
  },
  "standard": {
    "parser": "babel-eslint"
  },
  "license": "ISC",
  "dependencies": {
    "@firebase/app-types": "^0.3.2",
    "@material-ui/core": "^3.2.0",
    "@material-ui/icons": "^3.0.1",
    "@zeit/next-sass": "^1.0.1",
    "body-parser": "^1.18.3",
    "express": "^4.16.3",
    "express-rate-limit": "^3.2.1",
    "express-session": "^1.15.6",
    "firebase-admin": "^6.0.0",
    "isomorphic-unfetch": "^3.0.0",
    "memorystore": "^1.6.0",
    "next": "^7.0.1",
    "node-sass": "^4.9.3",
    "react": "^16.5.2",
    "react-dom": "^16.5.2",
    "styled-jsx-plugin-sass": "^0.3.0"
  },
  "devDependencies": {
    "babel-eslint": "^10.0.1",
    "eslint": "^5.6.1",
    "webpack": "^4.20.2"
  },
  "engines": {
    "node": "8.x.x"
  }
}

And my app.yaml file looks like:

runtime: nodejs8
env_variables:
  NODE_ENV: production
handlers:
- url: /.*
  script: server.js

And Im serving up my project with express on port 8080.

Answer 1

When your application is returning 500 errors, make sure to look at the stdout and stderr logs of your application in Stackdriver Logging at https://console.cloud.google.com/logs/viewer. Double check that you are looking at "GAE Application" resource picker.

Looking at the error message, it seems that the .next folder does not exist in your app. This .next folder is a folder that is usually generated via a "build step", and I see indeed that you have "build": "next build" as a script in your package.json.

You should not be using prestart to perform this build step, first because App Engine does not run prestart on instance startup, but also because in general this would be bad for performances.

You have two ways to create this folder:

1. Generate .next on your machine before deploying, to do so, you can change your deploy script to be: "deploy": "npm run build && gcloud app deploy". (And also make sure that the .gcloudignore file does not contain .next or does not include the content of the .gitignore file)

2. Run this build step on Google Cloud's servers after deploying: The Node.js App Engine runtime will execute any gcp-build script if present. This means that you can add this script: "gcp-build": "npm run build" to your package.json. When doing so, I recommend you to add .next to the .gcloudignore file so that the .next folder does not get uploaded at deployment time.

In addition, note that you can simplify your app.yaml to just runtime: nodejs8:

• NODE_ENV is automatically set to production, you can remove it
• Your handlers section is equivalent to the default one.

Answer 2

I've just managed to make it work. Here's what I found:

First of all, it doesn't work with Yarn. It seems gcp-build command does not run when there is yarn.lock file.

Now in package.json, use following scripts:

  "scripts": {
    "gcp-build": "next build",
    "start": "next start -p $PORT"
  }

And make sure to declare Next and all modules needed in next.config.js as dependencies (not devDependencies)

FYI I only have runtime: nodejs10 in my app.yaml and only scripts I have are in pages folder and next.config.js