How to force app engine upload node_modules

Question

In my project we are using nodejs with typescript for google cloud app engine app development. We have our own build mechanism to compile ts files into javascript ,then collect them into a complete runable package, so that we don't want to relay on google cloud to install dependencies, instead we want to upload all node packages inside the node_modules to google cloud.

But it seems google cloud will always ignore the node_modules folder and run npm install during the deployment. Even I tried to remove 'skip_files: - ^node_modules$' from app.yaml, it doesn't work, google cloud will always install packages by itself.

Does anyone have ideas of this of deploy node app with node_modules together? Thank you.

Answer 1

I observed the same issue.

My workaround was to rename node_modules/ to node_modules_hack/ before deploying. This prevents AppEngine from removing it.

I restore it to the original name on installation, with the following (partial) package.json file:

  "__comments": [
    "TODO: Remove node_modules_hack once AppEngine stops stripping node_modules/"
  ],
  "scripts": {
    "install": "mv -fn node_modules_hack node_modules",
    "start": "node server.js"
  },

You can confirm that AppEngine strips your node_modules/ by looking at the Docker image it generates. You can find it on the Images page. They give you a commandline that you can run on the cloud console to fetch it. Then you can run docker run <image_name> ls to see your directory structure. The image is created after npm install, so once you use the workaround above, you'll see your node_modules/ there.

Answer 2

The newest solution is to allow node_modules in .gcloudignore.

Below's the default .gcloudignore (one that initial execution of gcloud app deploy generates if you don't have one already) with the change you need:

# This file specifies files that are *not* uploaded to Google Cloud Platform
# using gcloud. It follows the same syntax as .gitignore, with the addition of
# "#!include" directives (which insert the entries of the given .gitignore-style
# file at that point).
#
# For more information, run:
#   $ gcloud topic gcloudignore
#
.gcloudignore
# If you would like to upload your .git directory, .gitignore file or files
# from your .gitignore file, remove the corresponding line
# below:
.git
.gitignore

# Node.js dependencies:
# node_modules/ # COMMENT OR REMOVE THIS LINE