AWS CodeBuild tsc error TS2307: Cannot find module

Question

I have created a couple different TypeScript cdk pipelines, and they all encounter the same tsc error during the CodeBuild phase.

Two of these pipelines were replicated via the cdk docs:

• https://docs.aws.amazon.com/cdk/latest/guide/codepipeline_example.html
• https://cdkworkshop.com/20-typescript/70-advanced-topics/200-pipelines.html

Essentially, npm i (or npm ci) + tsc works fine locally, but when done over CodeBuild, it appears my dependencies don't have their dependencies installed, causing tsc to break.

Has anyone else encountered this issue? Or in other words, why does npm install + tsc behave differently on CodeBuild?

CodeBuild Log 1:

[Container] 2021/03/11 19:11:35 Waiting for DOWNLOAD_SOURCE
[Container] 2021/03/11 19:11:36 Phase is DOWNLOAD_SOURCE
[Container] 2021/03/11 19:11:36 CODEBUILD_SRC_DIR=/codebuild/output/src038984068/src
[Container] 2021/03/11 19:11:36 YAML location is /codebuild/readonly/buildspec.yml
[Container] 2021/03/11 19:11:36 Processing environment variables
[Container] 2021/03/11 19:11:36 No runtime version selected in buildspec.
[Container] 2021/03/11 19:11:36 Moving to directory /codebuild/output/src038984068/src
[Container] 2021/03/11 19:11:36 Registering with agent
[Container] 2021/03/11 19:11:36 Phases found in YAML: 2
[Container] 2021/03/11 19:11:36  PRE_BUILD: 1 commands
[Container] 2021/03/11 19:11:36  BUILD: 2 commands
[Container] 2021/03/11 19:11:36 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
[Container] 2021/03/11 19:11:36 Phase context status code:  Message: 
[Container] 2021/03/11 19:11:36 Entering phase INSTALL
[Container] 2021/03/11 19:11:36 Phase complete: INSTALL State: SUCCEEDED
[Container] 2021/03/11 19:11:36 Phase context status code:  Message: 
[Container] 2021/03/11 19:11:36 Entering phase PRE_BUILD
[Container] 2021/03/11 19:11:36 Running command npm ci
added 772 packages in 7.728s

[Container] 2021/03/11 19:11:48 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2021/03/11 19:11:48 Phase context status code:  Message: 
[Container] 2021/03/11 19:11:48 Entering phase BUILD
[Container] 2021/03/11 19:11:48 Running command npm run build

> [email protected] build /codebuild/output/src038984068/src
> tsc

node_modules/@aws-cdk/aws-codebuild/lib/pipeline-project.d.ts(1,27): error TS2307: Cannot find module 'constructs' or its corresponding type declarations.
(...cascade of missing module errors)
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tsc`
npm ERR! Exit status 2

CodeBuild Log 2:

[Container] 2021/03/11 19:54:10 Waiting for agent ping
[Container] 2021/03/11 19:54:15 Waiting for DOWNLOAD_SOURCE
[Container] 2021/03/11 19:54:16 Phase is DOWNLOAD_SOURCE
[Container] 2021/03/11 19:54:16 CODEBUILD_SRC_DIR=/codebuild/output/src363431369/src
[Container] 2021/03/11 19:54:16 YAML location is /codebuild/readonly/buildspec.yml
[Container] 2021/03/11 19:54:16 Processing environment variables
[Container] 2021/03/11 19:54:16 No runtime version selected in buildspec.
[Container] 2021/03/11 19:54:16 Moving to directory /codebuild/output/src363431369/src
[Container] 2021/03/11 19:54:16 Registering with agent
[Container] 2021/03/11 19:54:16 Phases found in YAML: 2
[Container] 2021/03/11 19:54:16  INSTALL: 1 commands
[Container] 2021/03/11 19:54:16  BUILD: 2 commands
[Container] 2021/03/11 19:54:16 Phase complete: DOWNLOAD_SOURCE State: SUCCEEDED
[Container] 2021/03/11 19:54:16 Phase context status code:  Message: 
[Container] 2021/03/11 19:54:16 Entering phase INSTALL
[Container] 2021/03/11 19:54:16 Running command npm install
npm WARN read-shrinkwrap This version of npm is compatible with lockfileVersion@1, but package-lock.json was generated for lockfileVersion@2. Ill try to do my best with it!
npm WARN [email protected] No repository field.
npm WARN [email protected] No license field.
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})

added 762 packages from 491 contributors and audited 764 packages in 12.459s

28 packages are looking for funding
  run `npm fund` for details

found 0 vulnerabilities


[Container] 2021/03/11 19:54:30 Phase complete: INSTALL State: SUCCEEDED
[Container] 2021/03/11 19:54:30 Phase context status code:  Message: 
[Container] 2021/03/11 19:54:30 Entering phase PRE_BUILD
[Container] 2021/03/11 19:54:30 Phase complete: PRE_BUILD State: SUCCEEDED
[Container] 2021/03/11 19:54:30 Phase context status code:  Message: 
[Container] 2021/03/11 19:54:30 Entering phase BUILD
[Container] 2021/03/11 19:54:30 Running command npm run build

> [email protected] build /codebuild/output/src363431369/src
> tsc

node_modules/@aws-cdk/aws-codebuild/lib/artifacts.d.ts(1,21): error TS2307: Cannot find module '@aws-cdk/aws-s3' or its corresponding type declarations.
(...more missing module errors)
npm ERR! code ELIFECYCLE
npm ERR! errno 2
npm ERR! [email protected] build: `tsc`
npm ERR! Exit status 2

Other details:

• I ensured all my @aws-cdk/* deps were installed on the exact same version
• Reproduced issue on cdk 1.60.0, 1.92.0, 1.93.0
• PipelineStack class from CodeBuild Log 1:

// lib/pipeline-stack.ts
import * as cdk from '@aws-cdk/core';
import * as codecommit from '@aws-cdk/aws-codecommit';
import * as codepipeline from '@aws-cdk/aws-codepipeline';
import * as codepipeline_actions from '@aws-cdk/aws-codepipeline-actions';
import { SimpleSynthAction, CdkPipeline } from '@aws-cdk/pipelines';

export class PipelineStack extends cdk.Stack {
  constructor(scope: cdk.Construct, id: string, props?: cdk.StackProps) {
    super(scope, id);

    const repo = new codecommit.Repository(this, 'UploadRepo', {
      repositoryName: 'cdk-s3-sns-lambda'
    });

    const sourceArtifact = new codepipeline.Artifact();
    const cloudAssemblyArtifact = new codepipeline.Artifact();

    new CdkPipeline(this, 'CodePipeline', {
      crossAccountKeys: false,
      cloudAssemblyArtifact,
      sourceAction: new codepipeline_actions.CodeCommitSourceAction({
        actionName: 'CodeCommit',
        output: sourceArtifact,
        repository: repo
      }),
      synthAction: SimpleSynthAction.standardNpmSynth({
        sourceArtifact,
        cloudAssemblyArtifact,
        buildCommand: 'npm run build'
      }),
    });
  }
}


// bin/pipeline.ts
#!/usr/bin/env node
import 'source-map-support/register';
import * as cdk from '@aws-cdk/core';
import { PipelineStack } from '../lib/pipeline-stack';

const app = new cdk.App();
new PipelineStack(app, 'AppPipelineStack');

• Generated buildspec.yaml from CodeBuild Log 1:

{
  "version": "0.2",
  "phases": {
    "pre_build": {
      "commands": [
        "npm ci"
      ]
    },
    "build": {
      "commands": [
        "npm run build",
        "npx cdk synth"
      ]
    }
  },
  "artifacts": {
    "base-directory": "cdk.out",
    "files": "**/*"
  }
}

Answer 1

I have been facing the same issue.

The workaround in this GH-Issue (https://github.com/aws/aws-cdk/issues/13339) helped me out.

I changed my buildspec in CDK like this

synthAction: SimpleSynthAction.standardNpmSynth({
  sourceArtifact,
  cloudAssemblyArtifact,
  installCommand: 'npm i -g npm && npm ci'
}),

this will generate the following (inline) buildspec for the Pipeline Build step:

{
  "version": "0.2",
  "phases": {
    "pre_build": {
      "commands": [
        "npm i -g npm && npm ci"
      ]
    },
    "build": {
      "commands": [
        "npx cdk synth"
      ]
    }
  },
  "artifacts": {
    "base-directory": "cdk.out",
    "files": "**/*"
  }
}

After pushing the code and running the pipeline for the first time I had to modify the buildspec by myself because the pipeline mutation step is triggert after the build step. So the ne buildspec settings will apply after the mutation step.

Answer 2

Short answer

Use the same npm version locally and in CodeBuild. I suspect the issue arises because you are using a newer npm version locally than the one CodeBuild is using.

Longer answer

CodeBuild runs the build in a certain environment (Docker image), and this environment has certain versions of runtimes/packages installed, e.g. Node.js/npm. So if you are developing with npm v7.x and your CodeBuild uses npm v6.x, then you package-lock.json file used by the npm ci command will not be compatible with CodeBuild's environment, and the build will fail.
(When using CDK, the image is chosen for you as part of the SimpleSynthAction.standardNpmSynth action.)
Luckily you can change one of them to match the other.

Make your local version of Node.js/npm match the one used in CodeBuild

1. First find out which Docker image CodeBuild uses by going to CodeBuild > Build projects > your build project > Build details (new AWS console). In the Environment section, the image used to build is listed under Image.
2. Then find the runtime version used by that image in this overview (if you're interested, check out the image's Dockerfile). For example, the newest "nodejs" runtime supported is 14, and that is only supported by the "Ubuntu standard:5.0" image. The default image used by SimpleSynthAction.standardNpmSynth is "Ubuntu standard:4.0".
3. Use that runtime locally.

Make CodeBuild use your desired runtime version(s)

Again there are multiple ways to achieve this:

1. Choose another environment (Docker image) for your build. When you're using CDK, this can be done e.g. by specifying the environment property of your synthAction:

   new CdkPipeline(this, 'CodePipeline', {
       crossAccountKeys: false,
       cloudAssemblyArtifact,
       sourceAction: new codepipeline_actions.CodeCommitSourceAction({
           actionName: 'CodeCommit',
           output: sourceArtifact,
           repository: repo
       }),
       synthAction: SimpleSynthAction.standardNpmSynth({
           sourceArtifact,
           cloudAssemblyArtifact,
           environment: {
               buildImage: LinuxBuildImage.STANDARD_5_0
           },
           buildCommand: 'npm run build'
       }),
   });
   

2. Specify runtime versions in the buildspec file, e.g.

   phases:
     install:
       runtime-versions:
         nodejs: 14.x
   

   This might be a bit more risky because the runtime version(s) you specify could be incompatible with other runtime versions (e.g. those already in the chosen Docker image).
3. Install the desired runtime version, well, runtime, e.g. by overriding installCommand with 'npm i -g npm && npm ci' as suggested by the author of the GitHub issue referenced by @therealsebo. I personally wouldn't recommend this, it just seems a bit too much of a hack, and there is no guarantee that it will (continue to) work.

Bonus info

You can run CodeBuild locally to simulate the CodeBuild environment. From the guide it is also clear that you are making a decision as to which Docker image to use.

Conclusion

The problem you are describing may just be a special issue with npm v6.x vs. v7.x, but I think it represents a broader category of issues, all of which should be fixable by knowing which versions of different packages/runtimes CodeBuild uses, essentially aligning your expectations to reality (or changing the reality).