How to resolve this "error archiving directory: could not archive missing directory"

Question

At the beginning of my project, there are two terraform modules: base and reusable_module.

base/main.tf

# Provide abstraction to define a lambda function
terraform {
  required_version = "0.11.7"
}

variable "env" {}
variable "role" {}
variable "function_name" {
  default = ""
}
variable "lambda_filename" {}
variable "script_env_vars" {
  type = "map"
}

data "archive_file" "package_zip" {
  type = "zip"
  # There is a bug in Terraform which does not allow '..' in source_dir, thus we use path.root:
  # https://github.com/terraform-providers/terraform-provider-archive/issues/5
  source_dir = "${path.root}/scripts/"  # Path from top level module.
  # The output path has to be relative. Otherwise the buildkite will always show a diff.
  output_path = "./.terraform/${var.env}-${var.lambda_filename}.zip"
}

resource "aws_lambda_function" "lambda" {
  function_name = "${var.function_name}"
  description = "Simple function"
  role = "${var.role}"

  runtime = "python3.6"
  timeout = 300  // seconds. Max hard limit is 5 min.

  filename = "${data.archive_file.package_zip.output_path}"
  // The handler is always the file name + function name "handler".
  handler = "${var.lambda_filename}.handler"
  source_code_hash = "${data.archive_file.package_zip.output_base64sha256}"

  // Environment variables for the script.
  environment {
    variables = "${var.script_env_vars}"
  }
}

reusable_module/main.tf

variable "env" {}
variable "region" {}
variable "function_name" {}

provider "aws" {
  region = "${var.region}"
}

resource "aws_iam_role" "mylambda_role" {
  name = "${var.env}-mylambda-role"

  assume_role_policy = <<EOF
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Action": "sts:AssumeRole",
      "Principal": {
        "Service": "lambda.amazonaws.com"
      },
      "Effect": "Allow",
      "Sid": ""
    }
  ]
}
EOF
}

locals {
    default_function_name = "${var.env}-mylambda"
    final_function_name = "${var.function_name != "" ? var.function_name : local.default_function_name}"
}

module "mylambda" {
    source = "../base"
    lambda_filename = "mylambda"
    function_name = "${local.final_function_name}"
    env = "${var.env}"
    role = "${aws_iam_role.mylambda_role.arn}"
    script_env_vars = {
      DUMMY = "123"
    }
}

Module mylambda uses base/main.tf to create a lambda function.

Under reusable_module, there is a scripts directory where all the python scripts live.

Now I want to extend my project by reusing and instantiating this reusable_module/main.tf for different teams.

team_template/main.tf

variable "env" {}
variable "region" {}
variable "team" {}


module "team_template" {
  source = "../reusable_module"
  env = "${var.env}"
  region = "${var.region}"
  function_name = "${var.team}-essential-function"
}

# More resources specific to team_template

team-sales/main.tf

For creating a lambda per team

variable "env" {}
variable "region" {}

module "realdeal" {
  source = "../team_template"
  env = "${var.env}"
  region = "${var.region}"
  team = "sales"
}
# More stuff tailored for each teams

When I run terraform plan -var-file=dev.tfvars in team-sales, I got this error:

data.archive_file.package_zip: Refreshing state...

Error: Error refreshing state: 1 error(s) occurred:

• module.realdeal.module.team_template.module.mylambda.data.archive_file.package_zip: 1 error(s) occurred:

• module.realdeal.module.team_template.module.mylambda.data.archive_file.package_zip: data.archive_file.package_zip: error archiving directory: could not archive missing directory: /Users/antkong/Documents/Personal/wd/StackoverflowCode/terraform/lambda/team/scripts/

The problem is the data.archive_file.package_zip is looking for scripts in team_template/scripts.

But in this case I actually have no python code in team_template. I simply want to continue to use the python code in reusable_module/script.

The separation of files must be maintained. (Conway's law etc etc)

How can I do it?

Answer 1

Perhaps not applicable to your issue, but was applicable to mine when I encountered the same error.

If you mistakenly use source_dir instead of source_file for your archive_file you get this same error.

E.g. change this:

data "archive_file" "source" {
  type         = "zip"
  source_dir   = "${path.module}/lambda_function.py"
  output_path  = "${path.module}/function.zip"
}

To this:

data "archive_file" "source" {
  type         = "zip"
  source_file  = "${path.module}/lambda_function.py"
  output_path  = "${path.module}/function.zip"
}