AWS CDK how to create an API Gateway backed by Lambda from OpenApi spec?

Question

I want to use AWS CDK to define an API Gateway and a lambda that the APIG will proxy to.

The OpenAPI spec supports a x-amazon-apigateway-integration custom extension to the Swagger spec (detailed here), for which an invocation URL of the lambda is required. If the lambda is defined in the same stack as the API, I don't see how to provide this in the OpenAPI spec. The best I can think of would be to define one stack with the lambda in, then get the output from this and run sed to do a find-and-replace in the OpenAPI spec to insert the uri, then create a second stack with this modified OpenAPI spec.

Example:

  /items:
    post:
      x-amazon-apigateway-integration:
        uri: "arn:aws:apigateway:eu-west-2:lambda:path/2015-03-31/functions/arn:aws:lambda:eu-west-2:123456789012:function:MyStack-SingletonLambda4677ac3018fa48679f6-B1OYQ50UIVWJ/invocations"
        passthroughBehavior: "when_no_match"
        httpMethod: "POST"
        type: "aws_proxy"

Q1. This seems like a chicken-and-egg problem, is the above the only way to do this?

I tried to use the defaultIntegration property of the SpecRestApi CDK construct. The documentation states:

An integration to use as a default for all methods created within this API unless an integration is specified.

This seems like a should be able to define a default integration using a lambda defined in the CDK spec and therefore have all methods use this integration, without needing to know the uri of the lambda in advance.

Thus I tried this:

SingletonFunction myLambda = ...

SpecRestApi openapiRestApi = SpecRestApi.Builder.create(this, "MyApi")
                        .restApiName("MyApi")
                        .apiDefinition(ApiDefinition.fromAsset("openapi.yaml"))
                        .defaultIntegration(LambdaIntegration.Builder.create(myLambda)
                                    .proxy(false)
                                    .build())
                        .deploy(true)
                        .build();

The OpenAPI spec defined in openapi.yaml does not include a x-amazon-apigateway-integration stanza; it just has a single GET method defined within a standard OpenApi 3 specification.

However, when I try to deploy this, I get an error:

No integration defined for method (Service: AmazonApiGateway; Status Code: 400; Error Code: BadRequestException; Request ID: 56113150-1460-4ed2-93b9-a12618864582)

This seems like a bug, so I filed one here.

Q2. How do I define an API Gateway and Lambda using CDK and wire the two together via an OpenAPI spec?

Answer 1

I came up with a solution which is a bit simpler than the other answers here as it does not require stage variables or multiple deploys.

First, set the uri of the x-amazon-apigateway-integration to a variable like ${API_LAMBDA_ARN} and use the same type and httpMethod as in this example:

[...]
  "paths": {
    "/pets": {
      "get": {
        "summary": "List all pets",
        "responses": {
          [...]
        },
        "x-amazon-apigateway-integration": {
          "uri": "${API_LAMBDA_ARN}",
          "type": "AWS_PROXY",
          "httpMethod": "POST",
        }
      }
    }
  },
[...]

Then, you can use this construct (or an equivalent TypeScript implementation) to replace the variable during build time and create an API Gateway Http API based on the OpenAPI Document:

from aws_cdk import (
    core,
    aws_iam as iam,
    aws_lambda as _lambda,
    aws_apigatewayv2 as apigateway
)


class OpenApiLambdaStack(core.Stack):
    def __init__(
        self, scope: core.Construct, construct_id: str, **kwargs
    ) -> None:
        super().__init__(scope, construct_id, **kwargs)

        # function that handles api request(s)
        api_lambda = _lambda.Function([...])

        # read openapi document
        with open("openapi.json", "r") as json_file:
            content = json_file.read()
        # replace the variable by the lambda functions arn
        content = content.replace("${API_LAMBDA_ARN}", api_lambda.function_arn)
        openapi = json.loads(content)

        # create apigateway
        http_api = apigateway.HttpApi(self, "OpenApiLambdaGateway")
        # use escape hatches to import OpenAPI Document
        # see: https://docs.aws.amazon.com/cdk/latest/guide/cfn_layer.html
        http_api_cfn: apigateway.CfnApi = http_api.node.default_child
        http_api_cfn.add_property_override("Body", openapi)
        http_api_cfn.add_property_deletion_override("Name")
        http_api_cfn.add_property_deletion_override("ProtocolType")
        # let it fail on warnings to be sure everything went right
        http_api_cfn.add_property_override("FailOnWarnings", True)

        # construct arn of createad api gateway (to grant permission)
        http_api_arn = (
            f"arn:{self.partition}:execute-api:"
            f"{http_api.env.region}:{http_api.env.account}:"
            f"{http_api.http_api_id}/*/*/*"
        )

        # grant apigateway permission to invoke api lambda function
        api_lambda.add_permission(
            f"Invoke By {http_api.node.id} Permission",
            principal=iam.ServicePrincipal("apigateway.amazonaws.com"),
            action="lambda:InvokeFunction",
            source_arn=http_api_arn,
        )
        
        # output api gateway url
        core.CfnOutput(self, "HttpApiUrl", value=http_api.url)

Python users might also be interested in the openapigateway construct I've published to make this process even more straight-forward. It supports JSON and YAML.

Answer 2

There is an existing workaround. Here is how:

Your OpenAPI file has to look like this:

openapi: "3.0.1"
info:
  title: "The Super API"
  description: "API to do super things"
  version: "2019-09-09T12:56:55Z"

servers:
- url: ""
  variables:
    basePath:
      default:
        Fn::Sub: ${ApiStage}

paths:
  /path/subpath:
    get:
      parameters:
      - name: "Password"
        in: "header"
        schema:
          type: "string"
      responses:
        200:
          description: "200 response"
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/UserConfigResponseModel"
      security:
      - sigv4: []
      x-amazon-apigateway-integration:
        uri: 
          Fn::Sub: "arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${MySuperLambda.Arn}/invocations"
        responses:
          default:
            statusCode: "200"
        requestTemplates:
          application/json: "{blablabla}"
        passthroughBehavior: "when_no_match"
        httpMethod: "POST"
        type: "aws"

As you can see, this OpenAPI template refers to ApiStage, AWS::Region and MySuperLambda.Arn.

The associated cdk file contains the following:

// To pass external string, nothing better than this hacky solution: 
const ApiStage = new CfnParameter(this, 'ApiStage',{type: 'String', default: props.ApiStage})
ApiStage.overrideLogicalId('ApiStage') 

Here the ApiStage is used in props. It allows me to pass it to the cdk app with an environment variable during the CI for example.

const MySuperLambda = new lambda.Function(this, 'MySuperLambda', {
    functionName: "MySuperLambda",
    description: "Hello world",
    runtime: lambda.Runtime.PYTHON_3_7,
    code: lambda.Code.asset(lambda_asset),
    handler: "MySuperLambda.lambda_handler",
    timeout: cdk.Duration.seconds(30),
    memorySize: 128,
    role: MySuperLambdaRole
  });

  const forceLambdaId = MySuperLambda.node.defaultChild as lambda.CfnFunction
  forceLambdaId.overrideLogicalId('MySuperLambda')

Here, as previously, I'm forcing CDK to override the logical ids so that I know the id before deployment. Otherwise, cdk adds a suffix to the logical ids.

const asset = new Asset(this, 'SampleAsset', {
    path: './api-gateway-definitions/SuperAPI.yml',
  });

This allows me to upload the OpenAPI file directly on to the cdk bucket (no need to create a new one, this is amazing).

const data = Fn.transform('AWS::Include', {'Location': asset.s3ObjectUrl})

This is part of Cloudformation magic. This is where Fn::Sub and Fn::GetAtt are interpreted. I could not manage to make it work with !Ref function.

const SuperApiDefinition = apigateway.AssetApiDefinition.fromInline(data)

Create an api definition from the previously read file.

  const sftpApiGateway = new apigateway.SpecRestApi(this, 'superAPI', {
    apiDefinition: SuperApiDefinition,
    deploy: false
  })

Finally, create the SpecRestApi. Run and magic, this is working. You may still encounter 400 errors, probably because of uncorrect format in your OpenAPI file (and don't use !Ref).

Would I recommand this? Meh. This is pretty much a workaround. It is really useful if you want to use the OpenAPI format with dynamic variables, within your CI. Without much effort, you can deploy in dev and prod, just by switching 1 environment variable.

However, this feels really hacky and does not seem to fit in CDK philosophy. This is what I'm currently using for deployment but this will probably change in the future. I believe a real templating solution could have a better fit here, but right now, I don't really thought about it.