API Gateway resource policy is duplicated many times

[simonmarklar]

Description:

Our API template has been working for months but it suddenly stopped working on friday morning AEST, with an unhelpful message

Transform AWS::Serverless-2016-10-31 failed with: Internal transform failure.

After debugging I've found that the APIG Resource Policy is the culprit - removing the resource policy and the template transforms and deploys... We have 2 statements - 1 to allow all, one to deny all but 2 IPs on a certain path. The transformed template has multiple entries for both statements, and every time we add a lambda it grows. Eventually we get the "aws SAM returned fragment exceeding 460800 bytes" error

We use CfnDsl so the snippts below are modified off the Template tab in the cloudformation console. I've changed names but kept the setup in case something special is causing this

Steps to reproduce the issue:

1. Create a Serverless::Api resource with resource policies eg

restapi:
    Properties:
      Name:
        Fn::Join:
        - '-'
        - - Ref: prefix
          - Ref: deployenv
          - restapi
      StageName:
        Ref: deployenv
      MethodSettings:
      - ResourcePath: /*
        HttpMethod: '*'
        LoggingLevel: ERROR
      Cors:
        AllowHeaders: '''Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'''
        AllowOrigin: '''*'''
        AllowCredentials: false
      Auth:
        Authorizers:
          CognitoAuthorizer:
            UserPoolArn:
              Ref: cognitoUserPoolArn
        ResourcePolicy:
          CustomStatements:
          - Effect: Allow
            Principal: '*'
            Action: execute-api:Invoke
            Resource:
              Fn::Join:
              - ''
              - - 'arn:aws:execute-api:'
                - Ref: AWS::Region
                - ':'
                - Ref: AWS::AccountId
                - :*/*/*
          - Effect: Deny
            Principal: '*'
            Action: execute-api:Invoke
            Resource:
              Fn::Join:
              - ''
              - - 'arn:aws:execute-api:'
                - Ref: AWS::Region
                - ':'
                - Ref: AWS::AccountId
                - :*/*/*/jira/*
            Condition:
              NotIpAddress:
                aws:SourceIp:
                - Ref: jiraIPAZA
                - Ref: jiraIPAZB
      TracingEnabled: true
      GatewayResponses:
        DEFAULT_INTERNAL:
          StatusCode: 200
          ResponseTemplates:
            application/json: '{  }'
          ResponseParameters:
            Headers:
              Access-Control-Allow-Headers: '''Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'''
              Access-Control-Allow-Methods: '''DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'''
              Access-Control-Allow-Origin: '''*'''
    Type: AWS::Serverless::Api

2. add some (more than 2) lambdas with api events eg:

  testLambda:
    DependsOn: lambdaexecutionrole
    Properties:
      AutoPublishAlias: live
      Runtime: nodejs8.10
      Handler: lambda-handlers/hello/index.handler
      Description: test lambda
      CodeUri: s3://path-to-code
      Role:
        Fn::GetAtt:
        - lambdaexecutionrole
        - Arn
      Timeout: 120
      VpcConfig:
        SecurityGroupIds:
        - Ref: lambdaSG
        SubnetIds:
        - Ref: privateSubnetAZA
        - Ref: privateSubnetAZB
      Environment:
        Variables:
          COGNITO_USER_POOL_ID:
            Ref: cognitoUserPoolId
          PREFIX:
            Ref: prefix
          DEPLOYENV:
            Ref: deployenv
          BASE_URL:
            Ref: websiteBaseUrl
      Events:
        helloworld:
          Properties:
            Method: GET
            Path: /hello
            RestApiId:
              Ref: restapi
          Type: Api
        goodbyeworld:
          Properties:
            Method: GET
            Path: /goodbye
            RestApiId:
              Ref: restapi
          Type: Api
    Type: AWS::Serverless::Function
  helloApiPermission:
    DependsOn: testLambda
    Properties:
      Action: lambda:InvokeFunction
      FunctionName:
        Ref: testLambda
      Principal: apigateway.amazonaws.com
      SourceAccount:
        Ref: AWS::AccountId
    Type: AWS::Lambda::Permission

deploy to cloudformation

Observed result:

the transformed restapi resource has way too many entries in the apig policy:

"restapi": {
      "Type": "AWS::ApiGateway::RestApi",
      "Properties": {
        "Body": {
          "info": {
            "version": "1.0",
            "title": {
              "Ref": "AWS::StackName"
            }
          },
          "paths": {...snip},
          "x-amazon-apigateway-gateway-responses": {
            "DEFAULT_INTERNAL": {
              "responseParameters": {
                "gatewayresponse.header.Access-Control-Allow-Headers": "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'",
                "gatewayresponse.header.Access-Control-Allow-Methods": "'DELETE,GET,HEAD,OPTIONS,PATCH,POST,PUT'",
                "gatewayresponse.header.Access-Control-Allow-Origin": "'*'"
              },
              "responseTemplates": {
                "application/json": "{  }"
              },
              "statusCode": "200"
            }
          },
          "securityDefinitions": {
            "CognitoAuthorizer": {
              "in": "header",
              "type": "apiKey",
              "name": "Authorization",
              "x-amazon-apigateway-authorizer": {
                "providerARNs": [
                  {
                    "Ref": "cognitoUserPoolArn"
                  }
                ],
                "type": "cognito_user_pools"
              },
              "x-amazon-apigateway-authtype": "cognito_user_pools"
            }
          },
          "x-amazon-apigateway-policy": {
            "Version": "2012-10-17",
            "Statement": [
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*"
                    ]
                  ]
                },
                "Effect": "Allow",
                "Principal": "*"
              },
              {
                "Action": "execute-api:Invoke",
                "Resource": {
                  "Fn::Join": [
                    "",
                    [
                      "arn:aws:execute-api:",
                      {
                        "Ref": "AWS::Region"
                      },
                      ":",
                      {
                        "Ref": "AWS::AccountId"
                      },
                      ":*/*/*/jira/*"
                    ]
                  ]
                },
                "Effect": "Deny",
                "Condition": {
                  "NotIpAddress": {
                    "aws:SourceIp": [
                      {
                        "Ref": "jiraIPAZA"
                      },
                      {
                        "Ref": "jiraIPAZB"
                      }
                    ]
                  }
                },
                "Principal": "*"
              }
            ]
          },
          "swagger": "2.0"
        },
        "Name": {
          "Fn::Join": [
            "-",
            [
              {
                "Ref": "prefix"
              },
              {
                "Ref": "deployenv"
              },
              "restapi"
            ]
          ]
        }
      }
    },

Expected result:

only 1 entry per statement

[simonmarklar]

hmmm, am I going crazy or did the docs for the resource policy syntax change ? What I have written above worked since it was implemented a few weeks ago...

If it has changed, is the old format still supported?

[praneetap]

@simonmarklar This was a regression :( we are working on the fix right now #1183. Sorry about the pain caused! You can track the patch release here

[simonmarklar]

thanks for the response! i'll close this issue :)