AWS Control Tower Landing Zone Setup: Troubleshooting Account Limits and KMS Policies

#aws #devops #tutorial #cloud

While setting up a multi-account environment and trying to enable AWS Control Tower, I encountered two mysterious errors. Here's how I resolved them.

"Account limit reached" error (even though I hadn't reached the limit)
CloudFormation deployment failure (KMS key policy related)

Account Limit Error

When enabling Control Tower, it creates a Log Archive account and an Audit account. However, only the Audit account was created, and I got an error saying "Account limit reached" for the Log Archive account.

AWS Control Tower failed to set up your landing zone completely: AWS Control Tower cannot create an account because you have reached the limit on the number of accounts in your organization.

When I checked Service Quotas, the Default maximum number of accounts was set to 10, but my actual account count was clearly below that (only 3 including the management account).

I couldn't figure out the root cause (which is frustrating), but I resolved it by increasing the Service Quota to 15. Perhaps Control Tower internally reserves some account slots...?

CloudFormation Deployment Failure (KMS Key Policy Related)

Just when I thought I'd resolved the account limit error, I ran into a CloudFormation deployment error.

AWS Control Tower failed to set up your landing zone completely: AWS Control Tower failed to deploy stack(s): arn:aws:cloudformation:ap-northeast-1:XXXXXXXX:stack/AWSControlTowerBP-BASELINE-CLOUDTRAIL-MASTER/4787d990-8144-11f0-ad99-0625194a54fd. To continue, review the failed stack(s) and try again.

When I opened the CloudFormation console, I saw this:

I had configured CloudTrail and Config to use KMS for encryption, but apparently these services lacked the necessary permissions to access the KMS key.

Required Permissions:

Config: kms:Decrypt, kms:GenerateDataKey
CloudTrail: kms:GenerateDataKey*, kms:Decrypt

So I added the following to my KMS policy:

{
  "Sid": "AllowConfigToUseKms",
  "Effect": "Allow",
  "Principal": { "Service": "config.amazonaws.com" },
  "Action": ["kms:Decrypt","kms:GenerateDataKey"],
  "Resource": "arn:aws:kms:ap-northeast-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx"
},
{
  "Sid": "AllowCloudTrailToUseKms",
  "Effect": "Allow",
  "Principal": { "Service": "cloudtrail.amazonaws.com" },
  "Action": ["kms:GenerateDataKey*","kms:Decrypt"],
  "Resource": "arn:aws:kms:ap-northeast-1:xxxxxxxxxxxx:key/xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx",
  "Condition": {
    "StringEquals": {
      "aws:SourceArn": "arn:aws:cloudtrail:ap-northeast-1:xxxxxxxxxxxx:trail/aws-controltower-BaselineCloudTrail"
    },
    "StringLike": {
      "kms:EncryptionContext:aws:cloudtrail:arn": "arn:aws:cloudtrail:*:xxxxxxxxxxxx:trail/*"
    }
  }
}

After that, I deleted the failed CloudFormation stack AWSControlTowerBP-BASELINE-CLOUDTRAIL-MASTER and retried the Control Tower setup. All errors were resolved!