Make sure to include the following in your manual tests:

  • Hitting AWS limits (for example, VPCs, Amazon EBS, Elastic IP addresses).
  • Invalid product licenses or operating system registration.
  • Inactive AWS Marketplace AMI subscriptions.
  • Stacks launched within the user’s VPC that don’t use our NAT gateway/instance or internet gateway. This becomes an issue when an operating system doesn’t recognize the instances and repository registrations fail, which causes downstream failures.
  • Stack deletion. Make sure that deleting a CREATE_COMPLETE stack succeeds in a single try and doesn’t display a DELETE_FAILED status message.
  • Template validation. Launch your templates in the AWS CloudFormation console to check parameters and test the user experience. This review will make it easier to detect issues with parameter names, labels, defaults, and descriptions. For example:
    • If parameters appear in an “Other parameters” group in the console, they’re orphaned in the template. You’ll need to add these parameters to the appropriate group in the Metadata section of your template.
    • If you’ve defined default values that don’t comply with parameter constraints (AllowedValues or AllowedPatterns), the console will display an error message. You’ll need to fix the default value or revise the constraint.
    • The visual test pass will also help you identify cosmetic issues and inconsistencies such as typos in parameter labels and descriptions, or truncated descriptions. Be sure to follow the guidelines in the Follow parameter naming standards section of this guide.
    • For more information about parameter design and defaults, see the Design your parameters section of this guide.