New Relic AWS Control Tower Integration on the AWS Cloud

Quick Start Reference Deployment

QS

May 2021
Rohit Kaul, New Relic
Welly Siauw, AWS Enterprise Support
Shivansh Singh, AWS Quick Start team

Visit our GitHub repository for source files and to post feedback, report bugs, or submit feature ideas for this Quick Start.

This Quick Start was created by New Relic in collaboration with Amazon Web Services (AWS). Quick Starts are automated reference deployments that use AWS CloudFormation templates to deploy key technologies on AWS, following AWS best practices.

Overview

This Quick Start reference deployment guide provides instructions for deploying the New Relic AWS Control Tower integration on the AWS Cloud.

This Quick Start allows you to streamline the observability of your AWS Control Tower managed landing zone with New Relic using New Relic AWS integrations. Once you delpoy this Quick Start, any new enrolled accounts in your AWS Control Tower managed organizations are automatically monitored with your New Relic account, from the moment they are launched. It also lets you link existing accounts with New Relic, in case you’ve already set up your landing zone. You can then manage all your AWS operational data and insights from one place, with no need to hop back and forth between multiple AWS accounts or other set of observability tools.

This Quick Start is for developers, devops engineers, cloud administrators, and system integrators to fully automate observability of your AWS Control Tower landing zone with New Relic.

Amazon may share user-deployment information with the AWS Partner that collaborated with AWS on the Quick Start.

New Relic AWS Control Tower Integration on AWS

The New Relic AWS Control Tower integration Quick Start allows you to scale the observability of your AWS Control Tower managed landing zones with New Relic, right from the moment accounts are enrolled. You spend no time managing New Relic integration while automatically monitoring your AWS enviornment from New Relic.

New Relic AWS Control Tower integration requires that you grant read permission to operational telemetry data from your AWS account. To do this, use an AWS Identity and Access Management (IAM) role that uses IAM cross-account access. New Relic AWS Control Tower integration uses the Amazon CloudWatch API to collect telemetry data for your monitored AWS services. For more information, see Integrations and managed policies.

If you are new to New Relic, please refer to New Relic to learn about it.

AWS costs

You are responsible for the cost of the AWS services and any third-party licenses used while running this Quick Start. There is no additional cost for using the Quick Start.

The AWS CloudFormation templates for Quick Starts include configuration parameters that you can customize. Some of the settings, such as the instance type, affect the cost of deployment. For cost estimates, see the pricing pages for each AWS service you use. Prices are subject to change.

After you deploy the Quick Start, create AWS Cost and Usage Reports to deliver billing metrics to an Amazon Simple Storage Service (Amazon S3) bucket in your account. These reports provide cost estimates based on usage throughout each month and aggregate the data at the end of the month. For more information, see What are AWS Cost and Usage Reports?

Software licenses

This Quick Start requires that you have access to a New Relic account. You can create an account using the New Relic One pricing plan. Be sure to choose a Standard, Pro, or Enterprise tier for access to administrator features and support. If you don’t already have a New Relic account, you can sign up for a perpetually free access to New Relic from AWS Marketplace. The free tier includes 100 GB of data ingest every month, one full access user and unlimited basic users. With your New Relic One pay-as-you-go plan, you only pay for what you use beyond the free tier each month.

Architecture

Deploying this Quick Start builds the following resources in your AWS Control Tower managed accounts.

New Relic AWS Control Tower Integration diagram
Figure 1. New Relic AWS Control Tower integration architecture diagram
  • An account administrator enrolls new or existing AWS accounts in AWS Control Tower, which generates a lifecycle event.

  • The lifecycle event invokes the New Relic StackSet Lambda function via an Amazon EventBridge rule.

  • The New Relic StackSet Lambda function invokes the New Relic register Lambda function via Amazon Simple Notification Service (Amazon SNS) and provides input for the New Relic StackSet.

  • A dead letter queue collects messages that the New Relic register Lambda function fails to process.

  • A NerdGraph API key secret is retrieved from AWS Secrets Manager (not shown). The New Relic register Lambda function calls the New Relic NerdGraph mutation endpoint to link your AWS account with your New Relic account.

  • A New Relic onboarding Lambda function invokes the stack SNS topic and provides input to the New Relic StackSet, which includes a cross-account IAM role.

  • The New Relic StackSet creates a New Relic IAM cross-account role.

Planning the deployment

Specialized knowledge

This deployment requires a moderate level of familiarity with AWS services. If you’re new to AWS, see Getting Started Resource Center and AWS Training and Certification. These sites provide materials for learning how to design, deploy, and operate your infrastructure and applications on the AWS Cloud.

This Quick Start assumes familiarity with AWS Control Tower. For information about AWS Control Tower, see Getting Started with AWS Control Tower in the AWS Control Tower User Guide.

AWS account

If you don’t already have an AWS account, create one at https://aws.amazon.com by following the on-screen instructions. Part of the sign-up process involves receiving a phone call and entering a PIN using the phone keypad.

Your AWS account is automatically signed up for all AWS services. You are charged only for the services you use.

Technical requirements

Before you launch the Quick Start, review the following information and ensure that your account is properly configured. Otherwise, deployment might fail.

Resource quotas

If necessary, request service quota increases for the following resources. You might need to request increases if your existing deployment currently uses these resources and if this Quick Start deployment could result in exceeding the default quotas. The Service Quotas console displays your usage and quotas for some aspects of some services. For more information, see What is Service Quotas? and AWS service quotas.

Resource This deployment uses

AWS CloudFormation stacks

1

AWS CloudFormation StackSets

1

AWS CloudFormation StackSet instances

0–3001

AWS Lambda functions

42

IAM roles

4

IAM managed policies

13

Amazon S3 buckets

1

AWS Secrets Manager

1

Amazon SNS topics

2

1 This Quick Start deploys and maintains StackSet instances for each AWS account that you include in the deployment.

2 All AWS Lambda functions deployed by this Quick Start use unreserved concurrency.

3 This Quick Start deploys a cross-account trust IAM role and associated managed policy for New Relic AWS Control Tower integration on each AWS account that you include in the deployment.

Supported Regions

This Quick Start can be deployed in Regions that are supported by AWS Control Tower.

  • us-east-1

  • us-east-2

  • us-west-2

  • ap-south-1

  • ap-northeast-1

  • ap-northeast-2

  • ap-southeast-1

  • ap-southeast-2

  • ca-central-1

  • eu-central-1

  • eu-west-1

  • eu-west-2

  • eu-north-1

To find the most recent list of Regions supported by AWS Control Tower, run the following AWS CLI command:

aws ssm get-parameters-by-path --path "/aws/service/global-infrastructure/services/controltower/regions" --output json --query "Parameters[].Value"

For more information, see AWS Regional Services.

Certain Regions are available on an opt-in basis. For more information, see Managing AWS Regions.

IAM permissions

Before launching the Quick Start, you must sign in to the AWS Management Console with IAM permissions for the resources that the templates deploy. The AdministratorAccess managed policy within IAM provides sufficient permissions, although your organization may choose to use a custom policy with more restrictions. For more information, see AWS managed policies for job functions.

Prepare your AWS Control Tower account

AWS Control Tower must be deployed before launching this Quick Start. For information about setting up an AWS Control Tower landing zone, see Getting Started with AWS Control Tower.

Prepare your New Relic account

When using the New Relic One pricing plan, you must have an active New Relic account that is subscribed to a standard or higher pricing tier. With a pay-as-you-go plan, you pay only for what you use beyond the free tier (100 GB) each month.

Prepare for the deployment

  1. New Relic account ID

    This Quick Start integrates your landing zone with a single New Relic account. Log in to your New Relic account and find the account ID. For more information, see Account ID.

  2. New Relic NerdGraph User key

    For all deployment options, use a New Relic User key, and enter it as parameter in the deployment. This Quick Start uses New Relic NerdGraph API for linking your AWS accounts with New Relic. For more information, see Introduction to New Relic NerdGraph, our GraphQL API.

    Log in to your New Relic account. Create a User key, if you don’t have one already. For more information, see User key.

  3. New Relic NerdGraph API endpoint

    Determine if your New Relic account is in US or EU data center. For more information, see New Relic data centers. Use the default NerdGraph endpoint unless your New Relic account uses an EU data center. For more information, see NerdGraph endpoints

  4. (Optional) Existing AWS account ID list

    In case you’ve already set up your landing zone, and want to monitor existing AWS accounts enrolled in Control Tower managed organization with New Relic, supply the list of AWS account IDs (as comma separated string).

    • Log in to an AWS Control Tower management account.

    • For a list of managed accounts, navigate to AWS Control Tower.

    • Add your AWS account IDs.

Deployment options

This Quick Start can be deployed as an AWS CloudFormation stack in your AWS Control Tower management account.

This Quick Start enables all AWS integrations that are available in New Relic. For a list of integrations and data, see AWS integrations.

Deployment steps

Confirm your AWS account configuration

  1. Sign in to your AWS Control Tower management account using an IAM user role that has the necessary permissions. For more information, see Planning the deployment, earlier in this guide.

  2. Ensure that your AWS account is configured correctly, as discussed in the Technical requirements section.

Launch the Quick Start

If you want to enroll existing AWS accounts into New Relic AWS Control Tower integration, ensure that your AWS accounts are enrolled in AWS Control Tower. This Quick Start does not support enrolling AWS accounts from organizations that are outside of AWS Control Tower. Your list of existing AWS accounts must be formatted as a comma-delimited string (for example, account_id1,account_id2,account_id3).
  1. Sign in to your AWS Control Tower management account, and launch the AWS CloudFormation template using the following link:

  2. Check the AWS Region displayed in the upper-right corner of the navigation bar, and change it if necessary. This Region is where AWS Control Tower deploys. For other Regions, see Supported Regions, earlier in this guide. Choose Next.

  3. On the Specify stack details page, change the stack name if needed. Review the parameters for the template, and provide values for any parameters that require input. For all other parameters, review the default settings and customize them as necessary. For details about each parameter, see the Parameter reference section. When you finish reviewing and customizing the parameters, choose Next.

  4. On the Configure stack options page, you can specify tags (key-value pairs) for resources in your stack and set advanced options. When you’re finished, choose Next.

  5. On the Review page, review and confirm the template settings. Under Capabilities, select the two check boxes to acknowledge that the template creates IAM resources and might require the ability to automatically expand macros.

  6. Choose Create stack to deploy the stack.

  7. Monitor the status of the stack. When the status is CREATE_COMPLETE, the New Relic AWS Control Tower integration deployment is ready.

  8. To view the created resources, see the values displayed in the Outputs tab for the stack.

Test the deployment

When you enroll a new AWS Control Tower–managed account, the deployment sets up the IAM role for New Relic AWS Control Tower integration. You can then see the new account show up in the New Relic UI.

  1. Log in to your New Relic account.

  2. To view your AWS data, see Introduction to AWS integrations.

Cleanup

If you want to deploy this Quick Start for testing or demonstration purposes, and you don’t intend to use New Relic AWS integrations any longer, follow these steps to remove the AWS CloudFormation stack.

Deleting the New Relic AWS Control Tower integration Quick Start stack and StackSets also deletes the New Relic IAM roles deployed in your AWS accounts, which removes cross-account trust with New Relic. This Quick Start does not, however, remove account links to New Relic. To remove an account from New Relic, see the subsequent section Uninstall the new Relic integration.

Remove the AWS CloudFormation stack

The time to complete this step depends on how many AWS accounts are included in your New Relic AWS Control Tower Integration deployment. If deleting the AWS CloudFormation stack times out, it’s safe to retry this step.

  1. Sign in to the AWS CloudFormation console in your Control Tower management account.

  2. To remove all of the deployed AWS resources, delete the New Relic AWS Control Tower Integration stack. This also deletes all stack set instances, which include any IAM roles deployed for enrolled AWS accounts.

  3. When the stack is deleted, navigate to AWS CloudFormation StackSet, and search for the New Relic stack set to confirm it was removed.

Uninstall the New Relic integration

After you confirm that the AWS CloudFormation stack was removed, remove all enrolled accounts from New Relic.

  1. Log in to your New Relic account.

  2. To uninstall the AWS integration from New Relic, see Uninstall infrastructure integrations.

Best practices for using New Relic AWS Control Tower integration on AWS

For a list of best practices for New Relic AWS Control Tower integration, see the Infrastructure monitoring best practices guide.

For more information, see Best practices guides.

Security

To enhance your security, see New Relic’s Intro to authentication (SAML SSO) for users on original user model. Modify the high-security mode settings, and review the account audit logs (NrAuditEvent). For more information, see Security and privacy.

FAQ

For a list of troubleshooting topics, see New Relic troubleshooting

Q. Can I purchase New Relic from AWS Marketplace?

A. Yes. You can access and purchase New Relic from AWS Marketplace, which offers pay-as-you-go pricing. This includes access to the perpetual free tier. For an additional discount, see Support Information.

Q. What capabilities does New Relic provide?

A. New Relic is an observability platform that helps you build better software. You can import telemetry data from any source to help you understand a system and how to improve it. For more information, see Introduction to New Relic.

Q. Can I ingest additional, external data into New Relic?

A. Yes. New Relic is best used as a single source of truth for operational data, regardless of the data’s origins. In addition to New Relic’s open-source instrumentation agents, New Relic provides a catalog of integrations and open-source tools for ingesting data, such as Prometheus, Fluentd, and Logstash. If these integrations don’t suit your needs, New Relic’s open-source Telemetry SDKs let you build your own integration.

New Relic offers several APIs for retrieving MELT (metrics, events logs, and traces) data types into without using an installed agent. New Relic also offers open-source telemetry integrations that report data from OpenCensus, OpenTelemetry, DropWizard, Prometheus, and more. New Relic’s programmable platform lets you build New Relic One apps to connect system performance to business needs, such as business KPIs and customer engagements. For more information, see Get Data into New Relic.

Q. What is the data retention period for New Relic?

A. New Relic’s data-retention policy ranges from 8 to 395 days, and the ability to edit data retention periods differs depending on your pricing plan. Reducing data retention to below its contracted value does not reduce New Relic’s data ingest charges. Conversely, increasing retention for targeted telemetry data types starts a conversation with New Relic about adjusting your data ingest charges. For more information, see Manage data retention.

Q. Can I delete data from New Relic?

A. After telemetry data is reported to New Relic and available for querying, it cannot be edited or deleted. This is a design decision that optimizes New Relic’s speed and performance at scale. Data expires and is purged when its retention period ends.

Q. How can I track my New Relic usage?

A. New Relic provides a UI for monitoring your usage and managing your data. You can also query your usage to get more detail than is available in the UI and set up alerts to get notifications about changes in your usage. For more information, see Query and alert on billing/usage data.

Q. How does New Relic help to ensure the security and privacy of my data?

A. To learn about how New Relic ensures security, see New Relic Security. For more information, see Security & privacy by design, Security controls for privacy, and Compliance and Certifications.

Q. How much does New Relic cost?

A. If you are an existing New Relic customer, see New Relic One pricing and billing. If you are new to New Relic, see New Relic pricing. For more information, Contact New Relic or see New Relic One pricing plan: Frequently asked questions. You can also sign up for a free New Relic account that allows 100 GB of free data per month.

Q. I encountered a CREATE_FAILED error when I launched the Quick Start.

A. If AWS CloudFormation fails to create the stack, relaunch the template with Rollback on failure set to Disabled. This setting is in the AWS CloudFormation console under Advanced on the Configure stack options page. With this setting, the stack’s state is retained, and the instance remains running so you can troubleshoot the issue. (For Windows, look at the log files in %ProgramFiles%\Amazon\EC2ConfigService and C:\cfn\log.) Also, see Amazon CloudWatch for errors associated with the AWS Lambda function NewRelicOnboardingFunction.

Q. I encountered a size-limitation error when I deployed the AWS CloudFormation templates.

A. Launch the Quick Start templates from the links in this guide or from another S3 bucket. If you deploy the templates from a local copy on your computer, or from a location other than an S3 bucket, you might encounter template-size limitations. For more information, see AWS CloudFormation quotas.

Q. I included a list of AWS accounts during the deployment, but none of the accounts show up in New Relic’s UI.

A. Ensure that your list includes only AWS account IDs that are managed by AWS Control Tower and that the account IDs are separated by commas. Also ensure that you deployed the stack in the same Region as your AWS Control Tower management account and that the New Relic NerdGraph user API key is correct. Lastly, inspect the Amazon CloudWatch logs for errors regarding the AWS Lambda function NewRelicOnboardingFunction.

Customer responsibility

After you successfully deploy this Quick Start, confirm that your resources and services are updated and configured — including any required patches — to meet your security and other needs. For more information, see the AWS Shared Responsibility Model.

Parameter reference

Unless you are customizing the Quick Start templates for your own deployment projects, keep the default settings for the parameters labeled Quick Start S3 bucket name, Quick Start S3 bucket Region, and Quick Start S3 key prefix. Changing these parameter settings automatically updates code references to point to a new Quick Start location. For more information, see the AWS Quick Start Contributor’s Guide.

Parameters for deploying New Relic AWS Control Tower integration

Table 1. New Relic configuration
Parameter label (name) Default value Description

New Relic account ID (NewRelicAccountNumber)

Requires input

New Relic account ID. See https://docs.newrelic.com/docs/accounts/accounts-billing/account-setup/account-id/

New Relic NerdGraph User key (NewRelicAccessKey)

Requires input

New Relic NerdGraph User key. See https://docs.newrelic.com/docs/apis/intro-apis/new-relic-api-keys/#user-api-key

New Relic NerdGraph API endpoint (NerdGraphEndpoint)

https://api.newrelic.com/graphql

New Relic NerdGraph endpoint URL. Use default unless your New Relic account uses an EU data center. See https://docs.newrelic.com/docs/apis/nerdgraph/get-started/introduction-new-relic-nerdgraph/#authentication

Table 2. Deployment configuration
Parameter label (name) Default value Description

Existing AWS account ID list (LaunchAccountList)

Requires input

Comma separated string of existing (enrolled with Control Tower) AWS account IDs that you wish to monitor with New Relic. See https://docs.aws.amazon.com/general/latest/gr/acct-identifiers.html

StackSet name. (StackSetName)

NewRelic-Integration

New Relic integration StackSet name

StackSet template URL. (StackSetUrl)

https://aws-quickstart.s3.amazonaws.com/quickstart-ct-newrelic-one/templates/newrelic-stack-set.yml

New Relic integration StackSet template URL

Table 3. AWS Quick Start configuration
Parameter label (name) Default value Description

Quick Start S3 bucket name. (QSS3BucketName)

aws-quickstart

S3 bucket for Quick Start assets. Use this if you want to customize your deployment. The bucket name can include numbers, lowercase letters, uppercase letters, and hyphens, but it cannot start or end with hyphens (-).

Quick Start S3 key prefix. (QSS3KeyPrefix)

quickstart-ct-newrelic-one/

S3 key prefix to simulate a directory for Quick Start assets. Use this if you want to customize your deployment. The prefix can include numbers, lowercase letters, uppercase letters, hyphens (-), and forward slashes (/). For more information, see https://docs.aws.amazon.com/AmazonS3/latest/dev/UsingMetadata.html.

Send us feedback

To post feedback, submit feature ideas, or report bugs, use the Issues section of the GitHub repository for this Quick Start. To submit code, see the Quick Start Contributor’s Guide.

Quick Start reference deployments

GitHub repository

Visit our GitHub repository to download the templates and scripts for this Quick Start, to post your comments, and to share your customizations with others.


Notices

This document is provided for informational purposes only. It represents AWS’s current product offerings and practices as of the date of issue of this document, which are subject to change without notice. Customers are responsible for making their own independent assessment of the information in this document and any use of AWS’s products or services, each of which is provided “as is” without warranty of any kind, whether expressed or implied. This document does not create any warranties, representations, contractual commitments, conditions, or assurances from AWS, its affiliates, suppliers, or licensors. The responsibilities and liabilities of AWS to its customers are controlled by AWS agreements, and this document is not part of, nor does it modify, any agreement between AWS and its customers.

The software included with this paper is licensed under the Apache License, version 2.0 (the "License"). You may not use this file except in compliance with the License. A copy of the License is located at http://aws.amazon.com/apache2.0/ or in the accompanying "license" file. This code is distributed on an "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either expressed or implied. See the License for specific language governing permissions and limitations.