How to apply auto tags to new resources in AWS using Cloud Custodian

Cloud Custodian logo — cloudcustodian.io | AWS logo — aws.amazon.com

Following on the previous post about enforcing tags with Cloud Custodian.

Simple AWS tag enforcement with Cloud Custodian

I wanted to demonstrate how to implement automatic tagging for new AWS resources using the same set of tools. This is a great way to standardize tagging across the board, but it’s important to carefully consider the implications before moving forward.

Without further ado, here are the steps to begin auto-tagging resources in your AWS account.

We’ll continue with our Infrastructure as Code (IaC) approach and either utilize CloudFormation or Terraform for the template. For this post, I’ll be using CloudFormation.

IAM Role and Instance Profile

As stated in my previous posts: “you always start with IAM” and since we are using an instance to…

1. Create the Lambdas and CloudWatch Events.
2. Pass the appropriate roles to the new resources
3. List, Fetch, and Put objects to S3.

…we will need to get very specific and add individual statements or policies to control the access of this particular instance. Alternatively, you can use managed policies to create the role, which may be more efficient. Just remember that this role will be assumed by the Lambda to make changes to the new resources.

Once you create the role, make sure you include the sts policy to allow lambda services to assume the role.

NOTE: An alternative is to include another role just for Custodian Lambdas and limit the Ec2 role to the bare minimum.

S3 Bucket (Cloud Custodian Files)

Please ensure to remember the following instructions for our S3 Bucket:

This S3 Bucket will be used to store our Cloud Custodian policies and logs generated from each custodian run. Please note that these logs can also be directed to CloudWatch, but that will be discussed in a separate post.

Be sure to implement a bucket policy granting read & write rights, with the Custodian role set as the only principal.

The goal here is to be able to develop a script within our deployment, to retrieve these files on a scheduled basis. Additionally, we need to create a script for uploading the logs or utilize CloudWatch for the logging function.

Please also note that you have the option to use your preferred Source Code Management (SCM) platform to maintain a collection of policies. If you choose to do so, you will need to adjust the fetching process accordingly.

CloudTrail + S3 Bucket (Trail Logs)

This process is quite straightforward. We must enable a CloudTrail (Trail) to trigger a CloudWatch Event, which will then execute the Custodian Lambda to automatically tag the new resources.

Please ensure that you include the appropriate bucket policies for S3, for example:

Statement:
-
Sid: "AWSCloudTrailAclCheck"
Effect: "Allow"
Principal:
Service: "cloudtrail.amazonaws.com"
Action: "s3:GetBucketAcl"
Resource:
!Sub |-
arn:aws:s3:::${BucketName}
-
Sid: "AWSCloudTrailWrite"
Effect: "Allow"
Principal:
Service: "cloudtrail.amazonaws.com"
Action: "s3:PutObject"
Resource:
!Sub |-
arn:aws:s3:::${BucketName}/AWSLogs/${AWS::AccountId}/*
Condition:
StringEquals:
s3:x-amz-acl: "bucket-owner-full-control"

EC2 Instance (Cloud Custodian)

Cloud Custodian is a very lightweight tool and therefore we don’t need high specs for this instance. Use the free tier type and storage class.

For the Userdata script, we’ll include the following:

#!/bin/bash -xe
#Update packages 
sudo yum update -y
#Install python & pip
sudo yum install python3 python3-pip -y
#Install custodian bin
sudo pip3 install c7n
#Generate custodian policies path
sudo mkdir /opt/custodian
#Generate script to fetch policies
echo $'#!/bin/bash\nBUCKET=${BucketName}\nsudo aws s3 cp s3://$BUCKET/ /opt/custodian/ --recursive' > /home/ec2-user/custodiansync
#Modify permissions, move script to hourly cron jobs, set execution
sudo chmod 775 /home/ec2-user/custodiansync
sudo mv /home/ec2-user/custodiansync /etc/cron.hourly/custodiansync sudo chmod +x /etc/cron.hourly/custodiansync

Please note that we are using the substitution function to pass the custodian bucket parameter to the new script we are creating locally. This script will run on an hourly basis to pull the logs and policies from S3.

Cloud Custodian Policy and Execution

For this post, we will concentrate on EC2 instances exclusively. Our goal is to ensure that all instances are automatically tagged with the username of the creator. Keeping that in mind, we will proceed to create the following YAML file:

policies:
- name: ec2-auto-tag-user
resource: ec2
mode:
type: cloudtrail
role: arn:aws:iam::{account_id}:role/Custodian
events:
- RunInstances
timeout:
- 10
filters:
- tag:Creator: absent
actions:
- type: auto-tag-user
tag: Creator
principal_id_tag: CreatorId

NOTE: Make sure you are using the name you gave to the role you created at the beginning of this post!

Save the file and upload it to the Custodian S3 Bucket we created a while back. The files will eventually be downloaded to the instance.

Alright, now with the policy file in our instance, we can manually run the policy by using the following command:

sudo /usr/local/bin/custodian run --output-dir=./logs --cache-period 0 --region us-east-1 /opt/custodian/policies/ec2-auto-tag-user.yml

Make sure you adjust the above to match your AWS resource region, file name, and the path of the policy. The successful execution of the policy will create a Lambda Function and a CloudWatch Event Rule based on CloudTrail events matching “RunInstances”; Meaning that any new instances will trigger the lambda to add the needed tags. ALL DONE! you can now stop the instance.

Conclusion

Hopefully the above gives you a better idea of how to better implement Cloud Custodian, to your company or client's cloud infrastructure, in a way that improves cloud governance and makes your architecture more responsive and receptive. Please note that the above is only a small demonstration of Cloud Custodian's capabilities!

BONUS: since the policy runs on CloudTrail Events, you can add more policies to the same document using the appropriate resource you wish to monitor and the respective trail event. eg:

VPC - CreateNatGateway
VPC - CreateKeyPair
VPC - CreateNetworkAcl
VPC - CreateNetworkInterface
VPC - CreateRouteTable
VPC - CreateVpc
VPC - CreateVPCEndpoint
VPC - CreateVpcPeeringConnection
VPC - CreateVpnConnection
VPC - CreateVpnGateway
VPC - CreateSecurityGroup
EFS - CreateFilteSystem