Please visit my website and subscribe to my youtube channel for more articles

https://devops4solutions.com/

Cloud Custodian is an open source framework which is used to monitor your AWS infrastruture and run the policies which are written in yaml against your whole AWS account/regions. The policies is like a rule which is put in the yaml file to find out if something is not compilant.

Basic Conecpts:

  1. Policy — c7n.policyDefined in yaml, specifies a set of filters and actions to take on a given AWS resource type.
  2. Resource — Any resource on AWS which you want to monitor
  3. Mode — c7n.policy (yes, policy)Defines how the policy will execute (lambda, config rule, poll, etc). Policies run in pull mode by default.Modes are responding to cloudwatch events and that can be the reason of running any policy locally or manually.If you want to pull all resources of a specific type based on your filter, you can remove the mode section to run it manually.

TIPS:

generally, think of the mode: cloudtrail policies as real-time remediation policies, if you have less urgent/more resource intensive policies you should run those on a cron outside of lambda

For ex —

  1. I need to see if my whole EC2 instances are properly tagged or not.
  2. All RDS instances should not be publicly available
  3. IF any EC2 instance is idle for 10 days then terminate it

Accounts Used

  1. security account where all installation and configuration of cloud custodian will reside
  2. Other accounts like sandbox-02 which will get monitored using this

Installation

Prerequisite:

Python

Check which python version is installed, if not then install python first

Installation of cloud custodian is completed successfully.

Configure AWS CLI

You need python-pip on that machine to install awscli

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpm

Once you configured your keys using aws configure, everything is stored under

cd ~/.aws/
pwd
/home/ec2-user/.aws
cat credentials

Write your first policy:

Create a file custodian.yml

vi custodian.yml

Put the below content in your file. This will fetch all the instances which are in running state.

policies:
- name: my-first-policy
resource: ec2
filters:
- "State.Name": running

Validate your policy

custodian validate custodian.yml

Dry run : This will show you what your policy will do before actually running it

custodian run --dryrun -s . custodian.yml

Now run your yaml file

custodian run --output-dir=. custodian.yml
Image for post
Image for post

Now we will modify the policy and rerun the same command and you will see in output that count is 0.

policies:
- name: my-first-policy
resource: ec2
filters:
- "State.Name": running
- "tag:Custodian":present
Image for post
Image for post

This is how we can write the rules as per your need.

Cloud Custodian Sample Examples

Click here . Here some predefined roles provided by the cloud custodian. You can use them directly or customize it as per your need.

I am trying few use cases as below

Example 1 : Account — Login From Invalid IP Address

This example needs you to create an IAM role with AWSLambdaFullAccess and we will assume that role in our yaml configuration. If you will not create this role you will get an error “ assert role, “Lambda function role must be specified”
AssertionError: Lambda function role must be specified”

Step 1 Create IAM Role

Go to AWS Console -> IAM ->Create Role -> Select Lambda Access -> AWSLambdaFullAccess -> Create Policy (cloud-custodian-role)

Image for post
Image for post
Image for post
Image for post

Step 2: Create yaml file

policies:

- name: invalid-ip-address-login-detected
resource: account
description: |
Notifies on invalid external IP console logins
mode:
type: cloudtrail
role:yourrolearn
events:
- ConsoleLogin
filters:
- not:
- type: event
key: 'detail.sourceIPAddress'
value: |
'^((158\.103\.|142\.179\.|187\.39\.)([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))|(12\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))$'
op: regex

Step 3 Run the yaml file

if you are not in custodian folder you always need to run the below command
source custodian/bin/activate
(custodian) custodian run --output-dir=. policy/ipaddress.yml
Image for post
Image for post

Example 2

Root Account login

Step 1 Create rootaccount.yml file

policies:

- name: root-user-login-detected
resource: account
description: |
Notifies Security and Cloud Admins teams on any AWS root user console logins
mode:
type: cloudtrail
role: yourrolearn
events:
- ConsoleLogin
filters:
- type: event
key: "detail.userIdentity.type"
value_type: swap
op: in
value: Root

Step 2 Run the yaml file

if you are not in custodian folder you always need to run the below command
source custodian/bin/activate
(custodian) custodian run --profile=local --output-dir=. policy/rootlogin.yml

Lambda function will get created for both the examples on your AWS console.

Image for post
Image for post

To test the lambda function manually for the root login policy, you can use this json

Example 3: Policy to find ec2 instances which doesn’t have tag owner and as part of action add the tag automatically. Create the policy tag.yml

policies:
- name: tagging-owner
resource: ec2
filters:
- "tag:owner": absent
actions:
- type: tag
key: owner
value: "Tagging example"

Now run it

custodian run --profile=local --output-dir=. policy/tag.yml

Cloud Custodian for multiple accounts

Click here

c7n-org is a tool to run custodian against multiple AWS accounts, Azure subscriptions, or GCP projects in parallel.

Installation

pip install c7n-org

It will get installed at the location /home/ec2-user/custodian/bin

In order to run c7n-org against multiple accounts, a config file must first be created containing information about all the accounts with regions.

Example of this config file , so we need to create an IAM role on those account that primary account can assume like this

accounts:
- account_id: '123123123123'
name: account-1
regions:
- us-east-1
- us-west-2
role: arn:aws:iam::123123123123:role/CloudCustodian

Let’s create IAM role on account (sandbox02) which you want to monitor

On sandbox2 account,I am creating a role “cloudcustodian-role”, select lambda access with Administrative permissions

Image for post
Image for post
Image for post
Image for post

How to assume this role on the cloud custodian account

Now on cloudcustodian account ->

create a new file called accounts.yml

accounts:
- account_id: '123123123123'
name: account-1
regions:
- us-east-1
- us-west-2
role: arn:aws:iam::123123123123:role/CloudCustodian

After running above command, you will get the list of EC2 instances which are in running state on that account. It will create the output directory and with the directory structure as mentioned below

Image for post
Image for post

Cross Account Access

I have two accounts (sandbox02 and security account)

Requirement

  1. I want to run some scripts from security account to sandbox02 account
  2. Need to create one role on sandbox account with permissions(for now i have given administrative permissions) to another account (security account)
  3. Security account will assume that role.
  4. Attach this role to EC2 Instance
  5. Need to remove the AWS default credentials which are present in ~/.aws

On Sandbox account ->

  1. Create a role “cloudcustodian-role”, put the security account number
Image for post
Image for post

Trust Relationship

Image for post
Image for post

Policy — Administrative access.

On security account

I created one role “cloudcustodian-crossaccountrole” and one policy

cloud-custodian-crossaccount-policy which assume the role which we have created in the sandbox-02 account

Trust relationship of the role is below

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}

Policy as mention below

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::aacountnumber ofsandbox:role/cloudcustodian-role"
}
]
}

Now in your accounts.yaml file, put the role which we have created on sandbox-02 account and create new yaml file ,just remove the hard coded role details from there.

cd /home/ec2-user/custodian/bin
c7n-org run -c policy/accounts.yml -s output -u policy/rootlogin_cross.yml
Image for post
Image for post

After this you will see two lamda function created successfully on sandbox02 account

Image for post
Image for post

Install cloud custodian emailer

Click Here for more details

Create a directory

$ mkdir mailer

Clone the repository:

$ git clone https://github.com/capitalone/cloud-custodian

Install dependencies (with virtualenv):

$ virtualenv c7n_mailer
$ source c7n_mailer/bin/activate
$ cd /home/ec2-user/mailer/cloud-custodian/tools/c7n_mailer
$ pip install -r requirements.txt

Install the extensions:

python setup.py develop

Create SQS queue

Go to AWS Console -> SQS -> Create QuickQueue

Create IAM role which has read access to this queue ( I have given administrative access also you can refine as per your need )

Image for post
Image for post

Now you have lambda as a trusted entity

Image for post
Image for post

You have to verify your email address also, go to AWS Console ->SES -> Identity Management -> Email Address -> Verify.

You have to verify all email addresses (all recipients list) and this is region specific.

Create a mailer.yml and put the below content

queue_url: https://sqs.us-east-1.amazonaws.com/1234567890/c7n-mailer-test
role: arn:aws:iam::123456790:role/c7n-mailer-test
from_address: you@example.com

Now let’s make a Custodian policy to populate your mailer queue. Create a test-policy.yml: Change the emailid and queue name

policies:
- name: c7n-mailer-test
resource: sqs
filters:
- "tag:MailerTest": absent
actions:
- type: notify
template: default
priority_header: '2'
subject: testing the c7n mailer
to:
- you@example.com
transport:
type: sqs
queue: https://sqs.us-east-1.amazonaws.com/1234567890/c7n-mailer-test

Now run

source c7n_mailer/bin/activate

Written by

Devops Automation Enginneer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store