Please visit my website and subscribe to my youtube channel for more articles

https://devops4solutions.com/

Cloud Custodian is an open source framework which is used to monitor your AWS infrastruture and run the policies which are written in yaml against your whole AWS account/regions. The policies is like a rule which is put in the yaml file to find out if something is not compilant.

Basic Conecpts:

TIPS:

generally, think of the mode: cloudtrail policies as real-time remediation policies, if you have less urgent/more resource intensive policies you should run those on a cron outside of lambda

For ex —

Accounts Used

Installation

Prerequisite:

Python

Check which python version is installed, if not then install python first$ virtualenv --python=python2.7 custodian
$ source custodian/bin/activate
(custodian) $ pip install c7n

Installation of cloud custodian is completed successfully.

Configure AWS CLI

You need python-pip on that machine to install awscli

yum install https://dl.fedoraproject.org/pub/epel/epel-release-latest-7.noarch.rpmsudo yum install python-pipsudo pip install awscli --upgrade pipRun the below command after logging to some user, for eg -ec2-user not as a root useraws configure

Once you configured your keys using aws configure, everything is stored under

cd ~/.aws/
pwd
/home/ec2-user/.aws
cat credentials

Write your first policy:

Create a file custodian.yml

vi custodian.yml

Put the below content in your file. This will fetch all the instances which are in running state.

policies:
- name: my-first-policy
resource: ec2
filters:
- "State.Name": running

Validate your policy

custodian validate custodian.yml

Dry run : This will show you what your policy will do before actually running it

custodian run --dryrun -s . custodian.yml

Now run your yaml file

custodian run --output-dir=. custodian.yml
Image for post
Image for post

Now we will modify the policy and rerun the same command and you will see in output that count is 0.

policies:
- name: my-first-policy
resource: ec2
filters:
- "State.Name": running
- "tag:Custodian":present
Image for post
Image for post

This is how we can write the rules as per your need.

Cloud Custodian Sample Examples

Click here . Here some predefined roles provided by the cloud custodian. You can use them directly or customize it as per your need.

I am trying few use cases as below

Example 1 : Account — Login From Invalid IP Address

This example needs you to create an IAM role with AWSLambdaFullAccess and we will assume that role in our yaml configuration. If you will not create this role you will get an error “ assert role, “Lambda function role must be specified”
AssertionError: Lambda function role must be specified”

Step 1 Create IAM Role

Go to AWS Console -> IAM ->Create Role -> Select Lambda Access -> AWSLambdaFullAccess -> Create Policy (cloud-custodian-role)

Image for post
Image for post
Image for post
Image for post

Step 2: Create yaml file

policies:

- name: invalid-ip-address-login-detected
resource: account
description: |
Notifies on invalid external IP console logins
mode:
type: cloudtrail
role:yourrolearn
events:
- ConsoleLogin
filters:
- not:
- type: event
key: 'detail.sourceIPAddress'
value: |
'^((158\.103\.|142\.179\.|187\.39\.)([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))|(12\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])
\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5])\.([01]?[0-9]?[0-9]|2[0-4][0-9]|25[0-5]))$'
op: regex

Step 3 Run the yaml file

if you are not in custodian folder you always need to run the below command
source custodian/bin/activate
(custodian) custodian run --output-dir=. policy/ipaddress.yml
Output of this command2018-06-14 13:50:08,544: custodian.policy:INFO Provisioning policy lambda invalid-ip-address-login-detected
2018-06-14 13:50:08,636: custodian.lambda:INFO Publishing custodian policy lambda function custodian-invalid-ip-address-login-detected
Image for post
Image for post

Example 2

Root Account login

Step 1 Create rootaccount.yml file

policies:

- name: root-user-login-detected
resource: account
description: |
Notifies Security and Cloud Admins teams on any AWS root user console logins
mode:
type: cloudtrail
role: yourrolearn
events:
- ConsoleLogin
filters:
- type: event
key: "detail.userIdentity.type"
value_type: swap
op: in
value: Root

Step 2 Run the yaml file

if you are not in custodian folder you always need to run the below command
source custodian/bin/activate
(custodian) custodian run --profile=local --output-dir=. policy/rootlogin.yml
Output of this command
2018-06-14 14:16:24,339: custodian.policy:INFO Provisioning policy lambda root-user-login-detected
2018-06-14 14:16:24,459: custodian.lambda:INFO Publishing custodian policy lambda function custodian-root-user-login-detected

Lambda function will get created for both the examples on your AWS console.

Image for post
Image for post

To test the lambda function manually for the root login policy, you can use this json

Example 3: Policy to find ec2 instances which doesn’t have tag owner and as part of action add the tag automatically. Create the policy tag.yml

policies:
- name: tagging-owner
resource: ec2
filters:
- "tag:owner": absent
actions:
- type: tag
key: owner
value: "Tagging example"

Now run it

custodian run --profile=local --output-dir=. policy/tag.yml

Cloud Custodian for multiple accounts

Click here

c7n-org is a tool to run custodian against multiple AWS accounts, Azure subscriptions, or GCP projects in parallel.

Installation

pip install c7n-org

It will get installed at the location /home/ec2-user/custodian/bin

In order to run c7n-org against multiple accounts, a config file must first be created containing information about all the accounts with regions.

Example of this config file , so we need to create an IAM role on those account that primary account can assume like this

accounts:
- account_id: '123123123123'
name: account-1
regions:
- us-east-1
- us-west-2
role: arn:aws:iam::123123123123:role/CloudCustodian

Let’s create IAM role on account (sandbox02) which you want to monitor

On sandbox2 account,I am creating a role “cloudcustodian-role”, select lambda access with Administrative permissions

Image for post
Image for post
Image for post
Image for post

How to assume this role on the cloud custodian account

Now on cloudcustodian account ->

create a new file called accounts.yml

accounts:
- account_id: '123123123123'
name: account-1
regions:
- us-east-1
- us-west-2
role: arn:aws:iam::123123123123:role/CloudCustodian
source custodian/bin/activatec7n-org run -c policy/accounts.yml -s output -u policy/custodian.yml --dryrunc7n-org run -c policy/accounts.yml -s output -u policy/custodian.yml

After running above command, you will get the list of EC2 instances which are in running state on that account. It will create the output directory and with the directory structure as mentioned below

Image for post
Image for post

Cross Account Access

I have two accounts (sandbox02 and security account)

Requirement

On Sandbox account ->

Image for post
Image for post

Trust Relationship

Image for post
Image for post

Policy — Administrative access.

On security account

I created one role “cloudcustodian-crossaccountrole” and one policy

cloud-custodian-crossaccount-policy which assume the role which we have created in the sandbox-02 account

Trust relationship of the role is below

{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com",
"lambda.amazonaws.com"
]
},
"Action": "sts:AssumeRole"
}
]
}

Policy as mention below

{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "VisualEditor0",
"Effect": "Allow",
"Action": "sts:AssumeRole",
"Resource": "arn:aws:iam::aacountnumber ofsandbox:role/cloudcustodian-role"
}
]
}

Now in your accounts.yaml file, put the role which we have created on sandbox-02 account and create new yaml file ,just remove the hard coded role details from there.

cd /home/ec2-user/custodian/bin
c7n-org run -c policy/accounts.yml -s output -u policy/rootlogin_cross.yml
c7n-org run -c policy/accounts.yml -s output -u policy/ipaddress_cross.yml
Image for post
Image for post

After this you will see two lamda function created successfully on sandbox02 account

Image for post
Image for post

Install cloud custodian emailer

Click Here for more details

Create a directory

$ mkdir mailer

Clone the repository:

$ git clone https://github.com/capitalone/cloud-custodian

Install dependencies (with virtualenv):

$ virtualenv c7n_mailer
$ source c7n_mailer/bin/activate
$ cd /home/ec2-user/mailer/cloud-custodian/tools/c7n_mailer
$ pip install -r requirements.txt

Install the extensions:

python setup.py develop

Create SQS queue

Go to AWS Console -> SQS -> Create QuickQueue

Create IAM role which has read access to this queue ( I have given administrative access also you can refine as per your need )

Image for post
Image for post

Now you have lambda as a trusted entity

Image for post
Image for post

You have to verify your email address also, go to AWS Console ->SES -> Identity Management -> Email Address -> Verify.

You have to verify all email addresses (all recipients list) and this is region specific.

Create a mailer.yml and put the below content

queue_url: https://sqs.us-east-1.amazonaws.com/1234567890/c7n-mailer-test
role: arn:aws:iam::123456790:role/c7n-mailer-test
from_address: you@example.com

Now let’s make a Custodian policy to populate your mailer queue. Create a test-policy.yml: Change the emailid and queue name

policies:
- name: c7n-mailer-test
resource: sqs
filters:
- "tag:MailerTest": absent
actions:
- type: notify
template: default
priority_header: '2'
subject: testing the c7n mailer
to:
- you@example.com
transport:
type: sqs
queue: https://sqs.us-east-1.amazonaws.com/1234567890/c7n-mailer-test

Now run

source c7n_mailer/bin/activatec7n-mailer --config mailer.yml --update-lambda && custodian run -c test-policy.yml -s .Output:
2018-06-18 15:13:15,738: custodian.policy:INFO policy: c7n-mailer-test resource:sqs region:us-west-2 count:1 time:0.00
2018-06-18 15:13:16,232: custodian.actions:INFO sent message:a7ff12b2-3625-4c56-9df5-79930cf5c7c1 policy:c7n-mailer-test template:default count:1
2018-06-18 15:13:16,232: custodian.policy:INFO policy: c7n-mailer-test action: notify resources: 1 execution_time: 0.49
Below command is to send any email which are in queue
c7n-mailer --run -c mailer.yml
Output:(c7n_mailer) [ec2-user@ip-172-31-24-18 mailer]$ c7n-mailer --run -c mailer.yml
2018-06-18 15:13:57,748 - custodian-mailer - INFO - Downloading messages from the SQS queue.
2018-06-18 15:13:58,159 - custodian-mailer - INFO - Sending account:cydaptivlabs-secops policy:c7n-mailer-test sqs:1 email:default to [u'khandelwal12nidhi@gmail.com']
2018-06-18 15:13:58,291 - custodian-mailer - INFO - Sending account:cydaptivlabs-secops policy:c7n-mailer-test sqs:1 email:default to [u'khandelwal12nidhi@gmail.com']
2018-06-18 15:13:58,510 - custodian-mailer - INFO - Sending account:cydaptivlabs-secops policy:c7n-mailer-test sqs:1 email:default to [u'khandelwal12nidhi@gmail.com']
2018-06-18 15:13:58,768 - custodian-mailer - INFO - Sending account:cydaptivlabs-secops policy:c7n-mailer-test sqs:1 email:default to [u'khandelwal12nidhi@gmail.com']
2018-06-18 15:14:08,783 - custodian-mailer - INFO - No sqs_messages left on the queue, exiting c7n_mailer.
If there are no messages in the queue, the following output will get displayed(c7n_mailer) [ec2-user@ip-172-31-24-18 mailer]$ c7n-mailer --run -c mailer.yml
2018-06-18 15:16:02,243 - custodian-mailer - INFO - Downloading messages from the SQS queue.
2018-06-18 15:16:12,333 - custodian-mailer - INFO - No sqs_messages left on the queue, exiting c7n_mailer.

Devops Automation Enginneer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store