AWS CloudTrail & AWS CloudWatch

Please visit my website and subscribe to my youtube channel for more articles

https://devops4solutions.com/

It is used for compliance monitoring, risk auditing, and overall governance of your environments

AWS CloudTrail provides you with the ability to log every single action taken by a user, service, role, or even API, from within your AWS account. Each action recorded is treated as an event which can then be analyzed for enhancing the security of your AWS environment.

  • Events: Events are the basic unit of measurement in CloudTrail. Essentially, an event is nothing more than a record of a particular activity either initiated by the AWS services, roles, or even an AWS user. These activities are all logged as API calls that can originate from the Management Console, the AWS SDK, or even the AWS CLI as well. By default, events are stored by CloudTrail with S3 buckets for a period of 7 days. You can view, search, and even download these events by leveraging the events history feature provided by CloudTrail.
  • Trails: Trails are essentially the delivery mechanism, using which events are dumped to S3 buckets. You can use these trails to log specific events within specific buckets, as well as to filter events and encrypt the transmitted log files. By default, you can have a maximum of five trails created per AWS region, and this limit cannot by increased.
  • CloudTrail Logs: Once your CloudTrail starts capturing events, it sends these events to an S3 bucket in the form of a CloudTrail Log file. The log files are JSON text files that are compressed using the .gzip format. Each file can contain one or more events within itself.

Creating your first CloudTrail Trail

To get started, log in to your AWS Management Console and filter the CloudTrail service from the AWS services filter. On the CloudTrail dashboard, select the Create Trail option to get started:

  1. This will bring up the Create Trail wizard. Using this wizard, you can create a maximum of five-trails per region. Type a suitable name for the Trail in to the Trail name field to begin with.
  2. Next, you can either opt to Apply trail to all regions or only to the region out of which you are currently operating. Selecting all regions enables CloudTrail to record events from each region and dump the corresponding log files into an S3 bucket that you specify. Alternatively, selecting to record out of one region will only capture the events that occur from the region out of which you are currently operating. In my case, I have opted to enable the Trail only for the region I’m currently working out of. In the subsequent sections, we will learn how to change this value using the AWS CLI:
Image for post
Image for post
  1. Next, in the Management events section, select the type of events you wish to capture from your AWS environment. By default, CloudTrail records all management events that occur within your AWS account. These events can be API operations, such as events caused due to the invocation of an EC2 RunInstances or TerminateInstances operation, or even non-API based events, such as a user logging into the AWS Management Console, and so on. For this particular use case, I’ve opted to record All management events.

Selecting the Read-only option will capture all the GET API operations, whereas the Write-only option will capture only the PUT API operations that occur within your AWS environment.

  1. Moving on, in the Storage location section, provide a suitable name for the S3 bucket that will store your CloudTrail Log files. This bucket will store all your CloudTrail Log files, irrespective of the regions the logs originated from. You can alternatively select an existing bucket from the S3 bucket selection field:
Image for post
Image for post
  1. Next, from the Advanced section, you can optionally configure a Log file prefix. By default, the logs will automatically get stored under a folder-like hierarchy that is usually of the form AWSLogs/ACCOUNT_ID/CloudTrail/REGION.
  2. You can also opt to Encrypt log files with the help of an AWS KMS key. Enabling this feature is highly recommended for production use.
  3. Selecting Yes in the Enable log file validation field enables you to verify the integrity of the delivered log files once they are delivered to the S3 bucket.
  4. Finally, you can even enable CloudTrail to send you notifications each time a new log file is delivered to your S3 bucket by selecting Yes against the Send SNS notification for every log file delivery option. This will provide you with an additional option to either select a predefined SNS topic or alternatively create a new one specifically for this particular CloudTrail. Once all the required fields are filled in, click on Create to continue.

With this, you should be able to see the newly created Trail by selecting the Trails option from the CloudTrail dashboard’s navigation pane, as shown in the following screenshot:

Image for post
Image for post

Monitoring CloudTrail Logs using CloudWatch

  1. First,you need to configure your Trail to send the captured log events to CloudWatch Logs.
  2. Define custom CloudWatch metric filters to evaluate the log events for specific matches
  3. Once a match is made, you can then additionally configure CloudWatch to trigger corresponding alarms, send notifications, and even perform a remediation action based on the type of alarm generated.
Image for post
Image for post

Create Cloud Watch Log Group

Go to AWS Console -> CLoudWatch -> Logs -> Create Log Group

In this section, we will be using the AWS CLI to integrate the Trail’s logs with Amazon CloudWatch Logs:

  1. First, we will need to create a new CloudWatch Log Group using the following command:
# aws logs create-log-group --log-group-name useast-prod-CloudTrail-LG-01
  1. Next, you will need to extract and maintain the newly created Log Group’s ARN for the forthcoming steps. To do so, type in the following command and make a note of the Log Group’s ARN, as shown here:
# aws logs describe-log-groups
Image for post
Image for post
  1. With the Log Group successfully created, we will now need to create a new IAM Role that will essentially enable CloudTrail to send its logs over to the CloudWatch Log Group. To do so, we first need to create a policy document that assigns the AssumeRole permission to our CloudTrail Trail. Create a new file and paste the following contents into that file. Remember to to create the file with a .json extension:
# vi policy.json 
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
  1. With the file created, use the create-role command to create the role with the required permissions for CloudTrail:
# aws iam create-role --role-name useast-prod-CloudTrail-Role-01 \ 
--assume-role-policy-document file://policy.json
  1. Once this command executed, make a note of the newly created role’s ARN. Next, copy and paste the following role policy document into a new file. This policy document grants CloudTrail the necessary permissions to create a CloudWatch Logs log stream in the Log Group that you created a while back, so as to deliver the CloudTrail events to that particular log stream:
# vi permissions.json
{
"Version": "2012-10-17",
"Statement": [
{

"Sid": "CloudTrailCreateLogStream",
"Effect": "Allow",
"Action": [
"logs:CreateLogStream"
],
"Resource": [
"<YOUR_LOG_GROUP_ARN>"
]

},
{
"Sid": "CloudTrailPutLogEventsToCloudWatch",
"Effect": "Allow",
"Action": [
"logs:PutLogEvents"
],
"Resource": [
"<YOUR_LOG_GROUP_ARN>"
]
}
]
}
  1. Next, run the following command to apply the permissions to the role. Remember to provide the name of the policy that we created during the earlier steps here:
# aws iam put-role-policy --role-name useast-prod-CloudTrail-Role-01 \ --policy-name cloudtrail-policy \ 
--policy-document file://permissions.json
  1. The final step is to update the Trail with the Log Group ARN as well as the CloudWatch Logs role ARN, using the following command snippet:
# aws cloudtrail update-trail --name useast-prod-CloudTrail-01 \ 
--cloud-watch-logs-log-group-arn <YOUR_LOG_GROUP_ARN> \
--cloud-watch-logs-role-arn <YOUR_ROLE_ARN>

With this you have now integrated your CloudTrail Logs to seamlessly flow into the CloudWatch Log Group that we created. You can verify this by viewing the Log Groups provided under the CloudWatch Logs section of your CloudWatch dashboard.

Written by

Devops Automation Enginneer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store