AWS Management & Governance - CloudFormation

 

Cloudform Templates: https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/template-reference.html

SUMMARY:

• Example 1: Create an EC2 instance using CloudFormation Template.
• Example 2: Create two resources (EC2 instance and S3 bucket). Follow the same steps are mentioned in Example1 for execution.
• Example 3: Create EC2 instance and attach a Security group with EC2 instance.
• Example 4: Create an IAM Group and Add a user to that group.
• Example 5: Serverless End to End Web application deployment using Beanstalk

Prerequisite:

• Download and Install VSCode
• Add CloudFormation Linter Extension to VSCode  
• Python latest version is to be installed on your system
• A default VPC for the users in the region must have been configured              

Example 1: Create an EC2 instance using CloudFormation Template.

Step 1 Open a file .yaml is visual studio code.

Step 2: Add Cloudformation Linter Extension to your VSCode

Step 3: Run the following commands in the terminal

           pip install cfn-lint.

           pip install pydot

Step 4: Create a new file with extension .yaml or .yml (main.yml)

Step 5: Write the below sample code to create an EC2 instance using the Cloudformation template (change the ImageId and KeyName accordingly)

Resources:

  MyEC2Instance:

   Type: AWS::EC2::Instance

   Properties:

     ImageId: ami-04893cdb768d0f9ee

     InstanceType: t2.micro

     KeyName: IrfanSydney

     Tags:

       - Key: Name

         Value: irfan-ec2-viacloudform

Step 6: Go to Cloudformation Service.

Step 7: Click on Create Stack button.

• Select option Template is ready
• Select Upload a Template file
• Choose your file

Step 8: Click on Next

Step 9: Provide Stack name and Click on Next

Step 10: Review and Create the stack.

Step 11: Verify the Events and once All the resources (EC2 instance) get created then check if the EC2 instance got created or not.

Experiment: Delete Stack and Check the EC2 instance also get deleted.

Ans: yes it is deleted also.

Example 2: Create two resources (EC2 instance and S3 bucket). Follow the same steps are mentioned in Example1 for execution.

Resources:

  MyEC2Instance:

   Type: AWS::EC2::Instance

   Properties:

     ImageId: ami-04893cdb768d0f9ee

     InstanceType: t2.micro

     KeyName: devops

     Tags:

       - Key: Name

         Value: Ec2-Demo

  S3Bucket:

    Type: 'AWS::S3::Bucket'

    DeletionPolicy: Retain

    Properties:

      BucketName: mumbairegionmy      

in example2, NEED TO REMEMBER THAT BUCKET NAME CANNOT HAVE UPPERCASE. ALL MUST LOWERCASE.

BUT when deleting the cloudformation, S3 bucket is not deleted!!

Example 3: Create EC2 instance and attach a Security group with EC2 instance.

Resources:

  MyEC2Instance:

    Type: AWS::EC2::Instance

    Properties:

      ImageId: ami-0916f5ee07e7b15d6

      InstanceType: t2.micro

      KeyName: IrfanSydney

      Tags:

        - Key: Name

          Value: irfan-EC2-viaCloudformation

      SecurityGroupIds:

        - sg-04fea68279857c128

if you have deleted the default VPC, then you will need to create a new security group cause if not the networks of the old security groups are different.

if you delete the cloudform for ec2 and security group, the ec2 will be deleted but security group will not.

Example 4: Create an IAM Group and Add a user to that group.

Resources:

  iamGroup:

    Type: AWS::IAM::Group

    Properties:

      GroupName: irfan-groupviacloudform

  iamUser:

    Type: AWS::IAM::User

    Properties:

      UserName: irfan-user-viacloudform

      Path: /

      Groups:

        - irfan-groupviacloudform

      LoginProfile:

        Password: <addyourpassword>

        PasswordResetRequired: false

if you delete the cloudform for user and groupuser, the user and groupuser also gets deleted!!

Example 5: Serverless End to End Web application deployment using Beanstalk via Cloudform

AWSTemplateFormatVersion: '2010-09-09'

Resources:

  sampleApplication:

    Type: AWS::ElasticBeanstalk::Application

    Properties:

      Description: AWS Elastic Beanstalk Sample Application

  sampleApplicationVersion:

    Type: AWS::ElasticBeanstalk::ApplicationVersion

    Properties:

      ApplicationName:

        Ref: sampleApplication

      Description: AWS ElasticBeanstalk Sample Application Version

      SourceBundle:

        S3Bucket: !Sub "elasticbeanstalk-samples-${AWS::Region}"

        S3Key: php-newsample-app.zip

  sampleConfigurationTemplate:

    Type: AWS::ElasticBeanstalk::ConfigurationTemplate

    Properties:

      ApplicationName:

        Ref: sampleApplication

      Description: AWS ElasticBeanstalk Sample Configuration Template

      OptionSettings:

      - Namespace: aws:autoscaling:asg

        OptionName: MinSize

        Value: '2'

      - Namespace: aws:autoscaling:asg

        OptionName: MaxSize

        Value: '6'

      - Namespace: aws:elasticbeanstalk:environment

        OptionName: EnvironmentType

        Value: LoadBalanced

      - Namespace: aws:autoscaling:launchconfiguration

        OptionName: IamInstanceProfile

        Value: !Ref MyInstanceProfile        

      SolutionStackName: 64bit Amazon Linux 2 v3.3.11 running PHP 8.0

  sampleEnvironment:

    Type: AWS::ElasticBeanstalk::Environment

    Properties:

      ApplicationName:

        Ref: sampleApplication

      Description: AWS ElasticBeanstalk Sample Environment

      TemplateName:

        Ref: sampleConfigurationTemplate

      VersionLabel:

        Ref: sampleApplicationVersion

  MyInstanceRole:

    Type: AWS::IAM::Role

    Properties:

      AssumeRolePolicyDocument:

        Version: 2012-10-17

        Statement:

          - Effect: Allow

            Principal:

              Service:

                - ec2.amazonaws.com

            Action:

              - sts:AssumeRole

      Description: Beanstalk EC2 role

      ManagedPolicyArns:

        - arn:aws:iam::aws:policy/AWSElasticBeanstalkWebTier

        - arn:aws:iam::aws:policy/AWSElasticBeanstalkMulticontainerDocker

        - arn:aws:iam::aws:policy/AWSElasticBeanstalkWorkerTier

  MyInstanceProfile:

    Type: AWS::IAM::InstanceProfile

    Properties:

      Roles:

        - !Ref MyInstanceRole

Check the BeanStalk Env and application these should be created and you will be able to access the application on browser.

When you delete this cloudformation, the EC2 also got deleted!

AWS Management & Governance - Cloudwatch

AWS CloudWatch Log Docs

AWS Cloudwatch is a monitoring service on the basis of metrics.

by default, when you create an EC2 instance, there will be a default monitoring at one of the tabs, but if you want to have your own custom monitoring (like using alarms or anything extra) you can use AWS Cloudwatch.

States of Alarm

In Alarm: - If the metric value is greater than a threshold value. E.g if you have set the threshold value of CPU utilization is > 50 and if EC2 instance CPU usage is > 50 then it is In Alarm State.

OK: If the metric value is less than a threshold value

Insufficient data: If the data which is collected by CloudWatch is not sufficient to determine the alarm then it is in an insufficient state.

EC2 Actions: These actions are performed on selected EC2 instances if an Alarm is triggered.

• Stop 
• Terminate 
• Reboot

Ensure you have the IAM role: CloudWatchFullAccess.

To create a CloudWatch Alarm:

Step 1: Select CloudWatch Service.

Step 2: Click on Create Alarm Button

Step 3: Click on Metrics

Step 4: Select EC2 instance because we are monitoring EC2 instance.

Step 5: Click on Per instance Metrics

Step 6: Copy Instance Id in the search box

Step 7: Select Instance Id and Metric as CPU Utilization.

Step 8 Click on Select Metric Button.

Step 9: Keep all the properties same except below

            Threshold Type: Static

             Select : Greater Than (you can chose any option like >=,< etc)

            than : 50 ( I am selecting 50 so that I need to set the alarm if Cpu utilization is more than 50% of selected EC2 instance)

Click on Next Button.

Step 10 : Select In Alarm

Step 11 : Select SNS topic for notificaiton.

Step 12: Select EC2 Action and select Terminate Instance.

Step 13: Click on Next Button

Step 14: Give the Alarm Name and Click on Create Alarm Button.    

CloudWatch Log Monitoring

Step 1: Configure your IAM role or user for CloudWatch Logs

             Create a Role (Cloudwatchrole) for EC2 instance

             Attach CloudWatchFullAccess permission

Step 2: Install and configure CloudWatch Logs on an existing Amazon EC2 instance

            Connect to EC2 instance and install the AWS Cloudwatch logs agent:

           >>> sudo yum update -y

           >>> sudo yum install -y awslogs

           If required change /etc/awslogs/awslogs.conf to monitor the file. change the log_group_name so that it will reflect in the aws console (easier to identify).

[/var/log/messages] datetime_format = %b %d %H:%M:%S file = /var/log/messages buffer_duration = 5000 log_stream_name = {instance_id} initial_position = start_of_file log_group_name = /var/log/messages-irfan

           By default, the /etc/awslogs/awscli.conf points to the us-east-1 Region. To push your logs to a different Region, edit the awscli.conf file and specify that Region.

            >>> sudo vi /etc/awslogs/awscli.conf

[plugins] cwlogs = cwlogs [default] region = ap-southeast-2

            >>> sudo systemctl start awslogsd

            >>> sudo systemctl enable awslogsd.service 

            >>> sudo systemctl status awslogsd

now go to cloudwatch, ENSURE YOU ARE IN THE RIGHT REGION, then check your logs.

AWS Management & Governance - System Session Manager (SSM)

SSM - AWS Documentation

Session Manager

It is a part of the System Manager service of AWS and it is used for patch management for AWS instances. Patching is installing/uninstalling/updating.

With Session Manager, you can manage your Amazon Elastic Compute Cloud (Amazon EC2) instances, edge devices, and on-premises servers and virtual machines (VMs).

To access SSM terminal:

Step 1: Create a role for EC2 service with permission AmazonSSMManagedInstanceCore

Step 2: Attach the above role to the EC2 instance that you want to patch. (ensure that you have detach other roles from the EC2).

Step 3: Select System Manager Service.

Step 4: On the left panel select the Session Manager link. (under Node Management).

Step 5: Click on Start Session button. (if EC2 is not found, make sure all traffic in the EC2 inbound and then reboot the EC2).

Step 6: You will see your instance here which is attached with Role. If you do not see in couple of minutes, restart/reboot the EC2 instance.

Step 7: Select EC2 instance in session Manager and Click on Start Session button. It will open a terminal. Check that httpd is installed or not.

>>> sudo systemctl status httpd 

To install apache via Run Command:

Step 1: Select Run Command from the left panel link and Click on Run Command button.

Step 2: Search for shell and Select AWS-RunShellScript

Step 3: Add below commands in the Run Command window

>>> sudo yum install httpd -y

>>> sudo systemctl start httpd

>>> sudo systemctl enable httpd

Step 4: Type in any comment (it is like commit).

Step 5: Select you EC2 instance. and Run the command

Step 6: Via the terminal above, check if httpd is installed on your EC2 instance or not.

ASSIGNMENT: AWS Management - IAM

The developer is developing index.html file , implementer is hosting that file on S3 and Tester is testing that file

1. Create groups called Developer  and Tester and implementer

Ans: Ok

2. create some users in Developer and Tester groups

Ans: Ok

3. Developers and Tester has readonly access to S3 buckets and Implementer has full access to S3 buckets

Ans: Developers - AmazonS3ReadOnly, Testers - AmazonS3FullAccess

4. Developers can access the S3 buckets using programs.

Ans: Developers - Programmatic Only (Share only Username and Password)

5. Testers can not access the S3 buckets using progs but there are some dedicated machines allocated to tester for Testing purpose so that they can access S3 buckets. 

Ans: Testers - Console only (Share only Access Key and Secret Key)

AWS Storage - S3 Lifecycle Management

 

THIS IS CONTINUATION OF S3 STORAGE CLASSES & LIFECYCLE. PLS CHECK THAT OUT FIRST BEFORE YOU CONTINUE THIS ARTICLE.

Management for the Lifecycle of S3 bucket objects depends upon their requirement like frequent access or Infrequently accessing or archiving the data.

Prerequisite: Bucket should be already created in a specified region.

Create Lifecycle Rule (to manage the costs and resources needed effectively):
Step 1: Select Management Tab in the bucket .
Step 2: Click on create Lifecycle Rule
Step 3: Provide the below information about lifecycle rule.

• Life Cycle Rule name: lcr1
• Choose Rule Scope: Apply to all objects in the bucket
• Life Rule Action: select below option
• Move current versions of objects between storage classes
• Define the transition actions
• Standard IA---> 30 days
• Intelligent Tiering ---> 60 days
• One Zone ---> 90 days
• Glacier:--->180 days
• Deep Archive:---> 365 days

Step 4: Click on Create Rule button and Rule should be created and Should be enabled.