AWS Networking & Content Delivery - VPC Network Access Control List

 

Network ACL

A network access control list (ACL) is an optional layer of security for your VPC that acts as a firewall for controlling traffic in and out of one or more subnets. You might set up network ACLs with rules similar to your security groups in order to add an additional layer of security to your VPC.

Network Access Lists - AWS Official Documentation

Note:

One Subnet can be associated with one and only one NACL but One NACL can have multiple subnet associated with it.

To create a custom NACL:

1. Select Security --->Network ACLs

2. You can see there is already a NACL defined for your subnets (Public and Private).

3. Click on create Network ACL button

4. Provide the following settings

    NACL Name: mynacl

    VPC: myvpc1

 Click on Create Network ACL

5. select mynacl and click on Subnet Association Tab and select public subnet.

6. Click on Inbound Rules and Click on Edit InBound Rules and Allow access to port number 22.

7. Create an Ec2 instance in myvpc-1 and subnet public subnet and try to connect with port number 22.

You will not able to connect because on NACL level we have opened port 22 for Inbound level not for Outbound level, so let's open port number 22 for outbound level as well. (I am opening ALL Traffic because we are not sure that outbound is port number 22 or something else port number)

8. Select mynacl and click on outbound rules tab and Edit Outbound rule by Allowing All Traffic.

Experiments

1. Host a website on EC2 instance on port number 80 and remove outbound rule and check you are able to access it externally or not.

2. Try to access google.com on your Ec2 instance.

AWS Networking & Content Delivery: Access AWS Services using AWS CLI

 

AWS CLI

step1: Install AWS CLI on local system

step2: verify if AWS CLI is installed or not

aws –version

(it is python dependent)

step3: Configure AWS on your local system

aws configure

(it will ask for access key, secret key, region name)

step4: If user has S3 full access then run command below to check that with AWS CLI, you can access the S3 services

aws s3 ls

AWS Networking & Content Delivery - Cloudfront CDN

 

AWS Edge locations are data centers closer to Regions and their purpose is different from Availability Zone(AZ) Data Centers. These Edge Locations are for Content Delivery Network (CDN). Like in AZ you can create an EC2 instance but when we are using CDN service from AWS then it automatically decides the nearest Edge location on behalf of us. Cloudfront is the service meant for selecting edge locations to keep data (temporarily/on cache) such as media files, API, images, etc with the lowest latency and high throughput.

1. Cloudfront caches and delivers dynamic content to end-users.
2. It is a global service and NOT region-specific.
3. You can define the Allow/Deny list of countries for Cloudfront distribution.
4. It supports IPV6.
5. You can enable the logging (s3 bucket).
6. Whatever you put after configuring the CDN, it will automatically push to edge location data centers.

How it works:

Step 0: Create a file index.html in the S3 bucket (srcbucket) and provide public access to this file.

Step 1: Search Service CloudFront or Goto Networking and Content Delivery --> CloudFront

Step 2: Click on Create Distribution button.

Step 3: Define below properties

         Origin Domain: S3 bucket name(srcbucket)

         Origin Path: No change because of index.html file is not in any folder of s3 bucket

           Other properties are default no change.

          Price class:- Edge location with best performance 

Step 4: Click on Create distribution.

ASSIGNMENTS: Docker Network Part 1 (Using Custom Network)

 

Challenge: Create a bridge network and run a web app container which stores the data in a database container.

Docker images:
webapp: ramansharma95/webapp
db: ramansharma95/mysql

webapp: This image is used as a webserver container which runs on apache. There is an index.php file hosted on this server in this directory (/var/www/html/index.php) and it is a simple webform to enter details in the database container.

db: This image is used as database container which runs on mysql and stores the data recorded/collected/received from the webserver - webapp. This database container's details should be a part of webapp's connection string.

For webapp container to be connected and functional, we need to create a database called company and a table in the company database called employee which has name, mobile fields IN the db container.

ASSIGNMENTS: Docker Network Part 3 (None Network)

 

None Network

When no IP address is assigned to the container you can run the container in none network. It is mostly used for applications that need to be tested in an isolated environment. After testing, we can disconnect the container from the network and connect it to another network.


Step 1: Create a centos container on none or null network

docker container run -it --name c1 --network none -d centos

Step 2: Inspect the container and verify that it is running on none networks

docker inspect c1

Step 3: Once the testing is done then remove none network from n1 container and attach bridge network

docker network disconnect none c1

docker network connect bridge c1

Step 4: Verify the n1 container is having bridge network

docker inspect c1

Docker Networking

Summary:

1) Bridge Network: Default & Custom

2) Host Network

3) None/Null Network

DOCKER NETWORKING 
In Docker, if 2 containers communicate to each other, it means they are in a network.

Do note that unlike centos, ubuntu does not have ping command in-built. so you need to install it in the containers themselves:

apt-get update

apt-get install iputils-ping  # 333kb 

 

To look at available docker network commands:

Command:-> docker network --help

connect	Connect a container to a network
create	Create a network
disconnect	Disconnect a container from a network
inspect	Display detailed information on one or more networks
ls	List networks
prune	Remove all unused networks
rm	Remove one or more networks

To find all the IP addresses on a system

Command:-> ip a

Find all the networks in docker

Command:-> docker network ls

Types of Networks in Docker

1) Bridge

1a) Default Bridge Network

It is the default network (docker0) in docker, which means if a container is created by default it is created on top of bridge network docker0.

If 2 or more containers get created on the bridge network then they are automatically in the same network, which means they can communicate with each other.

Step 1:- Create container c1 with ubuntu image

docker container run -it --name c1 -d ubuntu

Step 2:- Check c1 container is running on the docker0 network

ip a

you will find one veth.... on docker0

Also, check the CIDR for docker0

Step 3:- Check the IP address of the container, it should be in the CIDR range of the docker0 network. In my case, IP address of c2 is "172.17.0.2"

docker container inspect c1

Step 4:- Create container c2 with centos image

docker container run -it --name c2 -d centos

Step 5:- Check c2 container is running on the docker0 network

ip a

you will find one more veth.... on docker0

Also, check the CIDR for docker0

Step 6:- Check the IP address of the container, it should be in the CIDR range of the docker0 network. In my case IP address of c2 is "172.17.0.3"

docker container inspect c2

Step 7:- Check container c2 ping to c1. It should get a reply from c1 because both are in the same network (default bridge network)

docker exec -it c2 bash

ping 172.17.0.2

 

1b) Custom Bridge Network or User Define Bridge Network

When a network which is created by user or sysadmin so that specified containers can run on it then it is a custom bridge network.

To make your own custom bridge network, follow the steps below:
Step 1: List all the containers

docker network ls

Step 2: Create a new default Bridge network br1 (-d is drive)

docker network create -d bridge br1

Step 3: Verify network is created successfully. It should be listed in docker networks.

docker network ls

Step 4: Inspect br1 network to find more detailed information.

docker network inspect br1

•         "Driver": "bridge",
•         "EnableIPv6": false,
•         "IPAM": {
•             "Driver": "default",
•             "Options": {},
•             "Config": [
•                 {
•                     "Subnet": "172.18.0.0/16",
•                     "Gateway": "172.18.0.1"

Step 4.2: You can create a bridge network with your own subnet

docker network create -d bridge --subnet=192.168.0.0/16 --gateway=192.168.0.1 br2

docker inspect br2

•         "Driver": "bridge",
•         "EnableIPv6": false,
•         "IPAM": {
•             "Driver": "default",
•             "Options": {},
•             "Config": [
•                 {
•                     "Subnet": "192.168.0.0/16",
•                     "Gateway": "192.168.0.1"
•                 }

It means if the containers get created on this network layer then they have IP addresses in 192.168.0.1/16 range.

Step 6:- Create a container on top of br1 network on ubuntu base image.

docker container run -it --name c1 --network br1 -d ubuntu

Step 7: Inspect the container and it should have IP address within the br1 CIDR range.

docker container inspect c1

Step 8: Remove the container and network.

docker rm -f c1

docker network rm br1

docker network ls

Kubernetes Cluster Part 1: Creating a single Control-Plane cluster with kubeadm & creating a Calico Pod Network

PART 1: Creating a single Control-Plane cluster with kubeadm and creating a Calico Pod Network


kubeadm Tool:

The kubeadm tool helps you bootstrap a minimum viable Kubernetes cluster that conforms to best practices.

The kubeadm tool is good if you need:

• A simple way for you to try out Kubernetes, possibly for the first time.
• A way for existing users to automate setting up a cluster and test their application.
• A building block in other ecosystem and/or installer tools with a larger scope.

Before you begin, to follow this guide, you need:

1. One or more machines running a deb/rpm-compatible Linux OS; for example: Ubuntu or CentOS.
2. 2 GB or more of RAM per machine--any less leaves little room for your apps.
3. At least 2 CPUs on the machine that you use as a control-plane node.
4. Full network connectivity among all machines in the cluster. You can use either a public or a private network.
5. Check required ports. (if internal network, do not need to configure ports)
6. Control-plane node(s).

Installing runtime
By default, Kubernetes uses the Container Runtime Interface (CRI) to interface with your chosen container runtime.
If you don't specify a runtime, kubeadm automatically tries to detect an installed container runtime by scanning through a list of well known Unix domain sockets.

Runtime	Path to Unix domain socket
Docker	/var/run/docker.sock
Containerd	/run/containerd/containerd.sock
CRI-O	/var/run/crio/crio.sock

If both Docker and containerd are detected, Docker takes precedence. This is needed because Docker 18.09 ships with containerd and both are detectable even if you only installed Docker. If any other two or more runtimes are detected, kubeadm exits with an error.

Installing kubeadm, kubelet and kubectl

· kubeadm: 

the command to bootstrap the cluster.

· kubelet: 

the component that runs on all of the machines in your cluster and does things like starting pods and containers.

· kubectl: 

the command line utility to talk to your cluster.


Infrastructure
Example: 3 Virtual Machines (1 Master Node and 2 Worker Nodes). There must be network connectivity among these VMs.


Installation of kubeadm on Ubuntu (Both on Master and Worker Nodes)

sudo apt-get update && sudo apt-get install -y apt-transport-https curl

curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

cat <<EOF | sudo tee /etc/apt/sources.list.d/kubernetes.list
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

sudo apt-get update

apt install -qq -y kubeadm=1.21.0-00 kubelet=1.21.0-00 kubectl=1.21.0-00

sudo apt-mark hold kubelet kubeadm kubectl



OPTIONAL
So in particular case docker was using the groupfs which I changed to systemd

Create the file as:

vim /etc/docker/daemon.json

{ 

     "exec-opts": ["native.cgroupdriver=systemd"] 

}

systemctl restart docker 

systemctl status docker



Installation of kubeadm on RHEL/CentOS (Both on Master and Worker Nodes)
In case if you are using CentOS/RHEL

cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://packages.cloud.google.com/yum/repos/kubernetes-el7-\$basearch
enabled=1
gpgcheck=1
repo_gpgcheck=1
gpgkey=https://packages.cloud.google.com/yum/doc/yum-key.gpg \
https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg
exclude=kubelet kubeadm kubectl
EOF

# Set SELinux in permissive mode (effectively disabling it)

sudo setenforce 0
sudo sed -i 's/^SELINUX=enforcing$/SELINUX=permissive/' /etc/selinux/config
sudo yum install -y kubelet kubeadm kubectl --disableexcludes=kubernetes
sudo systemctl enable --now kubelet



Create The Master Server/Control-Plane & Calico Pod Network
On master machine run the below command

1. initialize kubeadm and dont forget to change the master server IP! with a CIDR range which you can define yourself.

     kubeadm init --apiserver-advertise-address=<<Master ServerIP>> --pod-network-cidr=192.168.0.0/16

        
2. mkdir -p $HOME/.kube

3. sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config

4. sudo chown $(id -u):$(id -g) $HOME/.kube/config

5. kubeadm token create --print-join-command (Copy the token from master to agent/worker nodes (remember to swapoff -a)) 

5.2 Alternatively if you want to auto-off swapoff, you can type this:

    sudo sed -i '/ swap / s/^\(.*\)$/#\1/g' /etc/fstab

6. Install Calico
6.1 kubectl create -f https://docs.projectcalico.org/v3.18/manifests/calico.yaml
6.2 kubectl get nodes (Wait for above command and run again it may take a minute or so to get all the nodes in ready state.)


OPTIONAL (Do not run below commands if Calico is configured properly by following the above steps)

Install the Tigera Calico operator and custom resource definitions.
kubectl create -f https://docs.projectcalico.org/manifests/tigera-operator.yaml

Install Calico by creating the necessary custom resource. For more information on configuration options available in this manifest, see the installation reference.
kubectl create -f https://docs.projectcalico.org/manifests/custom-resources.yaml

Note: Before creating this manifest, read its contents and make sure its settings are correct for your environment. For example, you may need to change the default IP pool CIDR to match your pod network CIDR.

Confirm that all of the pods are running with the following command.
watch kubectl get pods -n calico-system

Wait until each pod has the STATUS of Running.

Note: The Tigera operator installs resources in the calico-system namespace. Other install methods may use the kube-system namespace instead.

Remove the taints on the master so that you can schedule pods on it.
kubectl taint nodes --all node-role.kubernetes.io/master-

It should return the following.
node/<your-hostname> untainted

Confirm that you now have a node in your cluster with the following command.
kubectl get nodes -o wide

It should return something like the following.
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME <your-hostname> Ready master 52m v1.12.2 10.128.0.28 <none> Ubuntu 18.

EXTRA INFOS: Networking Homework

 

Homework by Raman (JUMP IBM PROGRAM) 1 March 2022:

1. what is networking between computers?
2. what is client server networking?
3. what is CIDR range?
4. Check an IP exist in that CIDR range.

Answers:

1. what is networking between computers?

A computer network comprises two or more computers that are connected—either by cables (wired) or WiFi (wireless)—with the purpose of transmitting, exchanging, or sharing data and resources. You build a computer network using hardware (e.g., routers, switches, access points, and cables) and software (e.g., operating systems or business applications).

Geographic location often defines a computer network. For example, a LAN (local area network) connects computers in a defined physical space, like an office building, whereas a WAN (wide area network) can connect computers across continents. The internet is the largest example of a WAN, connecting billions of computers worldwide.

types of network:

LAN - usually in a building

You can further define a computer network by the protocols it uses to communicate, the physical arrangement of its components, how it controls traffic, and its purpose.

Computer networks enable communication for every business, entertainment, and research purpose. The internet, online search, email, audio and video sharing, online commerce, live-streaming, and social networks all exist because of computer networks.

2. what is client server networking?

In a client/server network, a central server or group of servers manage resources and deliver services to client devices in the network. The clients in the network communicate with other clients through the server. Unlike the P2P model, clients in a client/server architecture don’t share their resources. This architecture type is sometimes called a tiered model because it's designed with multiple levels or tiers.

3. what is CIDR range?

Use Classless Inter-Domain Routing (CIDR) notation in the format <IPv4_address>/number, such as 10.10.0.0/16. Reserve the last 16 bits (65,536 addresses) of the IPv4 as 0s so that you can use them for various subnet IP addresses(within the same CIDR range) within the same IBM Cloud® VPC, such as 10.10.1.0/24

If you use an IP range outside of the ranges RFC 1918 defines (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16) for a subnet, the instances attached to that subnet might be unable to reach parts of the public internet. If you plan to configure VPCs that use both non-RFC-1918 addresses and also have public connectivity (floating IPs or public gateways), make sure to use a custom route that contains the Delegate-VPC action.

4. Check an IP exist in that CIDR range.

What comes after the slash determines the quad notation

Dotted quad notation

The smaller the number after the slash, the more IP addresses that you are allocating. The number after the slash represents the number of leading 1 bits in the subnet's prefix mask.

ASSIGNMENTS: Docker Network Part 2 (Using Host Network)

Step 1: Download required images

docker pull ramansharma95/webapp
docker pull ramansharma95/mysql

Step 2: Create a webapp container with network as host and DO NOT DO a port forwarding on port number 80 as per custom network cause this assignment is run on host network.

docker container run -it --name web --network host -d ramansharma95/webapp

Go to browser and check that you are able to see the default web page. localhost:80

Step 3: Go inside the container (web) and check the code for index.php cause you will need to edit it a little.

docker exec -it web bash

vi /var/www/html/index.php

edit the username from db to host ip address (192.168.33.10).

# you will find that in the index.php, it has connection configs to connect to a server named 'db' but need to edit to host ip address cause its on the same network and also the required username and password to enter that server.

Step 4:- Create a db container with network defined (note this does not require port forwarding)

docker container run -it --name db --network host -d ramansharma95/mysql

Step 5:- Go inside the container (db).

docker exec -it db bash

5.2 connect to mysql with username root and password whizlabs

mysql -uroot -pwhizlabs

5.3 Create a database company

show databases;

create database company;

show databases;

5.4 Create a table employee with name and mobile field.

use company;

create table employee ( name varchar(30), mobile varchar(30) );

5.5 Show all the records in this table

select * from employee;

Step 6: Go to browser and add some employees details in the webpage and check again records in employee table, it should have those records added.

Step 7: Show all the records in this table.

select * from employee;

NIC: Network Interface Card

NIC : Network Interface Card! 

every system has it and with it there is an IP address (Loopback IP). 

there is name resolution (you can use 127.0.0.1 or localhost or me) 

you can check it on vm:
cat /etc/hosts 

or 

if on windows: 

C:\Windows\System32\drivers\etc\hosts