Computation

Multitenancy: Sharing underlying hardware between virtual machine

AMI AMAZON MACHINE IMAGE

Machine Image Linux/ Window etc can be used to setup OS in EC2

• EC2 instance launched using existing AMI
• EC2 instance can be converted into AMI

AMI can be

• Public: AWS Provided
• Own Custom: Maintained & Owned by Customer
• Marketplace

AMI Boundary

• AMI are Built for Specific Region
• The AMI must be in the same region as that of the EC2 instance to be launched.
• If the AMI exists in a different region, you can copy that AMI to the region where you want to launch the EC2 instance
• The region of AMI has no bearing on the performance of the EC2 instance.

EC2 Image Builder

Automatically creation, maintain, validate & test EC2 AMI Images

• Simplify process of AMI creation
• Can be scheduled
• Free service but you need to pay for Ec2 instance creation

ELASTIC COMPUTE CLOUD (EC2)

Server that help to communicate with Servers on Amazon Data Center

• ES2 Instance spin off a virtual machine in AWS Server
• User can control what run on top of OS

EC2 Configuration:

• OS: Windows, Linux OS, MacOS
• CPU:
• RAM
• HDD: Instance Storage

  On termination of the instance, the default behavior of EBS Root Volume is to terminate the attached root volume

• Network Storage: EFS, EBS
• Firewall: Security Group
• Network: public IP Address, VPC
• Bootstrap Script

CPU type can be vertically scaled in EBS backed EC2 instance.

• Stop instance
• Right click->Instance Settings->change instance type
• Launch Instance

Types of EC2 = Instance Family

Visit : https://aws.amazon.com/ec2/instance-types/

Overview: https://instances.vantage.sh/

• 1. General Purpose:

  • Web Service, Code Repository

• 2. Compute Optimized:

  • Gaming, Scientific modelling, HPC(High Performance Computing), Batch processing, Media Transcoding

• 3. Accelerate Computing:

  • Floating Point, Graphics, Date pattern matching

• 4. Memory Optimized::

  • Redis DB, in Memory DB, Elastic Cache, Big Unstructured Data processing

• 5. Storage Optimized:

  • Locally Stored Data, Data Warehouse, High Frequency Online Transaction Processing, Data Warehouse, DB

Connect with EC2 Instance

Use Direct Connect or connect using SSH with public IP of EC2 Instance

• chmod 0400 Hassium.pem

• ssh -i Hassium.pem ec2-user@52.59.11.125

Speed up EC2

1. Golden AMI

Install all STATIC dependencies in AMI for future EC2 instances

2. Bootstrap Script/ EC2 User Data

Script run once on start with sudo root access to configure EC2 Instance to add DYNAMIC dependencies.

Can do tasks like:

• Install Update
• Install Software
• Download file
• Anything we want

3. Hybrid

Mix of User Data & golden AMI

4. Snapshot for DB & Volume

use snapshots to speed up booting with snapshot & Volume

Better Networking Performance

EC2 Enhanced Networking SR IOV

Higher bandwidth & PPS (packet per second), lower latency

1. Elastic Network Adapter(ENA) :

• upto 100Gbps

Elastic IP Address

Fix Public IP address which can be assigned to EC2 Instance from the pool of AWS Public IP

• We can have up to 5 IP Address by default
• Elastic IP are Charged when not in use
• Public & Private IP Of EC2 changes when we restart the EC2 instance. Elastic Ip stays the same
• Use Case: - EIP can be attached to any EC2 instance dynamically which help redirection and mask failure

Elastic Network Interface

Virtual Network Card for EC2 but can be created independently & attache to EC2 Instance on the fly

• Bound to AVZ
• Attached with EC2 Instance to provide
  • 1 primary private IPV4
  • One or more secondary IPV4
  • Public IPV4
  • One or More Security group
  • MAC Address

2. Intel VF 82599: 10Gbps- LEGACY

Elastic Fabric Adapter(EFA)

improved ENA for HPC

• Work only with Linux HPC Cluster
• Leverage MPI Message Passing Interface Standard
• Bypass underlying Linux OS to provide low latency reliable transport
• Great for inter node communication in tightly coupled workloads

EC2 Instance Meta data

Info about EC2 instance available form EC2 Instance at http://169.254.169.254/latest/mata-data

• Let EC2 explore their meta data
• You can retrieve the IAM role name attached to your EC2 instance using the Instance Metadata service, but you can not retrieve the IAM policies themselves.

---

SECURITY GROUP

• Stateful
• Firewall around EC2 Instance
• Many to Many: Security group can be attached to multiple EC2 instance. EC2 can have multiple Security group
• Locked to region/ VPC
• By Default Block all Inbound requests and Allow all Outbound traffic
• Security rules must modify to allow specific type of traffic eg HTTPS
• Regulate:
  • Port Access
  • Authorize IP Range(IPV4, IPV6)
  • Protocol

PORT	Protocol	Usage
22	SSH(Secure Shell)	Log into Linux Instance
22	SFTP(Secure File Transfer Protocol)	Upload File using SSH
21	FTP(File Transfer Protocol)	Upload file into file share
80	HTTP	Unsecure Site
443	HTTPS	Secure Site
3389	RDS(Remote Desktop Protocol)	Log Into Window Instance

Shutdown Behavior

Shutdown by OS using $shutdown command can result into:

1. Stop (Default)
2. Terminate

State is defined by CLI attribute: InstanceInitiatedShutdownBehaviour

Termination Protection

Protect against accidental termination in AWS console or CLI

• If ShutdownBehaviour = Terminate and OS shutdown from OS -> EC2 will terminate because its from OS

EC2 Hibernate

Store RAM of EC2 to root EBS storage for fast boot of EC2

• EC2 does not terminate
• Limitation:
  • Root EBS Volume must be Encrypted
  • RAM size must be less than 150 GB
  • Available for on Demand & Reserved Instance
  • Time limited to 60 Days

Pricing Model

• One-minute minimum charge for Linux based EC2 instances

• 1. On Demand:

  Pay as you go, no upfront payment

• 2. Saving Plan:

  Consistent Usage cost per year for long time

  • EC2 : 72% discount
  • Compute Saving plan: uses machine learning to recommend optimal AWS resources and therefore reduces costs. 66% Saving Plan

• 3. Reserved Instances:

  75 % Billing discount to on demand instance.

  • 1-3 Year commitment & give big discount.
  • Reservation Period: 1Year= +discount | 3 year +++ discount
  • Purchase option: no upfront | partial upfront = +discount | All upfront ++discount
  • Types
    • 1.Reserved Instance

    Long Workload eg

    • 2. Convertible Reserved Instance:

    Flexible instances allow changing instance type: 54% discount

    • 3. Scheduled Reserved:

    need instance withing a time window over along interval

• 4. Spot Instances:

  Make available computation power for temporary on demand usage

  • 2 minute warning
  • Up to 90% discount
  • Spot price changes over time
  • Use case image processing, Batch job, distributed workload

  

  Cancel all spot request and then terminate spot instance because request will launch spot instance

  

5. Spot Fleet:

Set of Spot Instance + On Demand Instance (Optional) Spot fleet will try to meet target capacity within price constrain by launching spot instance based on

• Lowest price: short workload

• Diversify: distributed across pool for high Availability & long workload

• Capacity Optimized: Optimal Capacity for number of instance

• 6. Dedicated Host:

  For compliance requirement for software license

  • allow per core or per cpu renting of physical server
  • Can be reserved for 3 year
  • More expensive
  • Give access to underlying hardware

• 7. Dedicated Instances

Dedicated EC2 instance

• Soft version of Dedicated hosts
• Per instance billing
• Don't get access to underlying hardware

EC2 Troubleshooting

• InstanceLimitExceeded: reached max onDemand CPU /Region Limit = 64 vCPU by default for onDemand & Spot Instances
  • Launch EC2 in different region of request to increase limit
  • Can be seen in Limit in EC2 or Service Quota
• InsufficientInstanceCapacity : AWS don't have enough on demand capacity in AVZ
  • Wait for capacity to increase or make request smaller or change instance type
  • Launch EC2 in different EC2
• EC2 terminate immediately from Pending state:
  • Root EBS is corrupt
  • Don't have permission to decrypt root EBS
  • Reached EBS Volume limit
  • EBS Snapshot corrupt

EC2 Instance Status Checks

1. SYSTEM status checks

monitor the AWS systems on which your instance runs

• Monitor Problem with the underlying host:
  • Loss of network connectivity
  • Loss of system power
  • Software issues on the physical host
  • Hardware issues on the physical host that impact network reachability
  • Either wait for AWS to fix the host, OR
• Troubleshoot
  • Move the EC2 instance to a new host = STOP & START the instance (if EBS backed)

2. INSTANCE status checks

monitor the software and network configuration of your individual instance

• Monitor problem in EC2:

  • Incorrect networking or startup configuration
  • Exhausted memory
  • Corrupted file system
  • Incompatible kernel
  • Requires your involvement to fix

• Troubleshoot

  • Restart the EC2 instance, OR
  • Change the EC2 instance configuration

---

ELASTIC LOAD BALANCER(ELB) Service

Load Balancer

Severs that direct traffic to different Servers(EC2) based on work load

• Work on Region level in multiple AVZ
• Managed & upgraded by AWS.
• Works With:
  • EC2, ASG, ECS,
  • ACM(Certificate manger), Cloud Watch,
  • Route53, WAF, Global Accelerator

USAGE:

Expose Static DNS as single point of access to App

Each EC2 instance have own public IP. With Load Balancer we can use static DNS name to access EC2 instances.

• Hide private traffic from Public Internet

Hide EC2 Instance from Internet

Both the EC2 instances and the ALB are deployed on a VPC with the following CIDR 192.168.0.0/18.

• Security Group of ELB allow inbound traffic from anywhere 0.0.0.0/0 for HTTP & HTTPS request
• EC2 only allow traffic from ELB
• Configure the EC2 instances' to allow inbound traffic from security group of ALB at port 80.

Perform Health check on EC2 Instances

Switch EC2 Instances dynamically based on load and health of EC2 instance

Provide HTTPS end point

ACM (AWS Certificate Manger) & SNI(Server Name Indication)

SNI (Server Name Indication)

List of certificate to supported by list of host name to create SSL connection to ELB

• Load multiple Certificate to one web server
• Works with ALB, NLB to support multiple Certificate for Multiple Listeners
• Supports Cloudfront
• Does not support CLB

Cross Zone Load Balancing

Balance load equally across all instances across all AVZ

• When Cross-Zone Load Balancing is enabled, ELB distributes traffic evenly across all registered EC2 instances in all AZs.

Health Check 🩺

When you enable ELB Health Checks, ELB won't send traffic to unhealthy (crashed) EC2 instances.

• ALB perform Health Checks on a port & route(/health: commonly used)
• If response is 200 Instance is okay
• If response not 200, it is marked unhealthy and will be terminated while the ASG launches a new EC2 instance

Elastic Load Balancer types:

	CLB	ALB	NLB	GLB
Version	V1(Legacy deprecated))	V2	V2
Protocol	TCP, HTTP, HTTPS	HTTP/HTTPS, WebSocket	TCP, TLS(Secure TCP), UDP	GENEVE on port 6081
CrossZone	Off Free IAVZ	always On Free IAVZ	Off paid IAVZ
Layer	4&7	7	4	3
SSL	1 SSL/CLB	Many/ALB	Many/NLB
SNI	NO	YES	YES
Static	HostName	Host Name	HostName + 1 IP/AVZ(EIP)
Usage	Retired	Microservice	High Performance Gaming	FireWall

1. Classic Load Balancer(v1)

• Retiring and divided into NLB & ALB
• Both Layer 4 & 7: TCP, HTTP, HTTPS
• Cross Zone Load Balancing: Off by default(Free inter AZ if enabled)
• 📜 SSL: Support only one SSL Certificate per ELB
  • 1 CLB support only 1 App & 1 SSL Certificate
  • Does not support SNI to support multiple SSL certificate.
• 📌 static DNS Host Name name, No static IP.
• 🩺 Health checkup : TCP or HTTP based

Limitations:

• Multiple CLB needed for multiple Application
• Does not support SNI: Multiple CLB need to for multiple SSL Certificate

2. Application Load Balancer ALB(v2)

Route traffic to multiple HTTP/S applications across machines(target group)

• Great fit for Microservices & docker based application ECS
• High Latency: ~400mS(4X NLB)
• Layer 7 : HTTP/HTTPS, WebSocket
• Cross Zone Load Balancing: Always On(Cant be disabled & Free inter AVZ)
• 📜 SSL: Use SNI to support multiple SSL Certificate with multiple Listener
• 📌 static DNS Host Name name, No static IP.

Application server communicate with ALB using private IPV4

Application server don't see the client IP directly & clients IP info is embedded into header of request from ALB:

• X-Forwarded-Proto: Client Protocol

• X-Forwarded-For: IP

• X-Forwarded-Port: Port

• ALB targets with instance ID route to primary private IP in primary NIC targets using IP addresses route to any private IP from one or more NICs.

ALB Routing

Target Group

Group of EC2 instance with Health check

• Support port mapping to redirect to dynamic port on EC2 instance.
• Each target group can be an independent Microservice
• Can configure listener rules to route requests to different target groups based on the content of the application traffic.

ALB Can route multiple URL to Multiple Target group and health check can be done on target group level

Supported Target group by ALB

• EC2 Instances
• ECS Tasks
• Lambda Functions
• Private IP Addresses

Routing Parameters

• Source IP
• Host name:
  • Eg: home.amazon.com, user.amazon.com
• Request URL Path
  • Eg. user.amazon.com/order, user.amazon.com/address
• Query String & Header
  • Eg. user.amazon.com/user?id=123 && login=true

3. Network Load Balancer NLB (v2)

• Ulta High Performance: Gaming, Millions of request
• Low latency ~100mS
• Layer 4: TCP, TLS(Secure TCP), UDP
• Cross Zone Load Balancing: Off by default(Paid if Enabled inter AVZ 💰)
• 📜 SSL: Use SNI to support multiple SSl Certificate with multiple Listener
• 📌 Provide Static DNS name & 1 Fixed Static IP per AVZ.
  • Support Assigning Elastic IP

Supported Target group

• EC2 Instances
• Private IP Addresses
• Application Load Balancer

4. Gateway Load Balancer

• Provides Single Entry Exit along with Load Balancing at low level IP Packet Level
• Balance Load with Third party Virtual Appliances
• Layer 3: IP Packets
• Use GENEVE Protocol on port 6081

Supported Target group

• EC2 Instances
• Private IP Addresses

Use Case: Useful to create a firewall, Intrusion detection, filtering traffic before reaching Application:

Sticky Session(Session Affinity)

Route client traffic to same EC2 instance

• Make use of Cookie to route traffic using ELB with an expiration date
• Works with CLB & ALB
• Attached to Target Group
• Use case: Session management, Login Management

Cookie Types:

Application Based Cookie

Application Generated Cookie

• Custom Cookie

  • Custom Cookie generated by application
  • Can include any custom cookie attribute
  • Cookie name must be unique to each target group
  • Cookie name must not be: AWSALBAPP , AWSALB, AWSALBTG

• Application Cookie

  • Generated by ALB
  • Name: AWSALBAPP

Duration Based Cookie

Load Balancer Generated Cookie

• Name: AWSALB, AWSCLB
• Expire after some time defined be ALB

Connection Draining(CLB) / Deregistartion Delay(ALB,NLB)

Time to complete in flight request before deregister an EC2 instance

• Called as Deregistartion Delay for ALB, NLB & Connection Draining for CLB
• Waiting for existing connection request to complete
• ELB automatically route traffic to other instances

Draining period: Time allocated 1-3600 Sec(default 300S) to fullfil request

• Set low for short request
• Set high for long live request

---

AUTO SCALING GROUP (ASG)

Automatically Launch/ Terminate EC2 Instances based on load in Load Balancer

• Free of Cost, pay for resources used.
• Work With:-
  • EC2 Instance
  • EC2 Spot Fleet
  • ECS Services
  • Dynamo DB
  • Amazon Aurora

ASG Attributes:

• Load balancer Info: Terminate & launch new instance of unhealthy instance marked by ALB
• Scaling Policy
• Launch Configuration/ Template(Newer)

  Can provision capacity across multiple instance types using both On-Demand Instances and Spot Instances.

  • AMI+ Instance Type
  • EC2 User Data
  • EBS Volumes
  • Security Group
  • SSH Key Pair
• Network + Subnet Info
• Capacity:
  • Minimum: Instances launch immediately as group created
  • Desired/ Actual:(Default : Minimum)
  • Maximum- instances scale out as needed.

Scaling Policy

1. Manual Scaling:

Set Min, Max Manually

2. Dynamic Scaling:

Use Cloud Watch Matrices to define scaling policy

Dynamic Scaling Types:

2.1 Target Tracking Scaling

Keep cpu usage at max 40%

2.2 Simple/ Step Scaling

based on Cloud Watch Alarm:

• Availability: 40 % usage
• Cost: 70% usage
• Both: 50 % usage
• Custom Metric

2.3 Scheduled Scaling

Scale before a schedule based on know user pattern

2.4 Predictive Scaling

Use ML based on past traffic to scale automatically

Cool down Period

• Time allocated to wait after a scaling action for metric to get stabilize
• Default 300 seconds (5 minutes)

Default Termination Policy

• Kill instance with oldest launch template form AZ with most number of instance

ASG Life Cycle Hooks

Hooks can be used to troubleshoot of perform some action when instance get created or terminated

Placement Group

Placement strategy for EC2 instance in AWS Infrastructure

• Cluster:

  Spread in same Hardware, Same Rack in Same AVZ

  • Same Rack Same AZ
  • High risk: all EC2 fail at same time in AVZ
  • Lowest Latency: 10Gbps
  • Use case: Quick Big data & low latency app

• Spread:

  spread over different HW across different AZ to avoid failure

  • Different Rack in Different AZ
  • Limited to 7 Instance/group/AZ
  • Reduce risk of simultaneous failure
  • Use case: Highly available server app, Critical Application

• Partition:

  spread over partition on racks in different AZ

  • Different Racks across different AZ
  • Distributed across racks 7 Partition per AZ
  • 100s of EC2 per setup to give highly available app
  • Use case: Distributed Application, HBase, Apache Kafka

---