Mastering EC2: Unleashing the Power of Elastic Compute Cloud in AWS

Amazon Elastic Compute Cloud (EC2) is a popular web service offered by Amazon Web Services (AWS) that allows you to create and manage virtual machines, known as instances, in the AWS infrastructure. EC2 provides resizable and scalable computing capacity in the cloud

Instances: Instances are virtual machines in the cloud.

• EC2 (Elastic Compute Cloud) has transformed server provisioning.
• It provides virtual servers within minutes through a self-service model, revolutionizing information technology.

The Traditional Server Provisioning Process:

• In the past, getting a server involved a lengthy process of purchasing, delivery, installation, and network connection, taking months.
• Virtualization improved this process by eliminating the need for hardware orders, but manual processes still slowed down provisioning to weeks.

The Innovation of EC2:

• Amazon automated the provisioning process, removing the need for human intervention.
• This changed the perspective of the IT industry, raising expectations for resource availability and challenging traditional approaches.
• EC2’s fast provisioning and lack of time commitment attracted users from individuals to large organizations.

Unique Characteristics of EC2:

• EC2 uses virtualization to create virtual machines (instances) with operating systems.
• Unlike traditional virtualization, EC2 stores only necessary data when instances are not running, reducing costs.
• It supports two types of Amazon Machine Images (AMIs): S3-backed and EBS-backed, each with specific storage features.

Architecting and Managing EC2 Applications:

• Success with EC2 relies on understanding its unique characteristics and designing applications accordingly.

• Applications need to leverage EC2’s features to optimize performance and cost-effectiveness.

ADVANTAGES OF EC2

It enables users to quickly create, configure, and launch virtual servers known as EC2 instances to run applications and services.

• You can choose from a wide range of instance types based on your specific requirements, such as computing power, memory, and storage.
• EC2 instances can be easily scaled up or down based on demand, allowing you to pay only for the resources you need.
• You have full control over the instance, including the operating system and software configurations.
• EC2 instances can be accessed remotely, allowing you to connect to them and manage them over the internet.

AMAZON MACHINE IMAGES (AMIs)

When it comes to installing the operating system (OS) on your EC2 instance, the first step is to have a bootable image.

AMIs(Amazon Machine Images) are pre-configured templates with the operating system, applications, and software needed to launch an EC2 instance.

• AMIs are used as the foundation for launching EC2 instances.
• This streamlined process eliminates the need to manually install the OS and simplifies the setup of your EC2 instances.
• AWS provides a wide variety of pre-built AMIs, including different operating systems and software configurations.
• You can also create your own custom AMIs by starting with an existing AMI, modifying it as needed, and then saving it as a new image.
• AMIs can be shared with other AWS accounts, allowing you to collaborate with other users or distribute your custom configurations.
• When launching an EC2 instance, you can select the desired AMI and launch multiple instances from the same AMI if needed.

EC2 SIZING AND CONFIGURATION OPTIONS

• Operating Systems: EC2 supports a wide range of operating systems, including popular choices like Amazon Linux, Ubuntu, Windows Server, and more, allowing flexibility in choosing the desired OS.
• Instance Types: EC2 offers a wide range of instance types with varying compute, memory, storage, and networking capabilities to match specific workload requirements.
• Instance Families: Instances are grouped into families based on their intended use cases, such as general-purpose, memory-optimized, compute-optimized, storage-optimized, etc.
• Instance Sizes: Each instance family includes multiple size options with different resource allocations, allowing you to choose the right balance of CPU, memory, and storage for your application.
• Pricing Models: EC2 provides options for On-Demand Instances (pay-as-you-go), Reserved Instances (upfront payment for long-term usage), and Spot Instances (bid-based pricing) to optimize cost efficiency.
• Storage Options: EC2 instances offer various storage options, including instance store (ephemeral storage), Elastic Block Store (EBS) volumes, and Amazon S3 for object storage needs.
• Networking & Security: EC2 instances can be deployed within Virtual Private Cloud (VPC) environments, enabling customizable network configurations, subnet placement, security groups, and access control.
• Auto Scaling: EC2 Auto Scaling enables dynamic scaling of instances based on application demand, ensuring optimal performance and cost-efficiency by automatically adjusting the instance count.
• Load Balancing: Elastic Load Balancing (ELB) distributes incoming traffic across multiple instances, improving availability, scalability, and fault tolerance for your applications.
• Instance Metadata: EC2 provides instance metadata accessible from within the instance, allowing applications to retrieve valuable information about the instance’s configuration, networking, and attributes.

EC2 INSTANCE TYPES

• EC2 instances come in various types optimized for different use cases.
• AWS follows a naming convention like m5.2xlarge for instance types.

General Purpose Instances ( T, M,A):

The computation, memory, and networking resources in general-purpose instances are well-balanced. Common use cases for General Purpose Instances include hosting gaming servers, managing small databases, handling personal projects, etc. For instance, if you have an application with relatively equal computing, memory, and networking resource requirements, and optimization in any specific resource area is not crucial, a general-purpose instance can efficiently execute the program

Types:

• A1 — A1 instances are the ARM-based offering of EC2 instance types
• T2, T3 — x86 burstable instances. Burstable means that CPU performance can be increased when needed. The duration of performance exceeding baseline levels depends on the instance and Credits — special units accumulated when your instance operates at the base performance.
• T3a — Similar to T2 and T3 but based on the AMD processor architecture, while the previous instances use Intel Xeon architecture.
• M4, M5 — General-purpose instances, not burstable. M4 utilizes older Intel Xeon processors.
• M5n — Similar to M5 but with higher bandwidth.
• M5a — Instances powered by AMD processors.
• M6g — ARM-based instances of a new generation. Note that only this type has a “6” in the name. Generally, a higher number implies a newer platform.

Compute Optimized Instances(C) :

Compute-optimized instances are well-suited for applications demanding substantial computational power and benefit from high-performance CPUs. These instances can be utilized for various workloads, including web hosting, application deployment, and gaming servers, similar to general-purpose instances. They excel in supporting high-performance applications such as web servers and gaming servers.

Types:

C4, C5 — Main compute optimized instances, C5 uses newer hardware.

C5n — the same as C5, but with higher bandwidth.

Memory Optimized Instances (R, X, Z):

Memory-optimized instances are designed for workloads that involve processing large datasets in memory. In this context, ‘memory’ refers to RAM, which enables the concurrent execution of multiple tasks. The data stored in memory is utilized for executing central processing unit (CPU) tasks, loaded from storage to memory for efficient processing. This preloading process grants the CPU direct access to the computer program.

For scenarios where a workload demands the preloading of substantial volumes of data before executing an application, a memory-optimized instance is recommended. Examples of such workloads include high-performance databases or tasks requiring real-time processing of significant amounts of unstructured data. In these situations, opting for a memory-optimized instance is ideal, as it is tailored to run applications demanding both high memory capacity and performance

Types:

R4, R5 — Same as with C4 and C5 this is the main instances but R5 newer.

R5a — Instance based on AMD processors.

X1, X1e — the lowest prices per GiB of RAM, but CPU is slightly slower than in usual instances.

z1d — Both CPU and RAM are very high.

Storage Optimized Instances (I, D, H):

Storage-optimized instances are tailored for workloads requiring rapid, sequential read and write access to extensive datasets. Examples of such workloads include distributed file systems, data warehousing applications, and high-frequency online transaction processing (OLTP) systems. These instances are designed to minimize latency when applications access data.

Types:

• I3 — General purpose instance with SSD storage
• I3en — The same, but with faster NVMe hard disks.
• D2 — Intends for the really large and slow storage and use HDD disks.
• H1 — Basically, the same as D2, but with smaller capacity of HDD.

Accelerated Computing Instances (P, F, G, Inf1):

Coprocessors play a crucial role in accelerated computing instances by enhancing the efficiency of specific operations compared to software executed on CPUs. These operations include tasks such as floating-point numeric computations, graphics processing, and data pattern matching. A Hardware Accelerator/Co-processor is a computing component designed to expedite data processing. Accelerated computing instances find applications in graphics, game streaming, and application streaming due to their ability to optimize these functions effectively.

Types

• P2, P3 — General purpose instances with Nvidia GPUs onboard.
• Inf1 — Instances using with inference applications.
• G3, G4 — Machine learning optimized instances
• F1 — These instances using FPGAs as a hardware accelerators.

HPC Optimized(H)

High Performance Computing (HPC) enables scientists and engineers to address intricate, compute-intensive challenges like computational fluid dynamics (CFD), weather forecasting, and genomics. HPC applications usually necessitate instances with elevated memory bandwidth, low latency, high bandwidth network interconnect, and access to a fast parallel file system

• H series: High disk throughput and I/O performance

Price Plans

• Free Tier: It’s a reality — you can have a genuine EC2 instance for free! However, it’s not overly powerful. Only t2.micro instances are available for free, offering 1 CPU and 1 GiB of RAM. While this might not be a lot, it’s sufficient for a small web server running on Node.js and based on Linux. It’s important to note that this free tier is valid for only one year and has a usage limit of 750 hours per month.
• On-Demand: You can create an instance whenever you need, for the desired duration, and choose any instance type. You’ll have it almost immediately, but of course, it comes at a cost.
• Spot Instances: This is the most intriguing plan for saving costs if you don’t need an instance for the long term and won’t store important data. Amazon offers unused capacity at a price you propose, usually significantly lower (up to 90%) than on-demand prices. However, be cautious — someone offering a higher price can bump you out.
• Reserved Instances: These are instances that you reserve in advance. There are standard and convertible RIs. Standard ones cannot change their type, while convertible ones can be converted to another type on demand.
• Dedicated Hosts: These are physical servers dedicated to you. While not the cheapest option, if you have special licensing requirements or a large application, it could be the best proposition.

 

INTRODUCTION TO SECURITY GROUPS

• Security Groups are crucial for network security in AWS.
• They regulate inbound and outbound traffic for EC2 instances.
• Security Groups are rule-based and do not contain any actual data.
• Rules within Security Groups can reference IP addresses or other Security Groups.

1. Attachment flexibility: Security Groups can be attached to multiple instances for consistent security policies.
2. Region and VPC-specific:Security Groups are restricted to specific regions and VPC combinations.
3. 3. External impact:Security Groups operate outside EC2 instances, blocking trafficthat the instance won’t see.
4. Separate SSH security:It’s advisable to maintain a separate Security Group for SSH access.
5. Troubleshooting clues:Time-out suggests Security Group issues, while “connection refused” points to application errors or launch problems.
6. Default traffic behaviour: Inbound traffic is blocked by default, while outbound traffic is authorized by default.

USER DATA

• User Data refers to user-provided information that is passed to EC2 instances during their launch process.
• It can be in the form of scripts, configuration files, or plain text-based input.
• User Data allows customization and configuration of instances to meet specific requirements.
• During the instance launch, the provided User Data is executed automatically.
• User Data scripts can be used to install software packages, configure services, set up custom settings, or perform any other necessary tasks to prepare the instance.
• User Data is commonly used for system configuration, application deployment, and automation of instance setup.

Note:  The  User Data is executed only once, during the initial launch of the instance. Subsequent changes or configurations typically require other methods such as SSH or remote management tools.

PURCHASING OPTIONS FOR EC2 INSTANCE

1. On-Demand Instances:Ideal for short workloads with predictable pricing, billed by the second.
2. Reserved Instances:Best for long workloads, offering significant cost savings over time with 1-year or 3-year commitments.
3. Convertible Reserved Instances:Similar to Reserved Instances but with the flexibility to change the instance type later.
4. Savings Plans:Requires a commitment to a specific amount of usage, suitable for long workloads, available in 1-year or 3-year options.
5. Spot Instances: Well-suited for short workloads with low costs, but instances can be lost if demand exceeds supply (less reliable).
6. 6. Dedicated Hosts:Allows booking an entire physical server, providing control over instance placement.
7. Dedicated Instances:Ensures that no other customers share the underlying hardware.
8. Capacity Reservations:Enables reserving capacity in a specific Availability Zone (AZ) for any desired duration.

Conclusion
Tailoring to specific workloads and application needs, you have the flexibility to select from various Amazon EC2 instance types as detailed in this guide. Amazon EC2 consistently broadens its array of instance types through ongoing releases to accommodate diverse workloads .We discussed virtualization, EC2 characteristics, AMIs, and the importance of selecting the right instance types and sizing options based on your needs. Additionally, we touched upon pricing configurations to help optimize costs.

At the time of writing this blog, in AWS there are approximately 776 instance to choose from.  You can use www.ec2instances.info website that will help you to choose the right instance for your workload on the basis of many parameters.