EC2 instance is an amazon virtual machine, there are many EC2 instance types, find below the main ones:
- R: apps that need a lot of RAM -in memory caches
- C: apps that need good CPU – compute / databases
- M: apps that are balanced (think medium) – general / web app
- I: apps that need good local I/O (instance storage) – databases.
- G: apps that need a GPU – video rendering / machine learning.
- T2/T3: burstable instances (up to a capacity)
- T2/T3: unlimited burst
- On Demand: pay per hour or second with no commitment, it is low cost, flexible and used for short term, dev/testing and you have a predictible price.
- Reserved: you get a significant discount (1y-3y), it is used for apps that have steady state and that require capacity:
- Convertible reserved instances: long workloads with flexible instances.
- Scheduled reserved instances, example: every Thu between 3 and 6 pm.
- Spot: you bid whatever price you want for instance, it is used for apps that have flexible start & end times.
- Dedicated instances: no other customers will share your hardware, billing is based on instance.
- Dedicated hosts: you book an entire physical server, for regulatory (no multi-tenant support), great for licensing,
Important: for prod environnement
Termination protection is turned off by default, you must turn it on.
EBS: 5 different types of EBS storage:
- General purpose SSD
- Provisionned iops SSD
- Throughput optimized HDD
- Cold HDD (lowest )
- EBS magnetic
AMI Types (EBS vs instance store):
- For EBS volumes: the root device for an instance launched from the AMI is an EBS volume created from an EBS snapshot.
- For Instance store volumes: the root device for instance launched from the AMI is an instance store volume created from a template stored in S3.
- Instance store volume is an Ephemeral storage: if stopped, you will lose all your data.
- Instance store is physically attached to the machine (EBS is a network drive).
- Instance Store Pros:
- Better I/O perofrmance
- Good for buffer / cache / scratch data / temporary content.
- Instance Store Cons:
- On stop or termination, the instance store is lost.
- You cant resize the instance store
- Backups must be operated by the user.
- EBS RAID Options (do it on OS not AWS):
- RAID0: increase performance (lots of iops), but more risk (if 1 disk fail, data lost).
- RAID1: increase fault tolerance (mirroring).
- RAID5 (not recommanded for EBS)
- RAID6 (not recommanded for EBS)
Elastic Network Interface vs Enhanced Networking vs Elastic Fabric Adapter:
- ENI: elastic network interface, virtual network card
- For basic networking, create a management network, use a network appliance in your VPC..
- Low cost
- EN: Enhanced networking: uses single root I/O virtualization, there is 2 types:
- ENA elastic network adapter: 100Gbps
- VF virtual function: 10Gbps
- Elastic Fabric Adapter: for machine learning or high performing computing (HPC) or OS bypass.
- Data in transit between an instance and an encrypted volume is also encrypted
- Encryption is supported on all Amazon EBS volume types
EFS: Elastic File System
- It grows automatically, great for file server or sharing data between EC2 instances
- You pay for what you use
- EFS support NFSv4 protocol
- You only pay for the storage you use
- Can scale up to the petabytes
- Can support 1000s of concurrent NFS connections
- Data is stored across multiple AZs within a region
- Read After write consistency
Amazon FSx for Windows and for Lustre:
- FSx for windows: built on windows server, use SMB and supports AD users, DFS (Distributed File System).. . Centralized storage for windows, Sharepoint, IIS or other native app for microsoft:
- Can be accessed from your on-premise infrastructure
- Can be configured to be MultiAZ (HA)
- Data is backed up daily to S3
- EFS: is linux only, use it if you need distribution..
- FSx for Lustre: for linux, if you process large data sets, millions of IOPS, machine learning, High Performance Computing (HPC), video processing, electronic design automation, financial modeling.
EC2 Placement Groups:
- Clustered placement group: within a single Availability Zone
- Pros: Low Network Latency / High network throughput
- Cons: if the rack fails, all instances fails at the same time.
- Use case: Big data job that needs to complete fast.
- Spread placement group: individual instances placed in different hardware (rack..), for single instance
- Can span across AZs
- Reduce risk of simultaneous failure
- EC2 instances are on different physical hardware
- Limited to 7 instances per AZ per placement group
- Use case:
- App that needs to maximize high availability.
- Critical apps where each instance must be isolated from failure from each other
- Partitioned Placement Group: think multiple instances in partition group.
- Up to 7 partitions per AZ
- Up to 100s of EC2 instances
- The instances in a partition do not share racks with the instances in the other partitions.
- A partition failure can affect many EC2 but won’t affect other partitions.
- EC2 instances get access to the partition: information as metadata
- Use case: HDFS, HBase, Cassandra, Kafka
- The in-memory (RAM) state preserved.
- The instance boot is much faster (os is not stopped/restarted)
- Under the hood:the RAM state is written to a file in the root EBS volume
- The root EBS volume must be encrypted
- Supported instance families: C, M and R.
- Instance RAM size must be less than 150GB
- Available for OnDemand and Reserved Instances.
- An instance cannot be hibernated more than 60days
- Use cases:
- long running processing
- saving the RAM state
- services that take time to initialize.
EC2 Best practices
- Manage access to AWS resources using IAM roles.
- Implement the least permissive rules for your security group (Firewall).
- Patch, update and secure regularly the operating system and applications on your EC2 instance.
- Use separate Amazon EBS volumes for the operating system and data.
- Ensure that the data volume persists after instance termination.
- Encrypt EBS volumes and snapshots.
- Use instance metadata and custom resource tags to track and identify your AWS resources.
- View your current limits for Amazon EC2 and plan in advance the request of any limit increases.
Backup and recovery
- Backup periodically your EBS volumes using snapshots.
- Create an Amazon Machine Image (AMI) from your instance to save the configuration as a template if needed for future installation.
- Deploy critical components of applications across multiple AZs.
- Monitor and respond to events.
- Test regularly the process of recovering your instances and Amazon EBS volumes if they fail.