Compute, Networking, and Data Ingestion
Design high-performing compute, network, and data pipelines: EC2 instance families and placement groups, Lambda and containers, CloudFront vs Global Accelerator, hybrid connectivity, and ingestion and analytics with Kinesis, Glue, Athena, and Lake Formation.
Compute That Fits the Workload
EC2 instance families
| Family (examples) | Optimized for | Typical workload |
|---|---|---|
| General purpose (M, T) | Balanced CPU and memory | Web servers, small databases. T instances are burstable |
| Compute optimized (C) | High CPU per GB of memory | Batch processing, gaming servers, video encoding |
| Memory optimized (R, X) | Large memory | In-memory databases, caches, big data analytics |
| Storage optimized (I, D) | Fast local storage | NoSQL databases, data warehousing on local disks |
| Accelerated computing (P, G, Inf, Trn) | GPUs and ML chips | ML training and inference, graphics |
Instance types ending in g (like m7g) use AWS Graviton processors, which usually give better price-performance for workloads that run on Arm.
Placement groups
| Strategy | What it does | Use it for |
|---|---|---|
| Cluster | Packs instances close together in one AZ | Lowest latency, highest throughput between nodes (HPC) |
| Spread | Puts each instance on distinct hardware (max 7 running instances per AZ per group) | A small number of critical instances that must not fail together |
| Partition | Groups instances into partitions on separate racks | Large distributed systems like Hadoop, Cassandra, Kafka |
Choosing a compute model
- EC2: full control, any OS, long-running processes, licensed software
- ECS or EKS on Fargate: containers without managing servers
- Lambda: event-driven code, up to 15 minutes per invocation, scales to zero
- AWS Batch: queues and runs large numbers of batch jobs on EC2 or Fargate
Network Performance
CloudFront vs Global Accelerator
Both use the AWS edge network, but they solve different problems:
| Amazon CloudFront | AWS Global Accelerator | |
|---|---|---|
| What it is | Content delivery network that caches at the edge | Network accelerator with two static anycast IPs |
| Protocols | HTTP/HTTPS (and WebSocket) | TCP and UDP, any application |
| Caching | Yes | No |
| Best for | Websites, APIs, video, downloads | Gaming, VoIP, IoT, non-HTTP apps; fast Regional failover; clients that need fixed IPs |
Connecting VPCs and on-premises
| Need | Service |
|---|---|
| Connect two VPCs privately | VPC peering (not transitive: A–B and B–C does not give A–C) |
| Connect many VPCs and on-premises networks | Transit Gateway (hub and spoke, transitive routing) |
| Expose one service to other VPCs without full network access | AWS PrivateLink (interface endpoint + Network Load Balancer) |
| Quick, encrypted on-premises link over the internet | Site-to-Site VPN |
| Consistent, high-bandwidth private link to AWS | AWS Direct Connect (takes weeks to provision; not encrypted by default, so add MACsec or a VPN on top when required) |
| Highly available hybrid link | Two Direct Connect connections at different locations, or Direct Connect with a VPN backup |
Why large networks move from peering to Transit Gateway:
Enhanced networking (the Elastic Network Adapter) gives EC2 higher bandwidth and lower latency, and Elastic Fabric Adapter (EFA) adds OS-bypass networking for tightly coupled HPC and ML jobs.
Data Ingestion and Transformation
Domain 3 includes moving data in, transforming it, and making it queryable. A common serverless pipeline:
| Service | Role in the pipeline | Exam cue |
|---|---|---|
| Kinesis Data Streams | Real-time, ordered, replayable stream; you manage consumers | ”real-time,” “multiple consumers,” “replay” |
| Amazon Data Firehose | Fully managed delivery into S3, Redshift, OpenSearch, and others, with buffering and optional transformation | ”near real-time,” “load into S3 with no code” |
| Managed Service for Apache Flink | SQL or Java stream processing | ”analyze the stream in real time” |
| AWS Glue | Serverless ETL, crawlers, and the Data Catalog | ”transform,” “convert CSV to Parquet,” “catalog” |
| Amazon Athena | Serverless SQL directly on S3; pay per data scanned | ”ad hoc queries on S3” |
| AWS Lake Formation | Builds and governs a data lake with table-, column-, and row-level permissions | ”fine-grained access to the data lake” |
| Amazon EMR | Managed Spark, Hadoop, and more | ”existing Spark jobs” |
| AWS DataSync | Online transfer from on-premises storage (NFS, SMB, HDFS) to S3, EFS, or FSx | ”migrate file shares” |
| AWS Storage Gateway | Hybrid access: on-premises apps use S3-backed file, volume, or tape storage | ”on-premises apps need cloud storage” |
For dashboards, the exam guide lists Amazon Quick, AWS’s business intelligence suite, which now includes what used to be Amazon QuickSight.
Common mistake: a pipeline dumps uncompressed CSV into one S3 prefix and analysts query it with Athena every day.
Athena charges by data scanned. Converting to columnar Parquet, compressing it, and partitioning by date often cuts both cost and query time dramatically, because each query reads only the columns and partitions it needs.
Key Takeaway
Match the instance family to the bottleneck, use placement groups for latency (cluster) or isolation (spread, partition), and pick Lambda, Fargate, or EC2 by how much control you need. Use CloudFront to cache HTTP content and Global Accelerator for static IPs and non-HTTP traffic. Know the ingestion chain: Kinesis or Firehose → S3 → Glue → Athena, governed by Lake Formation.
In a sandbox account:
- Upload a CSV file (any public data set, a few MB) to an S3 bucket.
- Run an AWS Glue crawler on the prefix to create a table in the Data Catalog.
- Query it in Athena and note the “data scanned” figure.
- Use an Athena
CREATE TABLE AS SELECT(CTAS) query to write the same data as Parquet, partitioned by a date or category column. - Run the same query against the Parquet table and compare data scanned.
Delete the buckets and tables when you’re done.
1. A multiplayer game uses UDP and needs static IP addresses and fast failover between Regions. Which service fits BEST?
Global Accelerator provides static anycast IPs, supports TCP and UDP, and fails over between Regional endpoints quickly.
2. An HPC application needs the lowest possible network latency between its EC2 instances. What should you use?
Cluster placement groups pack instances close together in one AZ for low latency and high throughput.
3. Clickstream data must be loaded into S3 in near real time with no custom consumer code. Which service fits BEST?
Firehose is fully managed and delivers streaming data into destinations like S3 with buffering, without you writing consumers.
4. A company connects 40 VPCs and two data centers and wants transitive routing with central management. What should they use?
Transit Gateway is a hub that routes between many VPCs and on-premises networks. VPC peering is not transitive.