High-Performing Storage and Databases
Choose high-performing storage and database solutions: block, file, and object storage; EBS volume types; EFS and the FSx family; S3 performance features; and when to pick RDS, Aurora, DynamoDB, ElastiCache, Redshift, and purpose-built databases.
Block, File, or Object?
The first storage question is always what shape the data access takes:
| Type | AWS service | Accessed by | Think of it as |
|---|---|---|---|
| Block | Amazon EBS (and instance store) | One EC2 instance (or a few, with io2 Multi-Attach) | A hard drive attached to a server |
| File | Amazon EFS, Amazon FSx | Many instances at once over NFS or SMB | A shared network drive |
| Object | Amazon S3 | Anything, over HTTPS APIs | Unlimited storage for whole files, addressed by key |
How each type connects to the things that use it:
EBS volume types
| Volume | Type | Use it for |
|---|---|---|
| gp3 | General-purpose SSD | The default. Baseline 3,000 IOPS and 125 MB/s at any size; buy more IOPS or throughput separately |
| io2 Block Express | Provisioned IOPS SSD | Latency-sensitive databases needing very high, consistent IOPS; supports Multi-Attach |
| st1 | Throughput-optimized HDD | Big sequential workloads: logs, data warehouses, streaming. Can’t be a boot volume |
| sc1 | Cold HDD | Rarely accessed sequential data at the lowest cost. Can’t be a boot volume |
Instance store is physically attached to the host. It’s the fastest disk you can get, but it’s ephemeral: data is lost when the instance stops or the host fails. Use it for caches, buffers, and scratch data only.
EBS volumes live in one AZ. To move a volume to another AZ, snapshot it and create a new volume from the snapshot.
File storage
| Service | Protocol | Use it for |
|---|---|---|
| Amazon EFS | NFS (Linux) | Shared files for many Linux instances, containers, and Lambda; elastic size; Regional (Multi-AZ) by default |
| FSx for Windows File Server | SMB, Active Directory | Windows apps and home directories needing SMB and AD integration |
| FSx for Lustre | Lustre | High-performance computing, ML training; can link to an S3 bucket |
| FSx for NetApp ONTAP | NFS, SMB, iSCSI | Migrating NetApp workloads; multiprotocol access |
| FSx for OpenZFS | NFS | Migrating ZFS workloads with very low latency |
S3 performance
- S3 scales automatically to at least 3,500 writes and 5,500 reads per second per prefix, so spreading keys across prefixes multiplies throughput.
- Multipart upload splits large files into parts uploaded in parallel (recommended above 100 MB).
- S3 Transfer Acceleration speeds long-distance uploads through CloudFront edge locations.
- Byte-range fetches download parts of a large object in parallel.
- S3 Express One Zone gives single-digit-millisecond access in one AZ for the most latency-sensitive workloads.
Choosing a Database
Performance levers for each
- RDS / Aurora: larger instance classes, Provisioned IOPS storage, read replicas for read-heavy load, RDS Proxy to pool connections (especially from Lambda, which can open thousands), and a cache in front.
- DynamoDB: design the partition key for even distribution (a hot key throttles), use on-demand capacity for unpredictable traffic, global secondary indexes for other query patterns, and DAX for microsecond reads.
- ElastiCache: lazy loading (cache on miss) or write-through (update cache on write), with a TTL to limit staleness.
- Redshift: columnar storage for analytics; Redshift Spectrum queries data directly in S3.
Common mistake: a design stores user sessions, a product catalog with millions of reads per second, and clickstream events all in one RDS instance.
Each of those has a better fit: sessions in ElastiCache or DynamoDB, hot catalog reads behind a cache, and clickstreams in Kinesis and S3. The exam rewards purpose-built databases matched to the access pattern.
Key Takeaway
Pick block, file, or object first, then the right flavor: gp3 by default, io2 for demanding databases, EFS for shared Linux files, FSx for Windows for SMB, FSx for Lustre for HPC, and S3 for objects at any scale. For databases, match the access pattern: relational to RDS or Aurora, key-value to DynamoDB, analytics to Redshift, caching to ElastiCache.
Choose the storage or database for each requirement, then check your answers.
- 50 Linux web servers need the same shared folder of uploaded files.
- A Windows .NET app needs a shared drive joined to Active Directory.
- A shopping cart needs millisecond reads and writes for millions of users.
- Analysts run complex SQL over 20 TB of sales history.
- A Lambda function overwhelms an RDS database with connections.
- A genomics job needs a high-throughput file system backed by data in S3.
Answers: (1) EFS, (2) FSx for Windows File Server, (3) DynamoDB, (4) Redshift, (5) RDS Proxy, (6) FSx for Lustre linked to S3.
1. A fleet of Linux EC2 instances in several AZs needs concurrent access to the same POSIX file system. Which service fits BEST?
EFS is a shared NFS file system that many Linux instances across AZs can mount at once.
2. A DynamoDB-backed product page needs microsecond read latency for very popular items, with minimal code changes. What should you add?
DAX is an in-memory cache for DynamoDB that serves reads in microseconds through a DynamoDB-compatible API.
3. Hundreds of concurrent Lambda functions are exhausting connections on an Aurora database. What is the BEST fix?
RDS Proxy pools and reuses database connections, protecting the database from connection storms.
4. Which EBS volume type is the general-purpose default with a baseline of 3,000 IOPS regardless of size?
gp3 provides 3,000 IOPS and 125 MB/s baseline, and lets you add IOPS and throughput independently of size.