High-Performing Storage and Databases
Module 4 · Design High-Performing Architectures

High-Performing Storage and Databases

Choose high-performing storage and database solutions: block, file, and object storage; EBS volume types; EFS and the FSx family; S3 performance features; and when to pick RDS, Aurora, DynamoDB, ElastiCache, Redshift, and purpose-built databases.

Block, File, or Object?

The first storage question is always what shape the data access takes:

TypeAWS serviceAccessed byThink of it as
BlockAmazon EBS (and instance store)One EC2 instance (or a few, with io2 Multi-Attach)A hard drive attached to a server
FileAmazon EFS, Amazon FSxMany instances at once over NFS or SMBA shared network drive
ObjectAmazon S3Anything, over HTTPS APIsUnlimited storage for whole files, addressed by key

How each type connects to the things that use it:

Block: EBSAZ AEC2EBS volumeOne instance, one AZFile: EFSEC2AZ AEC2AZ BEC2AZ CEFS file systemNFSMany clients, across AZsObject: S3EC2appLambdafunctionBrowserpresignedS3 bucketHTTPS APIAny client, by key, over HTTPS

EBS volume types

VolumeTypeUse it for
gp3General-purpose SSDThe default. Baseline 3,000 IOPS and 125 MB/s at any size; buy more IOPS or throughput separately
io2 Block ExpressProvisioned IOPS SSDLatency-sensitive databases needing very high, consistent IOPS; supports Multi-Attach
st1Throughput-optimized HDDBig sequential workloads: logs, data warehouses, streaming. Can’t be a boot volume
sc1Cold HDDRarely accessed sequential data at the lowest cost. Can’t be a boot volume

Instance store is physically attached to the host. It’s the fastest disk you can get, but it’s ephemeral: data is lost when the instance stops or the host fails. Use it for caches, buffers, and scratch data only.

EBS volumes live in one AZ. To move a volume to another AZ, snapshot it and create a new volume from the snapshot.

File storage

ServiceProtocolUse it for
Amazon EFSNFS (Linux)Shared files for many Linux instances, containers, and Lambda; elastic size; Regional (Multi-AZ) by default
FSx for Windows File ServerSMB, Active DirectoryWindows apps and home directories needing SMB and AD integration
FSx for LustreLustreHigh-performance computing, ML training; can link to an S3 bucket
FSx for NetApp ONTAPNFS, SMB, iSCSIMigrating NetApp workloads; multiprotocol access
FSx for OpenZFSNFSMigrating ZFS workloads with very low latency

S3 performance

  • S3 scales automatically to at least 3,500 writes and 5,500 reads per second per prefix, so spreading keys across prefixes multiplies throughput.
  • Multipart upload splits large files into parts uploaded in parallel (recommended above 100 MB).
  • S3 Transfer Acceleration speeds long-distance uploads through CloudFront edge locations.
  • Byte-range fetches download parts of a large object in parallel.
  • S3 Express One Zone gives single-digit-millisecond access in one AZ for the most latency-sensitive workloads.

Choosing a Database

Data and access patternServiceRelational, transactions, joins (SQL)RDS · Aurora for scale + HAKey-value, any scale, single-digit msDynamoDB · + DAX for µs readsAnalytics over large data sets (OLAP)RedshiftIn-memory cache, sessions, leaderboardsElastiCacheHighly connected data (social, fraud rings)Neptune (graph)JSON documents, MongoDB-compatibleDocumentDBTime series (IoT, metrics)Timestream for InfluxDB

Performance levers for each

  • RDS / Aurora: larger instance classes, Provisioned IOPS storage, read replicas for read-heavy load, RDS Proxy to pool connections (especially from Lambda, which can open thousands), and a cache in front.
  • DynamoDB: design the partition key for even distribution (a hot key throttles), use on-demand capacity for unpredictable traffic, global secondary indexes for other query patterns, and DAX for microsecond reads.
  • ElastiCache: lazy loading (cache on miss) or write-through (update cache on write), with a TTL to limit staleness.
  • Redshift: columnar storage for analytics; Redshift Spectrum queries data directly in S3.
⚠️ Common Pitfall: Using a Relational Database for Everything

Common mistake: a design stores user sessions, a product catalog with millions of reads per second, and clickstream events all in one RDS instance.

Each of those has a better fit: sessions in ElastiCache or DynamoDB, hot catalog reads behind a cache, and clickstreams in Kinesis and S3. The exam rewards purpose-built databases matched to the access pattern.


Key Takeaway

Pick block, file, or object first, then the right flavor: gp3 by default, io2 for demanding databases, EFS for shared Linux files, FSx for Windows for SMB, FSx for Lustre for HPC, and S3 for objects at any scale. For databases, match the access pattern: relational to RDS or Aurora, key-value to DynamoDB, analytics to Redshift, caching to ElastiCache.

🧪 Try It: Match the Storage

Choose the storage or database for each requirement, then check your answers.

  1. 50 Linux web servers need the same shared folder of uploaded files.
  2. A Windows .NET app needs a shared drive joined to Active Directory.
  3. A shopping cart needs millisecond reads and writes for millions of users.
  4. Analysts run complex SQL over 20 TB of sales history.
  5. A Lambda function overwhelms an RDS database with connections.
  6. A genomics job needs a high-throughput file system backed by data in S3.

Answers: (1) EFS, (2) FSx for Windows File Server, (3) DynamoDB, (4) Redshift, (5) RDS Proxy, (6) FSx for Lustre linked to S3.

🧠 Knowledge Check

1. A fleet of Linux EC2 instances in several AZs needs concurrent access to the same POSIX file system. Which service fits BEST?

2. A DynamoDB-backed product page needs microsecond read latency for very popular items, with minimal code changes. What should you add?

3. Hundreds of concurrent Lambda functions are exhausting connections on an Aurora database. What is the BEST fix?

4. Which EBS volume type is the general-purpose default with a baseline of 3,000 IOPS regardless of size?