Troubleshooting Like a TAM
A structured troubleshooting method for scenario questions, the AWS signals to check at each layer, and worked examples: an unreachable EC2 instance, ALB 5xx errors, S3 Access Denied, and API throttling.
Why Scenario Questions Matter
Many TAM interviews include a troubleshooting scenario: “A customer calls and says their site is down. What do you do?” The interviewer isn’t only testing whether you know the answer. They’re watching how you think: whether you ask clarifying questions, narrow the problem methodically, consider the business impact, and know when to escalate.
A Method You Can Say Out Loud
- 1 Scope
Who, what, since when, how bad?
- 2 What changed?
Deploys, config, traffic, certs
- 3 AWS Health
Is it us or them?
- 4 Layers
DNS → edge → LB → compute → data
- 5 Hypothesis
Test one thing at a time
- 6 Mitigate
Restore service, then root cause
Scope questions to ask first:
- Is it all users or some? One Region, one AZ, one customer segment?
- When did it start? Exactly? Does it line up with a deploy or a traffic spike?
- What’s the business impact? Revenue loss? Safety? Regulatory?
- Has a support case been opened, and at what severity?
Mitigate first. A TAM’s first priority is restoring service: roll back the deploy, fail over, shift traffic, scale out. Root cause comes after customers are no longer affected.
Signals at Each Layer
| Layer | What to check | Where |
|---|---|---|
| DNS | Does the name resolve? Right target? TTLs? | Route 53, dig / nslookup |
| Edge | CDN errors, WAF blocks, certificate expiry | CloudFront metrics and logs, WAF logs, ACM |
| Load balancer | Healthy target count, 4xx vs 5xx, latency | ELB CloudWatch metrics, access logs |
| Network | Security groups, NACLs, routes, NAT | VPC Flow Logs, Reachability Analyzer |
| Compute | CPU, memory, disk, status checks, app logs | CloudWatch, CloudWatch Logs, SSM |
| Data | Connections, CPU, storage, replica lag, throttling | RDS Performance Insights, DynamoDB metrics |
| Control plane | Who changed what, and when? API errors? | CloudTrail, AWS Config |
Worked Scenarios
”I can’t SSH to my EC2 instance.”
- Check status checks. A system check failure points to AWS hardware (stop and start to move hosts). An instance check failure points to the OS (full disk, bad kernel, broken network config).
- Check the security group (port 22 from your IP?), the NACL (both inbound and outbound ephemeral ports, since NACLs are stateless), and the route table (an internet gateway route for a public subnet).
- Does the instance have a public IP or Elastic IP? Is the key pair right?
- A better long-term answer: use Systems Manager Session Manager so no inbound port is needed at all.
”Our ALB is returning 5xx errors.”
- 502: the target closed or reset the connection, or returned a malformed response. Check app crashes and keep-alive timeouts.
- 503: no healthy targets are registered. Check health checks and the target group.
- 504: the target took too long. Check slow downstream calls, like the database.
- Compare
HTTPCode_ELB_5XX(generated by the load balancer) withHTTPCode_Target_5XX(returned by your app). That one comparison tells you which side to investigate.
”S3 says Access Denied.”
Permissions on AWS are evaluated across several policies, and an explicit Deny anywhere wins. Check, in order: the IAM identity policy, the bucket policy, Block Public Access settings, SCPs from AWS Organizations, KMS key policy (for encrypted objects), object ownership, and VPC endpoint policies. CloudTrail shows the denied call.
”We’re getting ThrottlingException.”
The customer is hitting an API rate limit or service quota. Short-term: retries with exponential backoff and jitter. Long-term: request a quota increase, cache results, batch calls, and set CloudWatch alarms on quota usage. This is a classic proactive TAM finding: quotas checked before the launch, not during it.
Common mistake: the interviewer says “the site is down,” and the candidate immediately says “it’s probably the security group.”
Even if you’re right, you’ve skipped scoping and impact. Interviewers want to hear questions first, a method, and an explicit mitigate-then-investigate priority. Think out loud.
When to Escalate
As a TAM you are rarely the person with root access to the customer’s systems. Your job is to get the right experts engaged fast:
- Make sure a case exists at the right severity with clear impact and timeline
- Engage the support engineer on the case and, if needed, the service team through internal escalation paths
- Keep a single source of truth for the timeline and next update time
- Tell customer leadership what you know, what you don’t, and when they’ll hear next
Key Takeaway
In scenario questions, scope first, check what changed, check AWS Health, walk the layers, test one hypothesis at a time, and mitigate before root cause. Know the classic patterns (status checks, SG vs NACL, 502/503/504, explicit deny, throttling with backoff) and show that you know when and how to escalate.
Set a timer for 8 minutes and answer out loud:
“It’s 9:15 a.m. A retail customer’s checkout API latency jumped from 200 ms to 6 seconds at 9:02. Nothing was deployed today. The VP of E-commerce is on the phone.”
Cover: your scoping questions, what you’d check first and why, what you’d tell the VP in the first 5 minutes, and two possible causes (for example, database connection exhaustion after a traffic spike, or a downstream dependency timing out). Record yourself and listen back. Did you mitigate before you investigated?
1. An ALB returns 503 errors to all users. What is the most likely cause?
503 from an ALB typically means there are no healthy targets to send traffic to.
2. Why must NACL rules allow outbound ephemeral ports for inbound SSH to work?
Unlike security groups, NACLs don't track connections, so response traffic needs its own rule.
3. During a production outage, what is a TAM's first priority?
Mitigate first; root cause analysis comes after customers are no longer affected.
4. What is the best short-term client-side response to ThrottlingException?
Backoff with jitter spreads retries out and avoids making throttling worse.