A repeatable troubleshooting method
- Identify: users, sites, applications, start time, frequency and last known good state.
- Establish impact: total outage, degradation, one direction, one protocol or one path.
- Collect changes: configuration, software, carrier, security policy, certificate, DNS or endpoint updates.
- Form hypotheses: rank by evidence and blast radius.
- Test minimally: choose the fastest safe observation that can disprove a hypothesis.
- Implement safely: record rollback and avoid changing several variables at once.
- Validate end-to-end: confirm user transactions, monitoring and redundancy.
- Document: timeline, root cause, contributing factors and preventive actions.
Questions that quickly narrow scope
| Question | What it separates |
|---|---|
| One user or everyone? | Endpoint/local access versus shared infrastructure. |
| IP address works but name fails? | DNS versus routing/application. |
| One direction or both? | Return path, ACL, NAT or asymmetric state. |
| Small traffic works but large transfers fail? | MTU, loss, windowing or inspection issues. |
| One VLAN/site/provider path? | Fault-domain and topology boundary. |
| New sessions fail but old sessions work? | Load balancer, DNS, session table, certificate or control-plane change. |
| Intermittent at a fixed interval? | Timers, lease/rekey, health checks, flapping or scheduled jobs. |
Layer-by-layer evidence
| Layer/domain | Key evidence |
|---|---|
| Physical | Link state, optic levels, cable, CRC/input errors, speed/duplex, power. |
| Layer 2 | VLAN, trunk, STP state, MAC learning, port-channel, loop/security events. |
| Layer 3 | Address/mask, ARP/ND, route lookup, adjacency, ECMP, VRF and return path. |
| Transport | TCP handshake, resets, retransmissions, zero window, UDP loss and ports. |
| Security/NAT | Rule match, state/session, translation, VPN selector, IPS/WAF action. |
| Services | DNS, DHCP, NTP, AAA, certificate and directory dependencies. |
| Application | HTTP status, server logs, database connection, thread/queue/resource health. |
A practical command toolkit
# Windows ipconfig /all route print arp -a ping HOST tracert HOST Test-NetConnection HOST -Port 443 nslookup HOST # Linux ip address ip route get DESTINATION ip neigh ping -c 4 HOST tracepath HOST ss -ntup dig HOST curl -vk https://HOST/
show interfaces show interfaces counters errors show vlan brief show interfaces trunk show spanning-tree show etherchannel summary show ip arp show ip route show ip protocols show logging
Reading a TCP failure
| Packet pattern | Likely direction |
|---|---|
| Repeated SYN, no SYN-ACK | Path/filter/server listening issue or response cannot return. |
| SYN → RST | Host or intermediary actively rejects; service may not be listening. |
| Handshake succeeds, then TLS alert | Certificate, protocol/cipher, SNI or application gateway issue. |
| Retransmissions and duplicate ACKs | Loss, congestion, reordering or path instability. |
| Zero window | Receiver/application cannot consume data quickly enough. |
| FIN/RST after regular interval | Idle timeout, health check, proxy or application timer. |
A packet capture shows what crossed the capture point, not automatically where the loss occurred. Capture on both sides of firewalls, load balancers or tunnels to compare.
Performance troubleshooting
High latency and low throughput are not synonymous. Establish bandwidth, round-trip time, loss, jitter, packet size, TCP window behaviour and server response time.
- Check interface drops and queue drops separately from physical errors.
- Measure one flow and multiple flows.
- Look for duplex mismatch, microbursts and oversubscribed uplinks.
- Confirm MTU through tunnels and VPN overhead.
- Use server timing to separate network transfer time from application processing.
- Compare good and bad paths at the same time.
Safe changes and escalation
Before a change, save current state, identify affected traffic, set a rollback trigger and ensure out-of-band access where possible. During a major incident, maintain a timeline and one change owner so parallel teams do not invalidate each other's tests.
An effective escalation includes topology, exact source/destination, timestamp/timezone, test result, relevant configuration, packet/log evidence, recent changes and what has already been ruled out. “Network is slow” is not an actionable escalation.
Interview scenarios
- User cannot open a website: test IP configuration, DNS, route, TCP 443, TLS and proxy/security policy in sequence.
- BGP up but prefix missing: inspect received routes, policy, next-hop reachability, best-path and RIB installation.
- VLAN works on one switch only: compare access VLAN, trunks, allowed VLAN list, STP and VTP/manual database state.
- VPN shows connected but subnet unreachable: inspect selectors, route, NAT exemption, encryption counters and remote return path.
- Application fails every 10 minutes: correlate session idle timeout, rekey, health monitor, service restart and scheduled resource exhaustion.
Interview focus
Use these points to practise concise, operations-focused answers. Explain the concept first, then give a verification or troubleshooting example.
- What should be defined before running commands? The symptom, scope, impact, timeline and last known good state.
- Why should only one variable be changed at a time? So the result can be attributed to that change and rollback remains controlled.
- What does repeated TCP SYN without SYN-ACK indicate? The response is absent due to filtering, routing, server listening or return-path issues.
- Why capture on both sides of a device? To determine whether packets entered, were transformed and left the boundary.
- What belongs in an escalation? Exact flow, timestamp, topology, evidence, recent changes, tests and ruled-out causes.
Quick self-check
1. What should be defined before running commands?
2. Why should only one variable be changed at a time?
3. What does repeated TCP SYN without SYN-ACK indicate?
4. Why capture on both sides of a device?
5. What belongs in an escalation?
Frequently asked questions
1. Should troubleshooting always start at OSI Layer 1?
2. Is ping enough to prove an application is healthy?
3. What is the difference between root cause and contributing factor?
Standards and further reading
- RFC 793 / RFC 9293 — Transmission Control Protocol
- RFC 1122 — Requirements for Internet Hosts
- NIST SP 800-61 Rev. 2 — Computer Security Incident Handling Guide
