Service Mesh Architecture
Fast recall
Quick revision
Core answer
Sidecar proxy model, what cross-cutting concerns the mesh handles, control plane vs data plane, and when the operational cost is justified.
Confusing service mesh with API gateway (different layers — east-west vs north-south), or installing a mesh prematurely (5 services don't need it).
Discuss data plane vs control plane separation, eBPF-based meshes (Cilium) vs sidecar meshes (Istio/Linkerd), and how the mesh enables zero-trust networking.
Interview-ready guide
Interview answer
The problem
In a 50-microservice architecture in 5 languages, every service needs mTLS, retries, timeouts, circuit breaking, distributed tracing, metrics, load balancing, service discovery. Maintaining client libraries for 5 languages — drift, security gaps, version skew, bugs.
A service mesh moves all of this into a sidecar proxy — a separate Envoy process next to every service. App doesn't know it exists; it talks to localhost; proxy handles everything.
Architecture
Data plane: Envoy sidecars next to each app pod, intercept all traffic. App → localhost:proxy → mTLS → next pod's proxy → app. Transparent to application.
Control plane: Istiod / Linkerd controller. Operators define policies (mTLS required, retry budget, traffic split for canary). Control plane converts to Envoy config and pushes dynamically.
What the mesh provides
Security: automatic mTLS, certificates issued + rotated by control plane, identity-based authz, zero-trust networking.
Reliability: retries with backoff, timeouts, circuit breaking via outlier detection, bulkheading.
Traffic management: canary deploys (5% to v2), A/B routing on headers, fault injection (chaos), traffic mirroring (shadow to v2).
Observability: distributed tracing automatic, per-service/per-route metrics, topology visualization — no app code change.
Service mesh vs API gateway — different layers
API gateway: north-south traffic. Edge of system, client-facing. Public TLS, auth, rate limit per API key, CORS. Kong, AWS API Gateway.
Service mesh: east-west traffic. Between services. mTLS, retries, traffic shifting. Istio, Linkerd, Consul Connect.
Mature stack uses both — gateway protects perimeter, mesh enforces internal zero-trust.
Sidecar vs eBPF
Sidecar (Istio, Linkerd): Envoy per pod. Language-agnostic, mature. 50-100MB per sidecar, 1-3ms added latency, doubled pod count.
eBPF (Cilium): in-kernel network policies + telemetry, no sidecar process. Lower latency and memory. Less mature for L7 features (gRPC retries still need proxy).
Emerging hybrid: eBPF + minimal proxy where L7 needed.
When to use
- 20+ microservices
- Multi-language
- Need consistent zero-trust mTLS
- Centralized traffic management for canary / A/B / chaos
- Compliance requires uniform encryption + auth
When not to use
- < 10 services — client libraries are fine
- Single language — Resilience4j + OpenTelemetry + Spring Cloud covers most of what mesh does
- Team can't operate Envoy/Istio
Real cost: latency, memory, operational complexity. Break-even around 20-50 services.
Common mistakes
- Installing too early — 5 services don't need Istio
- Treating mesh as just an LB — it's much more
- Bypassing mesh for some services — inconsistent security
- Business logic in Envoy filters — filters are infrastructure, not domain
- Not budgeting latency — 1-3ms per hop × multiple hops adds up
- Defaulting to Istio — Linkerd is simpler/faster for most needs
Recommendation
Use a service mesh once you cross 20+ microservices, especially in multi-language setups, and need consistent mTLS, retries, traffic management, and observability. Linkerd for simplicity, Istio when you need its advanced features. Pair with an API gateway at the edge for client-facing concerns. For smaller fleets, lightweight libraries (Resilience4j + OpenTelemetry + Spring Cloud) are far simpler and cover the same ground without the operational tax. eBPF-based meshes (Cilium) are the emerging alternative for lower overhead.
Service Mesh Architecture
Direct Answer
A service mesh is a dedicated infrastructure layer for service-to-service communication — deployed as a sidecar proxy (Envoy) next to every service — that handles mTLS, retries, timeouts, traffic shifting, observability, and circuit breaking without changing application code. The control plane (Istio, Linkerd, Consul) configures all sidecars dynamically. Use a mesh once you have 20+ microservices and need consistent reliability/security policies; below that, lightweight client libraries are simpler.
Common Mistakes & Pitfalls
Confusing service mesh with API gateway (different layers — east-west vs north-south), or installing a mesh prematurely (5 services don't need it).
Check your understanding
Answer these without looking back at the article.
- How does mTLS rotation work in Istio without service restarts?
- Walk through a canary deployment using Istio traffic splitting.
- Compare Istio and Linkerd — when does each win?
- What changes when you replace a sidecar mesh with eBPF (Cilium)?
Related interview prep categories
1Want a guided curriculum order?
These modules are also sequenced in our flagship backend interview track — open it when you prefer an ordered path instead of browsing by topic.