Building Scalable Microservices: Architecture Patterns and Best Practices

A client came to us wanting to "move to microservices" for their e-commerce platform. Their monolith was slow to deploy, hard to scale, and teams stepped on each other's code. Classic microservices motivation.

We didn't start splitting services. We started by asking: "Which specific pain points are you trying to solve?" The answer: deployments took 45 minutes (because the test suite was slow), and the inventory system couldn't scale independently from the storefront. Two problems, not twenty.

We extracted two services — inventory and storefront — and kept everything else as a modular monolith. Deploy time dropped to 8 minutes. Inventory scaled independently during flash sales. Total effort: 3 months instead of the 18-month "full microservices rewrite" they'd been quoted elsewhere.

At Pillai Infotech, we've built microservices architectures for companies scaling from thousands to millions of users. The lesson every time: the right number of services is the minimum that solves your actual problems.

When NOT to Use Microservices

This section comes first because it's the most important. Most teams that adopt microservices do it too early, with too many services, and regret it.

Don't Use Microservices If:

• Your team is under 20 engineers: Microservices add operational overhead — CI/CD per service, distributed tracing, service discovery, inter-service communication. A small team spends more time managing infrastructure than building features.
• You're building a new product: You don't know your domain boundaries yet. Service boundaries drawn on day one will be wrong. Start with a modular monolith and extract services when you understand the domain.
• Your "scaling problem" is actually a database problem: Adding read replicas, caching, or query optimization solves 80% of scaling issues without splitting services.
• Your "deployment problem" is actually a testing problem: If deploys are slow because tests are slow, fix the tests. Microservices make deployment faster per service but harder overall (coordinated releases, contract testing).

Drawing Service Boundaries: The Hard Part

Bad service boundaries are the #1 cause of microservices failure. If your services are too small, you get a "distributed monolith" — tightly coupled services that must be deployed together. If too large, you haven't actually decomposed anything.

Domain-Driven Design (DDD) for Boundaries

The best tool for finding service boundaries is DDD's concept of bounded contexts. A bounded context is a section of the business domain with its own language, rules, and data model.

For an e-commerce system:

• Order context: Order, OrderItem, OrderStatus — focused on purchase lifecycle
• Inventory context: Product, Stock, Warehouse — focused on physical goods
• Payment context: Transaction, Refund, PaymentMethod — focused on money movement
• Customer context: Account, Address, Preferences — focused on user identity

Notice: "Product" means different things in each context. In Inventory, it has stock levels and warehouse locations. In Orders, it has a name, price, and quantity. In Payments, it's just a line item amount. Each context has its own model of the same real-world concept.

The Litmus Test for Service Boundaries

A service boundary is correct if:

1. The service can be deployed independently — without coordinating with other services
2. The service owns its data — no shared databases with other services
3. Changes are contained — most feature work touches only one service
4. The team can own it — a single team (2-8 people) can develop, deploy, and operate it

If you find yourself frequently making changes across 3-4 services for a single feature, your boundaries are wrong.

Communication Patterns

Synchronous: REST and gRPC

Our recommendation: REST for external APIs + gRPC for internal service-to-service communication. gRPC's type-safe contracts (Protobuf) and HTTP/2 streaming make it ideal for inter-service calls. REST's simplicity and tooling make it better for browser clients. For more on this, see our GraphQL vs REST comparison.

Asynchronous: Events and Messages

The most important insight in microservices architecture: prefer asynchronous communication over synchronous wherever possible. Async communication decouples services in time — the caller doesn't wait for a response, and the callee processes at its own pace.

• Event-driven (pub/sub): Service A publishes "OrderCreated" event. Services B, C, D subscribe and react independently. Publisher doesn't know or care about subscribers.
• Command-driven (queue): Service A sends "ProcessPayment" command to a queue. Payment service processes it. Guaranteed delivery, ordered processing.
• Event sourcing: Store the sequence of events as the source of truth (not the current state). Rebuild state by replaying events. Powerful but complex — use sparingly.

Tools: Apache Kafka (event streaming at scale), RabbitMQ (message queue, simpler), Amazon SQS/SNS (managed, good starting point), NATS (lightweight, fast).

Data Management: The Hardest Problem

Database per Service

Each microservice must own its data. No shared databases. This is non-negotiable — a shared database creates a coupling point that undermines every benefit of microservices.

This means:

• No cross-service JOINs: If Order Service needs customer data, it calls Customer Service's API — not its database.
• Data duplication is expected: Each service maintains a local copy of data it needs from other services. Eventual consistency, not real-time sync.
• Polyglot persistence: Each service chooses the best database for its use case. Orders in PostgreSQL, product catalog in MongoDB, search in Elasticsearch, sessions in Redis.

Distributed Transactions: The Saga Pattern

How do you handle a business transaction that spans multiple services? Example: placing an order requires inventory reservation, payment processing, and order creation — across three services.

The answer is the Saga pattern — a sequence of local transactions coordinated by events or a central orchestrator:

Choreography-based saga: Each service listens for events and reacts.

1. Order Service → creates order (PENDING) → publishes OrderCreated
2. Inventory Service → reserves stock → publishes StockReserved
3. Payment Service → charges card → publishes PaymentCompleted
4. Order Service → updates order (CONFIRMED)

If payment fails:
3. Payment Service → publishes PaymentFailed
4. Inventory Service → releases stock (compensating transaction)
5. Order Service → updates order (CANCELLED)

Orchestration-based saga: A central service coordinates the steps.

Order Orchestrator:
  1. Call Inventory.ReserveStock()
  2. If success → Call Payment.Charge()
  3. If success → Call Order.Confirm()
  4. If any step fails → call compensating transactions in reverse

Choreography is simpler for 2-3 services. Orchestration is better for complex workflows with 4+ steps.

Resilience Patterns

In a microservices architecture, failures are normal. Services go down, networks are unreliable, latency spikes happen. Design for failure.

Circuit Breaker

When a downstream service is failing, stop calling it. The circuit breaker pattern:

• Closed: Normal operation — requests pass through. If failures exceed threshold (e.g., 50% of last 10 calls), open the circuit.
• Open: All requests fail immediately (fast fail). After a timeout (e.g., 30 seconds), switch to half-open.
• Half-open: Allow one test request. If it succeeds, close the circuit. If it fails, stay open.

Retry with Exponential Backoff

Transient failures (network glitch, brief overload) often resolve on retry. But naive retries cause thundering herds. Use exponential backoff with jitter:

// Retry with exponential backoff + jitter
retryDelay = min(baseDelay * 2^attempt + random(0, baseDelay), maxDelay)

// Example: base=100ms, max=30s
// Attempt 1: ~100-200ms
// Attempt 2: ~200-400ms
// Attempt 3: ~400-800ms
// Attempt 4: ~800-1600ms

Bulkhead Pattern

Isolate failures to prevent cascade. If your payment service is slow, it shouldn't consume all your thread pool and make your entire application unresponsive. Allocate separate connection pools and thread pools per downstream service.

Timeout Everything

Every external call (HTTP, database, cache, message queue) must have a timeout. A missing timeout on one slow service can cascade through your entire system as threads pile up waiting. Set aggressive timeouts (2-5 seconds for service calls) and handle timeouts gracefully.

For advanced reliability patterns, see our SRE guide.

API Gateway: The Front Door

An API gateway sits between clients and your microservices. It handles cross-cutting concerns so individual services don't have to:

• Routing: Direct requests to the right service based on path, headers, or other criteria
• Authentication: Validate tokens once at the gateway, pass verified identity to services
• Rate limiting: Protect services from traffic spikes
• Response aggregation: Combine responses from multiple services into one client response
• Protocol translation: Accept REST from clients, call gRPC services internally

Popular gateways: Kong (full-featured, Lua plugins), Envoy (high-performance, Kubernetes-native), AWS API Gateway (managed), Traefik (auto-discovery, Docker/K8s).

Deployment & Scaling

Container Orchestration

Microservices and containers go together. Each service runs in its own container, scaled independently. Kubernetes is the standard orchestrator, but consider simpler alternatives for smaller deployments:

• 5-10 services: Docker Compose or ECS Fargate — simpler operational model
• 10-50 services: Kubernetes (managed: EKS, GKE, AKS) — full orchestration power
• 50+ services: Kubernetes + service mesh (Istio, Linkerd) — traffic management, mTLS, observability

Scaling Patterns

• Horizontal scaling: Run more instances of a service behind a load balancer. The default scaling strategy.
• Auto-scaling: Scale based on CPU, memory, request rate, or custom metrics. Kubernetes HPA (Horizontal Pod Autoscaler) or cloud auto-scaling groups.
• CQRS (Command Query Separation): Separate read and write paths. Write to a primary database, read from optimized read replicas or materialized views. Scales reads independently from writes.

For CI/CD patterns specific to microservices, see our CI/CD pipeline guide. Each service should have its own pipeline, deploying independently.

Microservices Anti-Patterns

1. The Distributed Monolith

Services that must be deployed together, share databases, or require coordinated releases. You got the complexity of microservices with none of the benefits. Fix: re-evaluate service boundaries, eliminate shared state.

2. Nano-Services

Services that are too small — single-function services with no independent business meaning. Each service adds operational overhead. If it can't justify its own deployment pipeline, it shouldn't be a separate service.

3. Synchronous Chains

Service A calls B calls C calls D synchronously. Latency compounds, and one slow service blocks the entire chain. Fix: use async events for non-critical paths, parallelize independent calls.

4. Shared Libraries with Business Logic

A "common" library that contains business rules used by multiple services. Updating the library requires redeploying every service. Fix: shared libraries should contain only infrastructure code (logging, auth, serialization), never business logic.

5. No Observability

Operating 20 services without distributed tracing, centralized logging, or service-level metrics. Debugging production issues becomes impossible. Fix: implement observability before going to production.