Circuit Breaker Design Patterns in HTTP Clients

In distributed microservice architectures, cascading system failures are one of the most severe threats to overall availability. If a downstream microservice experiences high latency or database connection depletion, upstream services calling that endpoint will block waiting for HTTP socket timeouts.

As worker threads stack up waiting for unresponsive downstreams, the calling service exhausts its own thread pool and crashes—propagating a chain reaction of failures across the entire system. To prevent cascading failure storms, resilient systems implement the Circuit Breaker Design Pattern. In this technical article, we explore state machine mechanics, failure rate thresholds, exponential backoff with full jitter, and Python implementation.

1. The Three States of a Circuit Breaker

A circuit breaker acts as an intelligent proxy between microservices, maintaining an internal state machine with three distinct operational states:

• CLOSED (Normal Operation): All outbound HTTP requests pass through to the downstream service. The breaker monitors recent execution metrics (failure count or error percentage over a rolling window). If failure rates remain below threshold, the breaker remains CLOSED.
• OPEN (Failing Fast): When downstream failures exceed the configured threshold (e.g. 5 failures within 30 seconds), the circuit breaker trips OPEN. All subsequent requests fail immediately with a local CircuitBreakerOpenException without making network calls, shielding the struggling downstream service.
• HALF-OPEN (Probing Recovery): After a reset timeout expires (e.g. 15 seconds), the breaker transitions to HALF-OPEN. It permits a limited number of trial requests through to the downstream service. If trial requests succeed, the breaker resets to CLOSED; if any trial request fails, the breaker trips back to OPEN.

2. Production Python Circuit Breaker Implementation

Below is a thread-safe, asynchronous Python Circuit Breaker implementation wrapping the httpx HTTP client library:

import asyncio
import time
import httpx
from enum import Enum

class CircuitState(Enum):
    CLOSED = "CLOSED"
    OPEN = "OPEN"
    HALF_OPEN = "HALF_OPEN"

class CircuitBreakerOpenException(Exception):
    """Raised when request is blocked due to an open circuit breaker."""
    pass

class AsyncCircuitBreaker:
    def __init__(self, failure_threshold: int = 5, recovery_timeout: float = 15.0):
        self.failure_threshold = failure_threshold
        self.recovery_timeout = recovery_timeout
        self.state = CircuitState.CLOSED
        self.failure_count = 0
        self.last_state_change = time.time()
        self._lock = asyncio.Lock()

    async def call(self, func, *args, **kwargs):
        async with self._lock:
            now = time.time()
            
            # Check if recovery timeout has elapsed to attempt trial request
            if self.state == CircuitState.OPEN:
                if now - self.last_state_change > self.recovery_timeout:
                    self.state = CircuitState.HALF_OPEN
                    self.last_state_change = now
                else:
                    raise CircuitBreakerOpenException("Circuit is OPEN. Downstream service unavailable.")

        try:
            # Execute actual HTTP request
            result = await func(*args, **kwargs)
            
            async with self._lock:
                if self.state == CircuitState.HALF_OPEN:
                    # Successful trial request resets circuit to CLOSED
                    self.state = CircuitState.CLOSED
                    self.failure_count = 0
                    self.last_state_change = time.time()
            return result

        except Exception as err:
            async with self._lock:
                self.failure_count += 1
                if self.failure_count >= self.failure_threshold or self.state == CircuitState.HALF_OPEN:
                    self.state = CircuitState.OPEN
                    self.last_state_change = time.time()
            raise err

# Usage Example with httpx AsyncClient
circuit_breaker = AsyncCircuitBreaker(failure_threshold=3, recovery_timeout=10.0)

async def fetch_user_profile(user_id: int):
    async with httpx.AsyncClient(timeout=2.0) as client:
        return await client.get(f"https://api.riddletech.co.ke/users/{user_id}")

async def safe_api_call(user_id: int):
    try:
        response = await circuit_breaker.call(fetch_user_profile, user_id)
        return response.json()
    except CircuitBreakerOpenException:
        # Fallback response when downstream is offline
        return {"status": "degraded", "user_id": user_id, "cached": True}
    except Exception as err:
        return {"status": "error", "message": str(err)}

3. Exponential Backoff with Full Jitter

When combined with retries, fixed retry delays cause thundering herd problems where thousands of clients retry at the exact same millisecond mark. AWS research recommends Exponential Backoff with Full Jitter:

import random

def calculate_backoff_with_jitter(attempt: int, base_delay: float = 1.0, max_delay: float = 30.0) -> float:
    """Calculates exponential backoff with full randomized jitter."""
    temp = min(max_delay, base_delay * (2 ** attempt))
    sleep_duration = random.uniform(0, temp)
    return sleep_duration

Conclusion

Implementing the Circuit Breaker pattern with exponential backoff and fallback defaults builds resilient microservice networks. By insulating downstream services from retry storms during outages, systems maintain stability under peak load.