Designing a Microservices Architecture for Failure

The Risk of the Microservices Architecture

Graceful Service Degradation

One of the best advantages of a microservices architecture is that you can isolate failures and achieve graceful service degradation as components fail separately. For example, during an outage customers in a photo sharing application maybe cannot upload a new picture, but they can still browse, edit and share their existing photos.

Change Management – Rolling Deployment

Health-check and Load Balancing

Modern service discovery solutions continuously collect health information from instances and configure the load-balancer to route traffic only to healthy components.

Self-healing

Failover Caching

Retry Logic

As a retry is initiated by the client (browser, other microservices, etc.) and the client doesn’t know that the operation failed before or after handling the request, you should prepare your application to handle idempotency. For example, when you retry a purchase operation, you shouldn’t double charge the customer. Using a unique idempotency-key for each of your transactions can help to handle retries.

Rate Limiters and Load Shedders

Fail Fast and Independently

Bulkheads

Circuit Breakers

Testing for Failures

Outro

Key Takeways

https://blog.risingstack.com/designing-microservices-architecture-for-failure/