Release It! - Chapter 4.3 Cascading Failures

A Cascading Failure is when a crack in one layer triggers a crack in a calling layer.  The failure jumps between the layers when bad behavior in the caller gets triggered by a problem in the called layer.  Resource pools often get exhaused in this scenario and Integration Points without timeouts is a sure way to cause a Cascading Failure.  Cascading Failure are crack accelerators so preventing them is very important.  Stop cracks from spanning layers and make sure calling layers can still function even after a lower layer goes dark.  Examine your resource pools to ensure they are safe.  Safe pools always limit the time a thread can wait to check out a resource.  Defend against Cascading Failures by using Circuit Breaker and Timeout  patterns.  Circuit Breakers prevent call outs to sick layers and Timeouts ensure that you can return from a call to a sick layer.