Taming the Beast of Intermittent Failures: A Laravel Circuit Breaker Tale

Photo by Troy Bridges on Unsplash

Ring, ring! The alarm bells are going off again. The once reliable third-party service, our good old friend (Trading Service), has started acting out like a rebellious teenager. Sigh. Any seasoned developer would get a few more grey hairs from inconsistent answers and sporadic failures.

Well, let’s not give up, ladies and gentlemen. This is not my first time at the game, and I’m here to share a small Laravel magic trick that has more than once kept me sane. Let’s talk about implementing a Circuit Breaker when everyone is together.

A Circuit Breaker? you ask. Are we going to dabble in some electrical engineering? Laughs in code. Not quite! A circuit breaker is a design pattern that stops a system from continually attempting to carry out an operation that is likely to fail.

Now, let’s get our hands dirty with some code.

Here is some Laravel code that illustrates how to remain resilient in the face of resistance from others:

public function handle(CompleteKyc $event)
{
    try {
        $delay = $this->applyCircuitBreaker();
        if ($delay !== null) {
            $this->release($delay);
        }
        $result = $this->createZanibalAccount($event);

        Cache::forget('failures');
        Cache::forget('circuit:open');
    } catch (Exception $exception) {
        if($exception->getCode() >= 500){
          $this->handleException($exception);
          $this->release(600);
        }
        
    }
}

You’ll see that this Laravel job/listener has a fair dosage of self-preservation ingrained in them.

The major performance is provided by the handle method. Using our helpful neighbor $this->applyCircuitBreaker(), it determines whether a circuit is open (signifying a recent failure). If the circuit is indeed open, it hibernates like a well-fed bear, delaying any future demands until the situation is under control and the coast is clear.

protected function applyCircuitBreaker()
{
    $lastFailureTimestamp = Cache::get('circuit:open');
    if (is_int($lastFailureTimestamp) && time() - $lastFailureTimestamp < 8 * 60) {
        return $lastFailureTimestamp + 600 + rand(1, 120);
    } else {
        $this->halfOpen = true;
        return null;
    }
}

In the applyCircuitBreaker method, the magic happens. It obtains the date of the most recent failure and, if it is older than eight minutes, requires a delay before the following request. For a touch of unpredictability, a random additional delay (between 1 and 120 seconds) is introduced. Variety is the flavor of life, after all, and even code needs to be jazzed up occasionally!

protected function handleException(Exception $exception)
{
    if ($this->halfOpen) {
        Cache::put('circuit:open', time(), 600);
        return $this->release(600);
    }
    if (!$failures = Cache::get('failures')) {
        Cache::put('failures', 1, 60);
    } else {
        Cache::increment('failures');
    }

    if (Cache::get('failures') > 10) {
        Cache::put('circuit:open', time(), 600);
    }
}

Should an exception occur (and let’s face it, exceptions happen to the best of us), the handleException method swoops into action like a superhero in a cape. It increments a failure count in our cache, but after ten failures, it drops the big bomb and opens the circuit, preventing further heartache (or in this case, request-ache) for a cooling-off period.

Like any good drama, there’s a plot twist. If the circuit is in a ‘half-open’ state — meaning it’s testing the waters after a previous failure — and an exception occurs, it immediately halts the operation and reopens the circuit.

protected function createZanibalAccount(CompleteKyc $event){
    //Your third party API
}

In the end, this code does more than just carry out duties; it also tells a story about overcoming difficulty, rising above failure, and making sure that our system can withstand the unpredictability of using third-party services. It recognizes that sometimes you have to take a step back, gather your breath, and only then go back into the fight, much like a stubborn boxer in the ring.

So the next time your third-party service decides to throw a tantrum, don’t panic! Just pull out your reliable circuit breaker, and let it do the heavy lifting. Just remember, as I always say, “Keep calm, code on!”

That’s it for now. Until next time, may your code be bug-free and your coffee strong.

Happy coding, my fellow Laravel enthusiasts!