Eventual consistency and cache

What is it?

Eventual consistency is a fancy name of doing something and only expecting the changed state after a while. And there is a guarantee that you will eventually get the expected state. Hence, consistency.

But that won’t work well with caching. And I had to deal with that recently when working with WhatsApp API.

When querying for a list of message templates from WhatsApp, we’ll cache the response for use later. It’s because message templates are not something that changes so frequently. Just cache it to reduce unnecessary API calls.

After creating a template, we won’t see it for a while if we call the listing API. That’s because of, you guessed it, eventual consistency.

Things will get worse because these responded message templates will be cached for a few minutes. Therefore, we won’t see the newly created template soon, even though it may be available on the WhatsApp API earlier.

How to deal with it?

First, let’s simply ignore the cache for, like 1 minute, after creating a new template. That would be enough for the WhatsApp API to return the newly created template.

The next improvement would be reducing the cache expiration time to, like 5s, in the first 1 minute. Let’s call that the catching-up duration. After that, use the original expiration time.

Another improvement would be verifying that the new template is available in the response and starting caching at that point. That would be a bit more complex, as we need to store the template id and more code to extract it from the response.

Finally, while that new template is not available, we can still construct it and manually inject it into our own response.

In the end, that works well. We can have both cache and eventual consistency playing together, minimizing the waiting time and the number of requests to an external platform! 🎉

Previously published at olvrng.github.io.