Caching in WebFlux - GraphQL context, using Caffeine

By no means do I know if this is the best way, or even a suggested way, but it's the way that I got it to work. Happy to share. 🤙

We needed to cache expensive and recursive results in a GraphQL context, using Spring Boot and WebFlux. One could argue that we should refactor the schema to ensure that it does not happen, but we wanted to keep that as flexible as possible so 🔩 that.

Only using the classic config with the famous and reliable @Cashable() annotations simply did not work. If I understand it correctly it just caches the Producer, and since it's executed every time it's subscribed to, the end result is just the same as not caching at all. Booger me.

Well how do you do it then? I ran into a few roadblocks along the way. Using futures to cache the results kinda worked. But I quickly noticed that this did not cache recursive calls in the same query, on a repeated nested schema mapped value, since the producers are already spun up before the first value is cached. Second call is perfectly cached however. But this still led to a few thousand (extreme edge case) calls for no reason at all.

So you have to make it atomic, this makes it possible to cache the "mid flight" result of a producer, and route the other producers there once you have your lil' result. Great.

So what happened when I decided to test it? 💩 hit the 🪭 instantly, because it was executing the cache at the same time that the Mono's where resolving (on same thread), leading to recursive update exception. It was solved by forcing it to be async by subscribing to it, and then having a fallback for empty Monos so that you're not stuck in a forever deadlock.

Anyway, here's the config and imlp:

@Configuration
public class CacheConfig {

    @Bean
    public AsyncCache<UUID, FooObject> fooCache() {
        return Caffeine.newBuilder()
                .expireAfterWrite(15, TimeUnit.MINUTES)
                .maximumSize(10_000)
                .buildAsync();
    }

    @Bean
    public AsyncCache<String, BarObject> barCache() {
        return Caffeine.newBuilder()
                .expireAfterWrite(60, TimeUnit.MINUTES)
                .maximumSize(5000)
                .buildAsync();
}

the values being arbitrary, use whatever you'd like.

public class CacheUtil {

    public static <K, V> Mono<V> cacheMono(AsyncCache<K, V> cache, K key, Supplier<Mono<V>> loader) {

        CompletableFuture<V> future = cache.get(key, (k, executor) -> {
            CompletableFuture<V> f = new CompletableFuture<>();

            // subscribe on a scheduler to guarantee async execution
            loader.get()
                    .subscribeOn(Schedulers.boundedElastic()) // forces async for same millisecond edge cases
                    .subscribe(
                            f::complete,
                            ex -> {
                                f.completeExceptionally(ex);
                                CompletableFuture.runAsync(() -> cache.asMap().remove(k, f)); // don't save the exceptions
                            },
                            () -> f.complete(null)); // stops deadlock on Mono.empty()

            return f;
        });

        return Mono.fromFuture(future);
    }

    public static <K, T> Flux<T> cacheFlux(AsyncCache<K, List<T>> cache, K key, Supplier<Flux<T>> loader) {
        return cacheMono(cache, key, () -> loader.get().collectList()).flatMapMany(Flux::fromIterable);
    }
}

and finally the actual use of one, I have not tried making into aspects because I'm lazy, and it's Friday evening. So it's just wrapper. Anyway, it's fairly simple:

    @Autowired
    private final AsyncCache<UUID, FooObject> fooCache;

    public Mono<FooObject> getFooOrSomething(UUID fooId) {
        return CacheUtil.cachedMono(fooCache, fooId, () -> fooService
                .findFoo(fooId));
    }

you can do all sorts of flatMapping, zipping or whatever in there, it's the final result of the publisher that is actually cached, no matter what that might be.

I know that there's no GraphQL code directly in here, but I still mention it because that's what ultimately created the need for me. 📈

Merry Christmas folk 🎄