I like this approach. I thought about building something similar for years with the idea of using a cost benefit analysis to achieve the same type of thing.

Cost benefit meaning, total time to retrieve the item to be cached, amount of space used in the cache to store it and then frequency of access.

The general idea was that frequency of access * time to retrieve the item to be cached is the benefit, while the spaces used to store is the cost.

The approach should naturally bias towards storing the most frequently accessed and slowest items that are small so that more could be stored.

Maybe one day I’ll get around to building it.

@farhadi beat me to it – you can just periodically check and clean your caches.

Even better, you can put your own code before it, something like MyCache.maybe_put that checks the size and TTL of various items and just not put the item if your code deems the cache full.

Cache capacity is bounded by quantity not by size, and it is not possible to reconfigure it at runtime.

Actually you can specify the capacity of the underlying cuckoo filter and the capacity of cache is always less than the capacity of the filter.

Assume the capacity of the filter as a sliding window of requests. For this sliding window of requests only those that are repeated more than once will be stored in the cache. For example a cuckoo cache with a capacity of 1024 will keep track of latest 1024 requests probabilisticly (not exactly the latest ones but some recent 1024 requests.) and among those requests repeated ones are cached in ets. So the capacity of cache at its highest utilization could be equal to the number of unique requests in the recent 1024 requests.
And with default configurations, duplicate requests are not counted more than 8 times, which means if you have let’s say a single request that is repeated 500 times in the last 1024 requests in our example, it is only counted 8 times, letting more requests to be tracked.

Knowing the average size of the items that you add to the cache can give you an approximate estimation of how big it could grow in size. And if you need, you can create a gen_server to periodically check the memory usage of the table and remove some elements with your own strategy.

Of course, it would be nice that this functionality is included in the library, so my gen_server can do the check and tell the lib “memory is tight, up the aggressiveness in reclaim” or the other way around, and the cache will just shrink or grow gradually.