Add support for cache size limits with cache replacement policies

[shaypal5]

Cachier-managed caches can naturally grow very big, especially if the decorator us used to cache large return values (in-memory models, larges corpuses, datasets, etc.).

The recently added ability to have stale cache entries automatically deleted is an improvement, but this still does not ensure a cache will not blow up over some given allowed size/threshold.

To be able to support such a user set limit support of Cache replacement policies is required - a way to prioritize the deletion of fresh (i.e. non-stale) cache entries to keep the cache under a certain size.

The goal of this new feature is, then, to allow users to provide a size limit for the cache while also optionally selecting one of several cache replacement policies to use, and know that they won't have to worry about their memory/hard disk/databased-backed filling up, resulting in slowdowns, crashes or unexpected costs.

This is also a necessary step for multi-core caching, as setting a hierarchy of caches, as in [memory, pickle, sql], only makes sense if the user can explicitly control the size of higher-priority, faster (but possibly limited in memory) cache cores while also assigning each with an adequate cache retention policy, to ensure, for example, that most cache entries that are requested often are usually in-memory and always on-machine, and that communicating with a database is done only when looking for more rare entries.

Notes:

• We should have a sensible default cache replacement policy, so this argument can be optional, as we can't expect users to really delve into the topic; Least Recently Used (LRU) seems reasonable, since when combined with automatically cleaning of stale cache entries it yields an approximation to the Time-aware, least-recently-used (TLRU) policy).
• A good set to support might be FIFO, LIFO, LRU and Random replacement (RR) - although added entry metadata, indexes (on database cores) and degradation in runtimes should all be considered when deciding which policies to support.
• It should be easy to add future implementations of a new cache replacement policy.
• If and where possible, cache replacement policies should be non-breaking; meaning, if a decorator is provided in a future run with a size limit argument and a cache policy, the cache should still be able to return values cached before this change, while the policy must be able to manage at least any new entry, and hopefully also old ones (or perhaps relegate the retention management to a policy that can always be universally introduced (e.g. FIFO and LIFO cannot be applied to existing caches where entries don't have write timestamps [for example, the memory core], but LRU can always be attached on an existing cache, as retrieval time tracking can start from where the point introduced, treating all existing entries as if they have been accessed some fixed X seconds before the policy was introduced to the cache).

Implementation thoughts and ideas (non compulsory):

• For easier additions of policies, perhaps a BaseCacheReplacementPolicy class should be devised, defining the expected API a policy must adhere to.
• Perhaps policies should get a managed way to write and read both the global (cache-wide) and per-entry metadata , allowing better extensibility, perhaps some support for switching policies and most important - less impact and clashes with the schemas and implementation of specific cores (e.g. we don't want one policy to add one column to the SQL table, and another add two, with different names).