Opening one form means the framework reads more than one definition: FormSchema, TableSchema, FormLayout, ProgramSettings and LanguageResource, none of them optional. These definitions still change after go live, but they change rarely while the system is running, and data like that is the best fit for a cache. Everything left is the cache's own problem: which data fits, how finely to slice it, how long to keep it, when to release it, and how to swap in a new copy once the source has been edited.
This post covers four things, of which the notification mechanism mentioned in Why ERP Needs a Definition-Driven Framework is the third:
- What goes into the cache, and in what unit
- How a cache knows its source has changed
- What the notification table looks like, and who writes that row
- Deciding that something really changed means looking at the version, not the time
The first decides whether the cache is worth it. The other three decide whether it quietly gives out wrong answers.
1. What goes into the cache, and in what unit
The pain: in a system with a thousand definitions, neither extreme works.
- Reading fresh on every request hands back all the cost saved by everything up to here
- Loading the lot at startup means memory billed against the total volume of definitions
The framework does not choose between the extremes. It makes the question smaller: when this thing is read, is what gets read the whole of it, or one of them?
| Base class | Use for | Key |
|---|---|---|
ObjectCache<T> |
The whole thing is one object, and the system has only one | Type name |
KeyObjectCache<T> |
There are many of one kind, and each read takes one of them | Type name plus that one's key |
Thirteen kinds of definition split in two along that line:
- Single object, nine of them:
SystemSettings,DatabaseSettings,DbCategorySettings,ProgramSettings,MenuSettings,PluginSettings,PermissionModels,CurrencySettings,UnitSettings - Keyed, four of them:
FormSchema,TableSchema,FormLayout,LanguageResource
The Purpose and Classification of Definition Files said the unit of caching is the unit of reading, and this section is the implementation side of that. A keyed cache only has to state one thing: how to read the item by that key when there is no hit. FormSchemaCache is exactly that shape:
public class FormSchemaCache : KeyObjectCache<FormSchema>
{
protected override FormSchema? CreateInstance(string key)
{
string progId = key;
return _storage.GetFormSchema(progId);
}
}
There is no warmup and no load list. Whichever form the first person opens is the form whose definition enters memory. How long it stays is decided by how often it is used, since the cache expires on a sliding basis, restarting the clock on every read, so the most read copies stay longest and the cold ones leave by themselves.
Forms pointed at from everywhere, such as employee and department, land at the hottest end. Someone opening one reads its FormSchema once, and another form's relationship pointing at it reads it again while the query is assembled, so during a run they barely ever go cold.
What happens one notch coarser: suppose every FormSchema went into one object. On the reading side there is no difference. The cost appears on the other side, where anyone editing any one form invalidates every form's definition at once, and they all get read again.
A cache's granularity is not only how much gets read at once. It also decides how much gets thrown away at once.
The cache holds more than definitions
The framework's cache holds another kind of thing, whose source is not a definition file but actual data in the database: a company's basic details, company level role grants, and a department's organization tree. The shape is identical to the definition caches, reading the source once when there is no hit.
Their reason for staying in memory is the same as the definitions', with one more on top. The stretch of the path that uses them should not be opening another query, since deciding whether a call has permission and deciding which database this one connects to both happen at the very front of a request.
The same goes for the application's own data. There is plenty in an ERP that changes rarely and gets queried on every request, such as tax rates, warehouses, approval levels and assorted parameter files, and going back to the database each time is not worth it. Those two base classes are not for the framework's exclusive use, and the copies an application wants take the same route.
Record the misses too
There is another pain that has nothing to do with hits. Someone keeps calling in with a ProgId that does not exist, missing every time, and so reading a file that is not there every time.
The keyed base class records the misses as well, so the same key afterward comes straight back as absent. That marker uses absolute expiry rather than sliding, so poking the same key repeatedly does not extend its life.
There is a counterexample too, one cache that deliberately turns this behavior off. Its keys are ones anybody can send in themselves, so recording misses would leave a marker in memory for every one of those keys, and what it saves is one index seek returning zero rows. Whether the default should apply depends on who decides that cache's keys.
Why it stays in the process's own memory
This is a question with reasons on both sides, and this series does not take one: does the cache live in each process's own memory, or on a shared cache server.
What this layer of the framework needs is specific. Definitions are read only, several get read on every request, and what is read is one shared instance. Putting an access pattern like that outside the process means adding several network round trips to every request, in exchange for something that was not going to change much anyway.
The framework does not nail this down either. The underlying storage is an interface of five methods (Contains, Set, Get, Remove, GetCount), and replacing it needs no change to any cache class. The one implementation shipped with the package wraps .NET's MemoryCache, though, so swapping in another is room left open, not a feature already there.
A website has several nodes sharing the traffic, and the number goes up and down with the traffic automatically. There are also background services that take no requests, running schedules and batches of their own. One node is one process, and they read the same set of definitions.
Editing a definition only ever happens on one of those nodes. That one is easy, since clearing its own cache on the way past finishes it. Nobody touches the old copy in the hands of the other nodes, and they carry on assembling SQL, rendering screens and running validation rules to the old shape, without a sound.
Every node holds its own copy, so every node has to know for itself when to throw it away.
2. How a cache knows its source has changed
The pain shows up during development: edit a definition file, refresh the screen, and what you see is still the old one. One process is enough to produce that, and multiple nodes only make the same thing larger.
That same post said each cache watches its own source, and this section unfolds it. "Source" is not one kind of thing, though. It might be a file, or it might be one or several tables in a database. The former has a modification time to look at, and the latter does not.
The framework's division of labor is to have storage report what signal it can provide, and have the cache side turn that into an expiry policy of its own. That step is in another method of the same FormSchemaCache:
var changeSource = _storage.GetChangeSource(DefineType.FormSchema, progId);
policy.ChangeMonitorFilePaths = changeSource.FilePaths;
policy.ChangeNotifyKey = changeSource.NotifyKey;
With definitions in files, what storage reports is a set of file paths. With them in the database, what it reports is a notification key. The two are never both set. Sometimes neither can be given, and that entry simply waits to expire on time. The cache side does not have to know which kind it got, and watches whatever storage gives it.
The signal is reported by storage, not guessed at by the cache.
Have the cache guess instead and every cache class has to write out where its things live and therefore what it should watch, and the day one more kind of storage appears, every one of them gets revisited.
Runtime is multi-node, so definitions have to go into the database
Where the processes above sit takes one of two shapes, and the shape decides which signal is available.
The file kind is the development shape. Definition files belong in version control and have to follow the branch, and an individual environment shares one definition directory on one machine anyway, so when somebody edits a file, the next read in any other process sees it.
An actual runtime environment is usually several machines. Each looks at its own file system, and editing the file on one leaves the others none the wiser. This is not slightly less efficient, it is receiving no signal at all. So definitions move into the database and invalidation switches to the notification table, at the cost of no longer being in version control.
That line only cuts the definition half. The caches in the first section whose source is the database have no file to watch, and take the notification table wherever definitions live. So the last two sections are both about the notification table route, and that route is present in any deployment shape.
What expiry conditions hang on one entry
That expiry policy, taken apart, is four conditions deciding how long an entry lives. They fall into two categories, and each category is one of two:
| Category | Expiry condition | How it is decided |
|---|---|---|
| Time | Sliding expiry | Thrown away only after a stretch with no read, with a read restarting the clock |
| Time | Absolute expiry | Thrown away at that moment, with reads not extending it |
| Change signal | File modification time | The modification time at creation is recorded, and anything different expires |
| Change signal | Notification key version | The version number at creation is recorded, and anything different expires |
Definition caches use sliding, and that miss marker uses absolute. The two change signals depend on which one storage reports.
Comparing a change signal only happens when the cache is used, since nothing comes along to push. So invalidation does not reload, it only makes the next read fetch the new value, and for keys nobody is reading, a source changed a hundred times does not send anything running a hundred times on their behalf.
3. What the notification table looks like, and who writes that row
The section above said the database kind of signal is a notification key. Where that key is recorded and who increments its version is what this section finishes.
| Column | Content |
|---|---|
cache_key |
The notification key, which is also the primary key |
cache_version |
How many times this key has been changed, incremented each time |
sys_update_time |
The database server's time, used for incremental fetching |
The mechanism itself is two steps. When the source data is changed, that stretch of business logic writes a row to the notification table on the way past and increments the version. Every node has a poller watching that table, and on seeing a version go up, invalidates the caches hanging on that key.
There is no routing table in between. The shape of a notification key is a convention, a group plus an entity, such as FormSchema:Order. The group is the name of the type being cached, so the key itself already says who this one is for, which means adding a cache requires registering it nowhere.
There is one notification table, sitting on the common category from Connections, Transactions and Query Conditions in the Data Access Layer, and that is what the poller watches. Source data on the company side still writes its version incrementing row back to this one table.
Why that row has to be written by a person
Remembering to increment the version after changing data is a line written by whoever writes the business logic. Seeing a shape like that, it is easy to think of letting the database do it: hang a trigger on that table, and there is nothing to write at all and no way to forget.
That route was not taken, because the unit of the dependency is wrong. What a cache depends on is a meaning, not a whole table:
- One company record gets ten columns edited, eight of which have nothing to do with the object in the cache
- Conversely, one meaningful change may touch three tables at once, while the cache only needs telling once
All a trigger can see is that this row of this table changed, and it cannot express the granularity of "this change means something to a particular cache", which only the person writing that business logic can judge.
The cost is that the line gets forgotten, with the symptom being other nodes holding an old value indefinitely, and the more often that copy is read the less likely it is to leave by sliding expiry. That cost is taken deliberately: forget to notify and only that one cache is wrong, while notifying too broadly makes every unrelated edit send every node to read again.
4. Deciding that something really changed means looking at the version, not the time
The notification table already has an update time on it, so surely comparing that tells you whether something changed. Three places break:
- One time value can hold several changes
- The update time written may be earlier than the moment that change actually became visible to anyone else
- Every node's clock is its own
What the framework does is separate two things:
Time is only there to fetch the increment cheaply. The version makes the call.
That update time column is indexed, so each round the poller fetches only the rows whose time is after where it last got to, and deliberately looks back a stretch further. Once fetched, the decision ignores time entirely and looks only at the version, acting only when it is greater than the one last recorded.
- Looking back a stretch further is what stops rows on the boundary being missed, the ones whose update time was written but which became visible later
- Requiring the version to be strictly greater is what stops that look back producing any duplicate action
Without the first there are misses, without the second there are duplicates, and neither reports an error. They only make the behavior right sometimes and wrong sometimes.
The first round only records the starting point and invalidates nothing. A freshly started process holds an empty cache, and the history in the notification table means nothing to it.
The clock question needs one fact only. The time written, the position each node records, and the threshold used for fetching all come from the database server, and not one of them from the node itself.
The polling loop also wraps each round, so a momentary database outage records one warning and carries on. Once that loop ends because of a single exception, invalidation is not delayed, it is stopped for good, while the whole process goes on looking perfectly healthy.
What the cache buys
The whole of it comes down to one sentence: the memory in the server's hands is spent only on the definitions somebody is actually using.
- The unit of caching is the unit of reading, so memory follows actual usage rather than the total volume of definitions
- With no warmup and no load list, a definition enters memory only when the first person opens that form
- How long it stays is decided by frequency of use, with the most read staying longest and the cold ones leaving by themselves
- Invalidation does not reload, so keys nobody is reading send nothing running for nothing
Those four together mean nobody has to plan the memory allocation in advance. The cost is on the other side. For the ones whose source is the database, an edit has to update the notification table deliberately, or other nodes will not get the latest value.
The next post, Session Management and Lifecycle, changes layer. Before business logic does anything it has to know who is calling and which company they are in, and those two answers have someone preparing them, and a lifetime of their own.
๐ Part of the thirty-post series ERP Architect's Notes: Designing a Definition-Driven Framework
๐ Original article on HackMD, in Traditional Chinese:
๐ https://hackmd.io/@jeff377/day-12-cache-invalidation
๐ข You are welcome to republish this, please credit the source.
๐ฌ Follow for more engineering notes:
Facebook ๏ฝ HackMD ๏ฝ GitHub ๏ฝ NuGet
Top comments (2)
์บ์์ ๋จ์๋ฅผ ์ฝ๋ ๋จ์๋ฟ ์๋๋ผ ๋ฒ๋ฆด ๋จ์๋ก ์ค๋ช ํ์ ์ ์ด ๋ช ํํด์. ํนํ ์ ๋ฌด ์๋ฏธ์ ๋ง์ถฐ ์๋ฆผ ๋ฒ์ ์ ์ฌ๋ฆฌ๋, ๋๋ฝํ๋ฉด ์์ฃผ ์ฐ๋ ์บ์๊ฐ ์ค๋ ์ด์๋จ๋๋ค๋ ๋น์ฉ์ ํจ๊ป ์ ์ด์ฃผ์ ๋ถ๋ถ์ด ์ข์์ต๋๋ค. ๋ฐ์ดํฐ ๋ณ๊ฒฝ๊ณผ ์๋ฆผ ๊ธฐ๋ก ์ฌ์ด์์ ์คํจํ๋ ๊ฒฝ์ฐ๋ฅผ ํตํฉ ํ ์คํธ์ ๋ฃ๊ณ ๊ณ์ ์ง๋ ๊ถ๊ธํด์. ๊ฒน์น๋ ์กฐํ ๊ตฌ๊ฐ๊ณผ ๋ฒ์ ๋น๊ต์ ์กฐํฉ์ ์งํค๋ฉด์, ์๋ฆผ ์์ฒด๊ฐ ๋น ์ง ๊ฒฝ์ฐ๋ฅผ ์ด๋ป๊ฒ ๋ฐ๊ฒฌํ ์ง๊น์ง ๋ณด๋ฉด ์ด์ ๊ธฐ์ค์ด ๋ ์ ๋ช ํด์ง ๊ฒ ๊ฐ์์.
Thanks, both questions point at real gaps. Neither is covered in Polhem today.
Planned fixes: