DEV Community

Johannes Lichtenberger
Johannes Lichtenberger

Posted on

2

In-Memory Performance for Big Data

This paper is really interesting -- pointer swizzling for traditional database systems with a Buffer Manager to compete with in-memory databases for smaller data sets:

http://www.vldb.org/pvldb/vol8/p37-graefe.pdf

Seems I "accidentally" implemented a similar mechanism for https://sirix.org as I'm setting in-memory page references on a cache miss and explicitly set them to null, once the Buffer Manager has to evict entries :-) However as I'm using object instances as keys I can simply set the reference to null instead of using another Cache for a child to parent mapping. I'll see if I can simply cache the Java hashCode values in an instance field to further improve performance a bit.

Have a great weekend :-)

Kind regards
Johannes

Top comments (0)

Great read:

Is it Time to go Back to the Monolith?

History repeats itself. Everything old is new again and I’ve been around long enough to see ideas discarded, rediscovered and return triumphantly to overtake the fad. In recent years SQL has made a tremendous comeback from the dead. We love relational databases all over again. I think the Monolith will have its space odyssey moment again. Microservices and serverless are trends pushed by the cloud vendors, designed to sell us more cloud computing resources.

Microservices make very little sense financially for most use cases. Yes, they can ramp down. But when they scale up, they pay the costs in dividends. The increased observability costs alone line the pockets of the “big cloud” vendors.

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay