DEV Community: Ongkar Dasgupta

Before billion-parameter neural networks, there was a simpler question - can a machine learn from experience? In 1958, Rosenblatt explored it with the perceptron. I went back to the original paper without any AI support to understand how modern AI began.

Ongkar Dasgupta — Tue, 25 Aug 2026 06:47:17 +0000

Ongkar Dasgupta

Aug 25

The Perceptron: When a Machine Could Learn from Experience

#discuss #ai #machinelearning #beginners

5 min read

The Perceptron: When a Machine Could Learn from Experience

Ongkar Dasgupta — Tue, 25 Aug 2026 06:30:46 +0000

The year is 1958. Electronic computers are found in some rooms, punch cards clatter through readers, and the idea of a “thinking machine” belongs more to science fiction than to engineering. One day, Frank Rosenblatt publishes a paper that changes the direction of the field of thinking machines : " The Perceptron: A Probabilistic Model for Information Storage and Organization in the Brain."

At this moment, most machines are built to follow explicit instructions. If you want a device to behave in a certain way, you must design every rule in advance. There is no standard notion of a system that can adjust its own internal parameters based on " experience ". Rosenblatt’s work proposes something different. It proposes a machine whose response to an input can change as it encounters more examples, without requiring every rule to be manually specified.

He calls this machine the perceptron. To explain its organization, Rosenblatt considers a version that responds to visual patterns. An image first falls on a simple artificial retina, made of sensory points that react to the incoming stimulus. These sensory points send signals to units in the projection area. Some incoming signals push a unit toward becoming active, while others push against it. Rosenblatt calls these excitatory and inhibitory connections. We can think of the unit as adding these positive and negative influences together. If the final amount is large enough to cross a certain threshold, the unit fires. Otherwise, it remains inactive. The active units pass signals to another group of units called the association area, and finally toward one of the several possible response units.
Figure 1 (taken from official paper) shows this complete path clearly.

Looking at the figure from left to right, the retina receives the input.
Its signals travel through the projection area A1 to the association area A2, and from there toward the possible responses R1,R2,…,Rn. Please note that the projection area is not much worth for our further discussion, as it is specific to the example mentioned above.

One thing to note here is that Rosenblatt explicitly labels some connections as random. This does not mean that the machine produces random answers. It means that he does not carefully choose beforehand which unit in one area should connect to which unit in the next. Different inputs will therefore activate different, partly overlapping groups of association units. Learning happens later by changing how effectively the active units can influence the responses.

To understand this, imagine showing the perceptron the same kind of pattern several times. Each time the pattern appears, a particular group of association units becomes active and one of the response units eventually responds. If we now reinforce that response - which essentially means telling the system that this is the response we want, something inside the perceptron changes. Rosenblatt gives each association unit a numerical value , and reinforcement changes the values of the units that were active. So when a similar pattern appears again and activates many of those same units, they no longer influence the response exactly as they did before. The response that was reinforced has now become more likely.

Now notice what has happened. Nobody went inside the perceptron and added a rule saying, “When you see this pattern again, choose this response.” The only thing that changed was the numerical values of some association units. Yet when the pattern appeared again, the machine was more likely to respond differently because of those changed values. This is the key learning of this paper - an experience changes something inside the machine, and that change affects what happens during the next experience.

But something even more interesting can happen. The next input does not have to be exactly the same as one seen before. Suppose a new pattern is similar enough that it activates some of the same association units whose values were changed during learning. Those changed units are still there. They can therefore influence the response even though this exact pattern has never appeared before. So now Rosenblatt can ask a much more important question.

Can the perceptron learn from the examples it has experienced and use those changes to respond correctly to a new example?

Rosenblatt studies exactly this distinction mathematically. He considers two values Pr and Pg.

Pr = probability of correctly responding to something already learned
Pg =probability of correctly responding to a new member of the same class

If the perceptron simply remembers its previous examples, we might expect Pr to improve while Pg remained poor. Rosenblatt finds that, as the system learns from more examples, both probabilities can approach the same limit :

Pr⟶ P∗ and Pg ⟶ P∗

where P∗ represents the limiting probability of a correct response.

In other words, the probability of correctly handling a new member of the class can eventually approach the same level as the probability of correctly handling examples seen during learning. In the paper, Rosenblatt further argues that this limiting probability
P∗ can be brought closer to 1 by increasing the number of association units in the system.

This is called generalization. If the perceptron learns from several similar members of a class, whatever it has learned can help it respond correctly to another member it has never seen before. In the future, the distinction between performing well on examples seen during learning and performing well on new examples from the same class is an idea that will remain central throughout the rest of machine learning.

However, Rosenblatt himself also points to a boundary. The perceptron can learn patterns and associations, but the same becomes difficult in problems involving relationships. He gives examples such as identifying “the object left of the square” or remembering “the pattern that appeared before the circle.” For such problems, Rosenblatt concludes that “some system, more advanced in principle than the perceptron” would be required.

This paper essentially teaches us something that now sounds obvious, but was anything but obvious in 1958.
A machine does not have to be given a separate rule for every situation it may encounter. Its internal numerical state can change through experience, and what changes during learning can help it respond even to inputs it has never seen before.

The perceptron could not learn every kind of relationship, but it had established the idea we needed first i.e a machine could learn.

Stay tuned for our next paper in the sequence that carries the AI journey forward !

Rosenblatt's Paper Link : https://web.engr.oregonstate.edu/~huanlian/teaching/ML/2020spring/extra/rosenblatt-1958.pdf

How does a database know where its data is ?

Ongkar Dasgupta — Sun, 09 Aug 2026 13:54:52 +0000

Recently, a quite an interesting research paper titled Virtual-Memory Assisted Buffer Management in Tiered Memory, authored by Yeasir Rayhan and Walid G. Aref came out in the database management ecosystem. Let us discuss about it in this article !

Instead of quickly summarizing what the authors built and listing their results, it would be great to understand the problem itself.

Although we won't go into every implementation detail of the paper, we will try to go deep enough to see why each idea becomes necessary. So, please bear with me in this journey.

We will start from a very ordinary database query, encounter one problem, solve it, see what new problem that solution creates, and keep following the trail. Interestingly, that trail will take us from databases to RAM, virtual memory, page tables, the TLB, and eventually even a modification of Linux's page migration mechanism itself.

Let's begin!

Every time we search for a record, update a value, or run a query on a large database, the processor eventually has to access the corresponding data from somewhere in the machine. Right?

Quite a substantial portion of this database may be stored persistently on a solid‑state drive (SSD), where it can be stored safely for the long term. However, frequent reading from the SSD would be far too slow. So the Database Management System (DBMS), the software that manages this database, keeps the most frequently accessed data in the computer's main memory (RAM), where the processor can access it much faster.

This already creates a very interesting problem.

A full fledged database contains far more data than what can fit in RAM all at once. The DBMS therefore has to keep track of what is currently in RAM, what is still on the SSD, and where the required data can be found when a database query asks for it.

Modern servers make this problem even more interesting because there could be additional kinds of memory between these two extremes of RAM and SSD.

And this is exactly what we are going to explore -
How does a DBMS keep track of data as it moves across these different kinds of memory?

What are these additional kinds of memory?
In modern servers, memory can be distributed across different hardware locations. There would be two such cases that are particularly relevant in this discussion.
Firstly, servers can contain multiple processor sockets, each holding a CPU package with directly attached main memory, typically DRAM (Dynamic Random-Access Memory). From the perspective of a processor running on one socket, its directly attached DRAM is its local RAM. It can also access DRAM attached to another socket, although that access is generally slower.

Secondly, we have another kind of memory access method called Compute Express Link (CXL). CXL is a high-speed interconnect that allows a processor to access additional memory attached through a CXL device, rather than limiting it to the RAM directly attached to the processor itself. One thing to note however is : From the processor's point of view, this additional memory can still be accessed using ordinary memory reads and writes but reaching it generally takes longer than reaching local RAM.

Memory resources of these two kinds including DRAM attached to another processor socket and CXL-attached memory are broadly referred to in the paper as remote memory (RMem).

So at this moment of study, our DBMS has quite a lot of choices. Instead of keeping data either in local RAM or on the much slower SSD, it can use RMem as an intermediate option.

How would you connect all of them ?

Simple. Data that is needed very frequently can stay in local RAM. When local RAM starts filling up, some data can be moved to RMem instead of being pushed all the way back to the SSD. And if that data becomes frequently needed again, it can be moved back into local RAM.
We have essentially gone from a two-level arrangement,' RAM - SSD ', to a three-level management ' RAM - RMem - SSD '.

But what exactly are we moving between these places?

A DBMS does not normally move an individual database record every time it needs one (quite obvious!). Instead, it organizes the database into fixed-size chunks called database pages. A page can contain multiple records, and each page is identified by a Page Identifier (PID).

So when a query eventually needs some data, the DBMS first needs the corresponding page. And because that page could now be in local RAM, somewhere in RMem, or only on the SSD, we finally arrive at a more technical version of the problem we started with in our title:
given a PID, how does the DBMS efficiently find where its page currently is?

One straightforward way is to maintain a Hash Table inside the DBMS. We can think of it as a lookup structure where the DBMS stores an entry connecting each page currently kept in memory to the address through which that page can be accessed. In simplified form, it looks something like:

PID 42 → memory address X

Now suppose a query needs Page 42. The DBMS cannot simply access Page 42 directly. It first takes PID 42, searches the Hash Table, obtains 'X', and only then accesses the page through that address. So before reaching the actual database page, we have introduced an additional lookup:

PID → Hash Table → memory address → actual page

The issue here is that page accesses happen continuously while a database is serving queries. At large scale, repeatedly searching this Hash Table introduces additional costs for the processor.
This raises our next question in this journey: if the purpose of the Hash Table is ultimately to translate a page identifier into a memory address, does the DBMS really need to perform that translation itself?

Well, it goes like our computer already has another mechanism whose entire job involves translating addresses: virtual memory !

When a program uses a memory address, that address is generally not the actual physical location of the data inside a RAM chip. Instead, the program works with a virtual address.

The processor translates this virtual address into a physical address using mappings maintained by the operating system in a structure called the Page Table. In a simplified form, we have something like:

virtual address X → Page Table → physical location A

Why introduce this extra layer at all? Because now the address visible to a program and the actual physical location of its data no longer have to be the same thing.

Suppose some data currently lives at physical location A and the program accesses it through virtual address X. If the operating system later moves that data to physical location B, it can simply update the mapping:

X → A
becomes
X → B

The program can still continue accessing X exactly as before. It does not need to know that the physical location underneath X has changed. And this gives us a very interesting possibility for our next database problem: What if a database page could similarly keep one fixed virtual address even while its actual physical location changes?

This is exactly the idea behind an earlier system called vmcache.

Instead of maintaining a Hash Table that repeatedly tells the DBMS where each database page is located, vmcache reserves a large range of virtual addresses and gives every database page its own fixed virtual address.

Suppose Page 42 is assigned virtual address X. That relationship does not change during the lifetime of the page:

PID 42 → virtual address X

If Page 42 is currently present in RAM, the OS Page Table takes care of the remaining translation:

PID 42 → virtual address X → Page Table → physical location A

The Hash Table lookup issue is solved ! The DBMS knows the fixed virtual address associated with the page, while the operating system's existing Page Table tells the processor where the page actually resides in physical memory ( there are two different scopes now !).

But what happens if Page 42 is not in RAM at all and currently exists only on the SSD? There would be no valid physical RAM location for X to point to. When the program tries to access X, the processor detects that the required page is not currently present in memory and triggers what is called a page fault. This transfers control to the operating system, allowing vmcache to bring the required database page from the SSD into RAM. Once loaded, the OS installs the corresponding virtual-to-physical mapping, and access can continue.

So vmcache already gives us an elegant solution when there are two levels RAM, followed by, SSD.
But Modern servers may also have _RMem sitting between them._

Now Page 42 does not simply have two possibilities, “in RAM” or “on SSD.” It may move like this:

local RAM → RMem → local RAM → RMem → SSD

And this leads directly to the question investigated in the paper we are discussing: Can the fixed-virtual-address idea of vmcache still work when the physical page is continuously moving across multiple memory tiers?

The authors extend vmcache for exactly this setting and call the resulting system vmcacheⁿ, where the n represents the possibility of having multiple memory tiers.

They preserve the same fundamental idea:

The virtual address of a database page remains fixed throughout its lifetime, while the physical memory backing that address is allowed to change.

Sounds like our problem is solved then?

Not quite. We have decided that the virtual address will remain fixed, but the actual page may still have to move between local RAM and RMem. If Page 42 is currently in local RAM and the DBMS decides that it should now reside in RMem, something still has to physically move that page from one memory tier to the other and change the mapping underneath its fixed virtual address.

To move Page 42 from local RAM to RMem, vmcacheⁿ needs to move the actual contents of that page to a physical location in RMem and then update the Page Table so that X points to this new location. The virtual address remains unchanged throughout this process. This movement of a page from one physical memory location to another is called page migration.

Linux already provides a system call called move_pages for doing exactly this. vmcacheⁿ can give it a collection of pages along with the memory location where each should be moved. Linux performs the migrations and updates their Page Table mappings, while their virtual addresses remain unchanged. Even better, move_pages can migrate multiple pages in one call rather than requiring a separate request for every page.

Our next problem got itself introduced here!

Remember that the processor needs the Page Table to translate virtual addresses such as X into physical locations. Looking up the Page Table for every memory access would itself be slow, so processors keep recently used translations in a small and extremely fast cache called the Translation Lookaside Buffer (TLB).

Suppose the processor has cached:

X → physical location A

vmcacheⁿ now migrates our page to RMem and the Page Table becomes:

X → physical location B

The Page Table is correct, but the processor may still have the old X → A translation sitting in its TLB. That cached entry must therefore be invalidated before the new mapping can safely be used. And if several processor cores have cached the same translation, the operating system may need to make all of them discard it. This operation is known as a TLB shootdown, and it is expensive because it can require interrupts between processor cores.

Now imagine vmcacheⁿ continuously moving large numbers of database pages between local RAM and RMem. These invalidations start adding up. Linux's move_pages already groups page migrations into batches, but by default it processes at most 512 pages in a migration batch. The authors realized that if more pages could be migrated together, the cost of TLB invalidation could be spread across a larger number of page movements.

This observation led them to modify the Linux page-migration interface itself. They introduced a new system call called move_pages2, which lets vmcacheⁿ control how many pages are grouped into a migration batch. It also handles failures more flexibly. Instead of certain errors on one page which makes the remaining pages in the request to be skipped, move_pages2 can record that failure and continue attempting the other eligible migrations.

And surprisingly, implementing move_pages2 required only about 150 lines of changes across four Linux kernel functions.

Our question now is: does all of this actually make the database faster?

To find out, the researchers used a server with two processor sockets. RAM attached to the processor running the workload acted as local RAM, RAM attached to the other socket acted as RMem, and a 960 GB NVMe SSD formed the final storage tier. So their experimental machine essentially gave them the same hierarchy we have been discussing:

Local RAM → RMem → SSD

The amount of local RAM was restricted to only 32 GB, forcing the system to decide what stays locally, what moves to RMem, and what eventually has to be obtained from the SSD.

The researchers then used two workloads. The first was TPC-C, a standard benchmark that simulates transactions performed by a real database system, such as placing orders and processing payments. The second was a random-read workload, where the system repeatedly requested data scattered across a large dataset. TPC-C shows whether vmcacheⁿ improves the system as a whole, while random reads make page migration between local RAM and RMem much more prominent, allowing move_pages2 to be examined more directly.

Let's focus on TPC-C first.

With TPC-C, vmcacheⁿ achieved 1.67× higher throughput than vmcache with 2× RMem, increasing to 3.82× with 4× RMem. More pages could remain in RMem instead of requiring slower SSD accesses.

But this improvement came from vmcacheⁿ as a whole, not specifically from move_pages2. For TPC-C, move_pages2 performed similarly to the existing move_pages because SSD accesses still accounted for a substantial part of the workload.

The difference became clearer with random reads, where movement between local RAM and RMem mattered much more. Here, move_pages2 achieved 1.42× higher query throughput and 1.32× higher page-migration throughput than move_pages. Its larger migration batches allowed the cost of TLB invalidation to be spread across more page movements.

However, the experiments revealed one important limitation:

Adding RMem does not automatically make the database faster.

Moving pages between local RAM and RMem has a cost of its own. When RMem was small, this migration overhead could outweigh the benefit of avoiding SSD accesses. As RMem capacity increased, keeping more pages in memory became increasingly worth the cost of moving them.

And that finally gives us an answer to the question we started with:

How does a DBMS keep track of data as it moves across different kinds of memory?

In vmcacheⁿ, the DBMS does not need to explicitly translate each PID into its changing physical address. Each database page is given a fixed virtual address, while the operating system's Page Table keeps track of which physical memory currently backs that address. The page may move from local RAM to RMem and later back again, but the address used by the DBMS remains unchanged.

The authors then solve the second problem created by this approach: moving those physical pages efficiently. Their move_pages2 system call gives vmcacheⁿ greater control over how migrations are batched, helping minimise costs such as TLB invalidation when page migration becomes a major part of the workload.

The interesting idea behind this paper is that the researchers did not eliminate the cost of managing data across different memory speeds. They moved part of the tracking problem to a mechanism the computer already has, and then identified page migration as the new bottleneck created by that decision.

Paper Reference: Rayhan, Yeasir, and Walid G. Aref. "Virtual-Memory Assisted Buffer Management In Tiered Memory." In Proceedings of the 22nd International Workshop on Data Management on New Hardware, pp. 1-5. 2026.

Paper Link: https://arxiv.org/abs/2603.03271

Trying to solve a frustrating problem: why is debugging AI systems still so unclear?

Ongkar Dasgupta — Wed, 18 Mar 2026 11:50:59 +0000

I’ve been thinking about something that keeps coming up when working with AI systems. When a model gives a wrong or weird output, it’s surprisingly hard to figure out what actually went wrong. Most of the time we’re digging through logs
or guessing where things broke.
As part of the AWS AI Ideas hackathon, I started building a concept called AutopsyAI. The idea is to look at the full pipeline from input to output and try to explain where things might have failed, instead of just showing the final result. It’s still early and not a full product yet, more like an attempt to explore whether this problem is worth solving in a structured way.
Curious how others here deal with this. If you’ve worked with AI systems in production, how do you debug failures today? Does this feel like a real problem, or is it already solved better than I think?

Please do checkout my article and support it!
https://builder.aws.com/content/3AeXXMtLdDuPwRL4xcEjJEBoQkB/aideas-autopsyai-the-missing-debugger-for-ai-systems