JustC

Posted on Jul 18

What Does It Mean to Know?

#ai #computerscience #discuss #learning

This is non-standard and first-principal take on AI. I will try and avoid jargon and stick to intuition.

Knowledge

Let's ask much simpler and very old question

What does it mean to "know" something?

I am not looking for a philosophical answer. Let's say we simply ask

How does a calculator "know" 2 + 2 is 4 not 5?

Calculator does not need to know, It follows some rigid rules.

Now suppose I ask:

How does some recommendation engine recommend a movie or a book to you?

Rules and lots of data.

Now suppose I ask:

Which one is the largest planet in solar system?

The last one seems different, doesn't it?

Which one is answered based on knowledge? All of them? last one? or none?

Intelligence

Let's build our very own intelligent machine. Let's not worry about how to build it, rather let's just focus only on what capabilities it will have.

For example, should it be able to:

Learn
Remember
Forget
Plan
Explain
Make mistakes
, and so on.

There is no right answer.

Let's see if I decide to choose just three

Learn facts
Follow rules
Remember
and remembering is situational so the system must be able to understand situation

Let's stop and recheck the choices if I was onto something..

Learn facts

But,,, What is a fact?

For example,

Third rocky planet from the Sun is Earth
Under standard pressure, water boils at 100°C
My name is X

These are facts.

Now consider

Cats have four legs

Facts?

Well... mostly

Some cats have three

Some are born without one

The certainty has become fuzzy

Now consider:

Cats are mammals That is irrevocably true with no known exception

Now consider:

If someone smiles, they are happy.

Certainly not a fact..

So our intelligent machine can't store facts, isolated facts. It needs to understand degrees of certainty and generalizations.

Second capability in the list, "Follow rules" is equally uncertain too. Rules change as per the situations.

Your intelligent car will follow rules like

Drive on the Left or Drive on the Right (within a region it typically is fixed)

But

Should I overtake now?

leads to rules that are purely situational.

Rules aren't changing, application of the rule is.

Remembering is situational.. Why???

If that is countered as

Who decides what is important?

Let's pick following facts

My name is X
Today is Saturday
I parked the car at Level 0
I am allergic to Peanuts

Tomorrow...

Which one of these facts our intelligent system must remember?

Not all of them.

Why?

Because intelligence isn't just remembering. It is about what deserves to be remembered.

That is much harder than it seems.

Still circling around fact..

Suppose we built two systems

System A remembers everything forever.

System B forgets almost everything, keeping only what seems likely to matter later.

At first glance, System A sounds smarter.

It isn't.

Remembering everything eventually becomes a burden.

The real challenge is deciding what can be forgotten safely.

System A will become impractical soon while System B is weirdly difficult. For example, from the facts that we learned, My name is X, I am allergic to Peanuts seem important enough to remember tomorrow, they are not changing. Where my car is parked, may not be, car may have been moved for various unknown reasons and subsequent facts about the car and the parking lot itself.

That single statement alone holds a significant idea that appears over and over again in AI, databases, and even in human cognition.

Information has no intrinsic importance. Its importance depends on future use

Let me elaborate with an example.

"My name is X"

It can be safely remembered as it does not change. Tomorrow the name is going to be X. But let's suppose the name is name of a waiter, you are talking to, in some cafe, you rarely frequent. Does this waiter or you need to remember each other's names?

Probably NO.

Now suppose the same setting but not a waiter, your family doctor.

Should your family doctor remember your name?

Absolutely. Same fact, different consumer, different retention needs.

Now about the Peanut allergy, that seemed important anyday, not just tomorrow, But why?

It is not because it is permanent. It's because, forgetting it has a very high cost

Importance isn't about permanence. It's about expected future utility

Now we are at a very important juncture where our intelligent systems is presented with just two facts.

X is allergic to peanuts
Food Y has peanuts.

One day X asks

"Can I eat Food Y?"

Nobody explicitly taught a new fact to the intelligent system that "X cannot eat food Y". yet it is expected from the would be intelligent system that it would advise X against Food Y.

Questions:
Where did that new knowledge came from?

It wasn't part of the training set.
It isn't a learned fact.
It wasn't stored in the system.

Essentially the questions itself is where the intelligence really begins. And, that's where modern AI starts to diverge from the traditional programming. The crux of the understanding is answer to the question