Connections

In the last episode What does it mean to KNOW I meandered through the idea of knowledge, purposely just glancing over what knowledge is or how knowledge is represented.
Let's bring back the example scenario from earlier episode expressed by just two facts.

1. X is allergic to peanuts
2. Food Y contains peanuts

When presented with these facts, humans, the intelligent being, are expected to derive with a NO to a simple question. We have already established that a basic computer can't. Let's elaborate with very simple database with just two records

Person:
  Name: X
  Allergy: Peanuts

Food:
  Name: Y
  Contains: Peanut

A basic computer with such basic database simply cannot infer a new knowledge that X should not eat Y. Such a rule is not programmed.

Now let's imagine some system, the way we did in earlier episode.

System A: Stores everything rigorously

Person:
  Name: X
  Allergy: Peanuts

Food:
  Name: Y
  Contains: Peanut

This system can not be used. Too basic..

System B: Same information but as relationships

X -> allergic to -> Peanuts
Y -> contains -> Peanuts

This system, system B is a progress but still fails a little tangential queries as, can X eat at "some" place? there aren't any relations about "some" place.

System C:
System C doesn't explicitly save relationships. Instead, it organizes information so that related ideas naturally end up close to one another.

By the way, possibilities are endless beyond these 3 systems.

From get-go "System C" seems a lot so let's just focus only on how such system can make sense of data.

Let's work out an example.

Let's suppose we have a contact list

Alice
Bob
Charlie
David

From the list alone can you answer, is Alice friend of David?

Nope. That's System A

Now let's have the contact list augmented with relationships

Alice -> Friend -> Bob
Alice -> Coworker -> David

Same question is answered easily, Alice is NOT a friend of David.
That's System B. Knowledge is explicitly maintained as Connections and all this is codified in a way. This system still can't infer anything that isn't explicitly mentioned.

Now imagine completely different system.

Nobody is saving the connections, the relationships.

Instead, the system, somehow, observes millions of conversation and discovers

Alice and Bob are often mentioned together.
Bob and Charlie are often mentioned together.
Charlie and David are often mentioned together.

Nobody ever created these relations explicitly.

Yet... the system starts behaving as though these relationships exist.

That's System C where relationships are implicit.

One more analogy to strengthen the idea.

A map has roads explicitly saved, that's System B. Whereas some other system watches thousands of travellers from city A to another city B and never to the moon. This system infers that there is a road from city A to city B but there are no roads to moon. That's System C.

Modern AI is surprisingly closer to System C. It does not have millions of connections, the edges, the relationships, the rules, etc. explicitly.

Instead after going through billions of documents, it develops internal structure where those relational concepts become strongly associated.

That raises an interesting question. If the relationships themselves aren't stored, what exactly is? What does the shape of that knowledge look like?