The Shape of Knowledge

In earlier episode I said the relationships are not explicitly stored, then what is stored? how is it stored?

Let's come up with some simplest strategies to explore how it might have been done. We need an example to start with, to make it possible to visualise.

Let's suppose we have three words

Cat
Lion
Apple

If I ask which two are similar? It is easier for humans, we have an advantage of explicit knowledge.

Hmm. How about we number these words

Cat = 94
Lion = 24
Apple = 344

Still nothing.. Just numbers, the numbers still don't tell anything about similarity, does it?

We need something that means Cat and Lion similar or literally closer than Cat and Apple.

See similar can in fact mean distance geometrically.. and the distance starts representing some sort of meaning.

It just so happens that, after decades of trying symbolic rules, logic, expert systems, and hand-crafted knowledge graphs, researchers found that geometry was an unexpectedly powerful way to represent knowledge.

Let's start with simple exercise.

One million words. The computer understands only numbers. Design a representation that somehow captures meaning.

Let's try some arrangements for this data.

How about "Tag Cloud" where words are tagged with a meaningful tags, not exactly numerical grouped but grouped nonetheless. distance here is not numerical, yet it is able to express what words are closer.. does it?

Words can belong to more than one tag cloud making the visualization a tad difficult. Tag itself it a word, where does that belong? Hmm.. we have a recursive representation. Is it good or bad? For humans maybe difficult but not for machines. Stil is it okay from machine perspective?

Let's illustrate with simple example.

Animals
-------
Dog
Cat
Wolf
Lion

Pets
----
Dog
Cat
Mouse

Wild
----
Wolf
Mouse

Food
----
Apple
Rice

Cat and Dogs are Pets and also an animal

Idea seems to be working.. without explicit rules we have expressed meaning through shared membership.

Now ask the earlier question again. Is cat more similar to lion or an apple? It is easier to say lion now as they would share more and more tag clouds..

Now imagine each tag exerting small pull. In this world, a cat is being pulled towards Animals cloud, as well towards felines

Cat is pulled simultaneously by:

• pets
• felines
• animals
• four-legged
• etc.

Likewise Lion is pulled by:

• animals
• felines
• four-legged
• wild

Now cloud is not a cloud but a vast landscape. Numerical values are not even primary cursors here, you just need to feel the distance by imagining the pull on the word.

Even though idea seems to be working, let's reassess the use of words as tag.

Our first cloud is Animals.

Animals
-------
Dog
Cat
Wolf
Lion

But but the word Animal itself belongs to another cloud

Living Things
-------------
Animals
Trees
Plants

But what is "Living Things", it belongs to another cloud

Biology
-------
Living Things
Cells
Virus ???

This can go on forever.

This is recursive semantics.

We humans, somehow, know when to stop going down this rabbit hole. Not so for computers unless it is rigidly implemented.

So let's forget word labels and imagine Cat, Dog, Lion, etc. being some point in a space. Somehow Cat and Lion land closer to each other than Cat and Apple. The proximity is expressed as distance between the points.

Loosening up criteria of the tag being a word and it being a number has lots of benefits. Numbers are numbers there is no hidden meanings that forces the semantic recursion.

Using numbers has one more obvious advantage.. now you don't need pre-create tags for categorization.

Now we are heading somewhere.. See proximus words form a cloud but what defines the measure of closeness. Do we even know the cloud boundary???

That's The Question and let's tackle that in next episode.