Every once in a while I like to do a review of the state of interactive narrative using LLMs.
In the past few months there has been some progress, but I’m still not sure if we are at a point where we can create unforgettable experiences .
I wonder if this is a natural limitation of LLMs or if we still haven’t found the right way to exploit them.
For those who don’t know how LLMs work: given a series of words (tokens) they determine which is the next word (or token) most likely to make sense, also adding some randomness in between.
My doubt is if the “most likely” approach is so core to the system that the random part will never be able to write very interesting things.
I don’t mean to say that we will never be able to automatically generate interesting interactive narrative, because eventually we will be able to, my question is if the architecture of LLMs is sufficient to achieve this or if we need a different architecture.
It’s not clear that the architecture can generate complex models of reasoning that can surprise us in fundamental ways.
Until we see a new generation of LLMs that can understand essential things like time and space, keep in memory everything that has happened in our story, and maintain coherence, have a sensitivity of our mental models to manage expectations, and other skills that writers use daily, we are still far away.
PS: as a person who likes precise definitions, it’s not surprising that I don’t use the term ‘AI’. In this case I use the more correct term LLM (Large Language Model) which represents deep learning models developed to write text.
Top comments (0)