DEV Community

Ulrich Kautz
Ulrich Kautz

Posted on Originally published at ulrichkautz.com on

I failed my own Turing Test

I have a lifelong affliction that has often massively frustrated me and certainly has held me back in countless ways over the years. Here it is: I suck at short-form writing. Or so I thought, until I ran into a conundrum last week. Note: this doesn’t mean I don’t suck, but if I do, then for other reasons than I thought. And that is freeing!

What I thought, or rather observed, was that I was entirely unable to write any short text about any topic I wanted to write about. I came to each text with an idea, usually an observation I made or a question that plagued me for which I found an answer I liked to share. Something that surely could be expressed in a few hundred words over the course of an hour or three. Or so I thought. What then inevitably happened, each time, was that the text blew up; both in space - thousands of words - and in time - weeks, sometimes months. Each time! It vexed me, but I simply couldn’t help myself and I didn’t know why! Until now, that is.

My newly gained perspective emerged from a somewhat distant place, yet closely related. As an engineer, I love to build. I love to find technical solutions for all the problems that I face - just because. In that spirit I was recently building a custom AI integration designed to ingest a person’s whole writing corpus - both content and style (if you must know: RAG + stylometrics behind MCP server). Armed with this corpus the AI is then enabled to emulate the person’s voice while drawing on their opinions, beliefs and knowledge to really imitate them as closely as possible. Sounds somewhat scary, but is not nefarious at all. Quite the opposite: it’s a prosthetic, a tool to help the likes of me to express themselves in their authentic voice when they are otherwise not able to. Be that for a strange ineptitude to write short-form texts, or social anxiety to share their thoughts, or overwhelm when using social media, or any reason you can think of.

In that process I came to a point where I had to fine tune my tool. My approach was a simple, blind experiment: a series of triplets of writing examples. One taken from my actual texts, one generated by the AI based on my integration and one generated by AI without any of my influence. Only the topics matched - roughly.

And here it got hairy: I could not tell them apart. Not at all! That absolutely stumped me. I should be able to recognize at least a text I wrote myself in the past, no? I mean, there are only so many options. Let’s say my integration indeed worked as advertised. Then it might “fool” me - that was the point - and I would not be able to tell it from my original writing. If, on the other hand, my integration was bad, then I might not see a difference between the texts generated with and without my voice. Or I might see three entire different styles. Still, there should always be at least one text that peeks out: either my authentic one, or the generic one. Neither was the case for me! Mind you: my writing problem forced me to spend countless hours of detailed research and long thinking on how to condense that into individual sentences that really express what I mean. How could I not recognize these products of weeks, if not months of sweat and tears?

That was surprising, to put it mildly. It also got me thinking. Well, investigating and then thinking. What I found is that there is a large body of research in cognitive psychology about the topic of text comprehension (Kintsch, van Dijk). It segregates the comprehension into three distinct levels:

  1. Surface Code: The How. The exact words; remembering a poem or a text passage verbatim
  2. Textbase: The What. The distilled information; knowing the exact facts of what was written.
  3. Situation Model: The Why. The underlying model, abstract concept; making sense of the writing.

Not everyone, it turns out, excels at all the levels equally. I am terrible at (1), ok’ish at (2) and excel at (3). That means if I read a text, I get what it’s saying, I mostly remember the points that were made, but I have not the foggiest as to the precise words. Or even shorter: I discard the envelope, but retain the message. This is why I cannot even tell my own writing apart from a machine generation - as long as the structure of the points being made is similar. Yes, that was the case. That was my only instruction to the text generation: “make this point”.

Learning this about myself was freeing. It matches flawlessly with my whole life experience. There are countless examples, starting in school never being able to recite any poem or text. I remember being baffled how friends - who were otherwise much better in school than I - told me they had to memorize math formulas to use them in the tests. Math was my only saving grace, but not for my ability to remember, but because the models (as I would call them now) made sense, and once I got them it was effortless to apply them when the patterns matched.

So how did that connect to writing short stories? Well, it didn’t. Or not immediately. All I knew at this point was that I understood something about my ability to read, that also frustrated me throughout my life, that wasn’t just me “sucking at reading”. I am actually pretty good at reading, just not in the way I measured it, which was: exactly remembering passages.

Understanding this felt empowering. I mulled it over and over in my head in the following days. I shared my insight with my wife. It took a while for the coin to drop: The Why. My brain’s happy place is “The Why”. I may want to share an anecdote, a short “aha” I experienced while tinkering. Then I start writing it down. Then something happens: My brain kicks into gear, and steers me into the direction of its feel-good place: let’s research what the related field of sciences knows about that. Let’s put that experience into a bigger perspective of the whole world. Let’s build a definite ontology around it. And don’t forget a practical framework, that can be used for all future cases for everyone everywhere and forever!

Well, it’s not that bad - but close. And yes, that shoe fits me perfectly. I want to wear it. I feel safe and giddy wearing it. Yet, it kept me from writing short stories, because any short-form I ever attempted simply devolved into a research paper or essay - most of which never saw the light of day, as that got me into a gazillion tangents, deep angst of “not getting it fully right” and just provided weeks or months of surface go astray or otherwise find a reason not to finish.

So here we are. I am writing a short-form text, yet I am drawing (almost) only on memory. I am using even a story arc, of sorts. I wrote it in hours, not a day, not a week, least of all a month. Knowing about myself how I comprehend text, and subsequently how it steers me in writing helped me to now see clearly when I veer off into essay territory. Not that I never want to write long-form again, quite the opposite, but I rather have the choice.

Of course, it wasn’t that easy and straight forward. There was much more reflection, discussions with my patient wife and, yes, research in between my insight and this article. But the cat is out of the bag, to quote a German proverb. You may be the judge of the outcome. Sure, this article is not as short as I wanted it to be and, yes, I could not help but investigate at least a little bit while writing. However I am sure it’s as with everything: it’s a journey. I just found my starting point.

Either way: how do you understand text and what keeps you from publishing all you want?

Top comments (0)