DEV Community: Richard Rodger

The Voxgig Podcast Chatbot: Triggering Ingestion, and some Debugging DX

Richard Rodger — Fri, 22 Mar 2024 14:45:10 +0000

This is the third post in a series I'm writing about a new Minimal Viable Product we've released at Voxgig that turns your podcast into a chatbot. Your visitors can now ask your guests questions directly! The first post is here: Building a Podcast Chatbot for Voxgig and you find the full list at the end of this post.

The problem with podcasts is that they tend to have new episodes. This is annoying because you can't just ingest all the episodes once and be done with it. You have to set up a regular process (once a day is sufficient) to download the podcast RSS feed, check for new episodes and changes to old episodes, and run part of the data ingestion process again.

To model this business logic, the system uses two separate messages:

aim:ingest,subscribe:podcast: subscribe to a podcast, called only once to setup the podcast in the system
aim:ingest:ingest:podcast: actually process each episode and ingest the content

The ingest:podcast message is the one we will run on a daily basis. For the moment, let's ignore the devops side of this system (it's still under development at the time of writing), and focus on processing the episodes of a podcast.

The podcast RSS gives us the list of episodes, so we need to download the RSS to get the latest version. This does mean that on the initial subscription, in our system, the RSS gets downloaded twice. We can easily solve this problem by adding a parameter to the ingest:podcast message (and we may yet do that in future), but for now, we are deliberately not going to solve this problem. The other more important parts of the codebase to work on. Downloading the same RSS twice to complete a one-time business process is not exactly a fundamental issue. Let's invoke "premature optimization is the root of all evil" and leave it at that for the moment.

This little decision lets us keep the ingest:podcast message implementation simpler for now. Let's look at the code, but this time also think about error handling. This message assumes we have already subscribed to a podcast in our system. That, for the moment, just means we saved an entry in the pdm/podcast table.

Since we have abstracted away the database (see the previous post in this series), we'll use the term table to refer to a generic idea of an entity store that can persist records that look like JSON documents, and can query these records, at least using to level properties. That's pretty much all we need. We use the convention entity-base/entity-name to describe these entity stores and give them some namespacing.

The podcast entry must exist, so we need to know its ID. This must be a parameter to the message action, let's call it podcast_id. We try to load this podcast and need to respond with an error if it does not exist.

Is this error a system error, one that is in some way fatal to the process, or would leave to corrupted data? Nope. This is an ordinary everyday business logic bug. Somehow a podcast ID got sent to us that isn't correct. We just reject it, but there's no need to panic. Thus, we don't throw an exception, or return a bad HTTP error, or do anything out of the ordinary. We just respond with a JSON message that indicates, by a convention of its schema, that there was a business problem.

The general schema that we use for message responses (when there is one!) is:

{
  ok: boolean,  // If true, message was accepted and processed successfully
  why: string,  // If ok is false, optionally supply a free-form debugging code useful to humans
  // Other properties, perhaps well-defined, providing additional response data
}

Let's put all this together in code:

async function ingest_podcast(this: any, msg: any, meta: any) {
  const seneca = this
  const debug = seneca.shared.debug(meta.action)

  const out: any = { ok: false, why: '' }

  const podcast_id = out.podcast_id = msg.podcast_id

  debug && debug('START', podcast_id)

  let podcastEnt = await seneca.entity('pdm/podcast').load$(podcast_id)

  if (null == podcastEnt) {
    out.why = 'podcast-not-found'
    debug && debug('FAIL-PODCAST-ENTITY', podcast_id, out)
    return out
  }

  // Process episodes here...

  return out
}

It's useful to have a "debug" mode for your microservices, that can produce additional log entries for debugging, both locally and when deployed. Some log entries might involve additional work to generate, so you want to avoid that when running normally.

Also, it's tedious to keep adding context information to each log entry, such as the name of the microservice, the name of the message action function, and so on. Thus we make use of a shared utility that provides a debug function: seneca.shared.debug, and we pass in the current message details (the meta parameter) so it can generate a full log entry.

If debug mode is not enabled, seneca.shared.debug will return null, so we can use that to short circuit any costly log entry code, using the idiom:

debug && debug(...)

We'll cover shared utilities in a later post, but if you want to look at the code, review ingest-prepare.ts.

Our code tries to load the podcast, and if it is not found, bails immediately by returning the response:

{
  ok: false,
  podcast_id: '...',
  why: 'podcast-not-found'
}

We do a few things here to make debugging easier. When deployed, debugging becomes harder, so the more information you can give yourself, the better.

As well as the expected ok and why properties, we also include the podcast_id parameter in the response. In a system under high load, you might only see the body of the response in the current section of the log that you're looking at, so give yourself as much context as possible.

For debug mode, we also emit a log entry, and add a test prefix that is easier to identify or search for: "FAIL-PODCAST-ENTITY". Ultimately you'll want to look at the line of code that caused the problem, but having a unique string is a great help when isolating the flow of the error. It also helps a great deal with working remotely with colleagues. This is a poor substitute for proper error codes for sure, but is a classic 80/20 solution that can be improved as needed.

You'll also notice some defensive pessimistic code here. We assume the message will fail, and initialize ok: false. This also helps us be lazy developers, as we only have to set the why property when returning early due to an error.

Most error conditions in this codebase are handled in the same way, and you can see them in the source code. We'll omit them in most cases it the rest of this series to keep the example code shorter.

Let's proceed to the next step: download and save the RSS:

  const batch: string = out.batch = msg.batch
  ...

  let podcastEnt = await seneca.entity('pdm/podcast').load$(podcast_id)

  let feed = podcastEnt.feed

  let rssRes = await seneca.shared.getRSS(debug, feed, podcast_id, mark)

  let rss = rssRes.rss
  let feedname = encodeURIComponent(feed.toLowerCase().replace(/^https?:\/\//, ''))

  await seneca.entity('pdm/rss').save$({
    id: 'rss01/' + feedname + '/' +
      podcastEnt.id + '-' + batch + '.rss',
    rss
  })

We introduce a new parameter, batch, which is a string identifying the current podcast ingestion process. This is a classic "batch" job run once a day, so we create a unique identifier to help track this job (and you thought using LLMs and AI models was sexy! No, production coding is the same old boring thing it's always been since the 1950's - batch jobs!).

The table pdm/rss is special - it is a folder in an S3 bucket where we dump the RSS files. It is not special within our code, however, and looks just like any other database table. We do specify the data entity id directly, as this will become the file name in the S3 bucket.

As we discussed in a previous post, using the same abstraction for different persistence mechanisms makes our code much cleaner and easier to refactor. Changing cloud provider, or even just deciding to use an actual database table in future, won't require any changes to this code. But more importantly, we can unit test our business logic locally without even setting up fake S3 docker containers or other nonsense.

We store the RSS because we will need it for debugging, and we might want to use it in other ways later. Also, it is good to have a record of the state of a given podcast RSS at a given date and time so we can track changes.

We get the individual episode details from the items property of the RSS (we parse it using the rss-parser package) to handle RSS variants. We loop over each episode and emit a process:episode event for each one.

We're not bothering to check for changes to episodes here. That logic should live in process:episode as that message "understands" episodes.

We also include the episode data in each message. This is a calculated risk. In general, you should keep your messages small, and reference data using an id. But in this case, we can safely assume the RSS content is "reasonable". If the content is too large, we'll just let this message fail and then think about whether we even want to solve the problem later.

  let episodes = rss.items

  out.episodes = episodes.length
  episodeEnd = episodes.length

  for (let epI = episodeStart; epI < episodeEnd; epI++) {
    let episode = episodes[epI]
    await handleEpisode(episode, epI)
  }

  async function handleEpisode(episode: any, epI: number) {
    await seneca.post('aim:ingest,process:episode', {
      episode,
      podcast_id,
      batch,
    })
  }

  out.ok = true

  return out

When deployed, the process:episode message is triggered asynchronously, but we still wait for the message queue to accept it (hence the await). Locally it is just a normal message and we wait for the actual business logic to complete.

Once we've sent all the process:episode messages, we're done. We set ok: true and return a response.

In the next post in this series, we'll look at the processing of each episode, including (finally!) our first usage of an LLM, which we'll use to extract additional information from the episode description.

The Voxgig Podcast Chatbot: First, Subscribe!

Richard Rodger — Wed, 28 Feb 2024 23:23:36 +0000

This is the third post in a series I'm writing about a new Minimal
Viable Product we've released at
Voxgig that turns your podcast into a chatbot. Your visitors can now
ask your guests questions directly!

The first post is here: Building a Podcast Chatbot for
Voxgig
and you find the full list at the end of this post.

We want to ingest a podcast. The podcast episodes are described in an RSS feed that also contains the metadata about the podcast. We send a message to the microservices system describing what we want to happen:

{
  aim:       'ingest',   // The "aim" of the message is the `ingest` service.
  subscribe: 'podcast',  // The instruction to the `ingest` service.
  feed:      'https://feeds.resonaterecordings.com/voxgig-fireside-podcast'
}

The system routes this message to our implementation code (we'll come back to how that happens later in this series). Since this is a TypeScript code base, our implementation is a TypeScript function:

async function subscribe_podcast(this: any, msg: any, meta: any) {
  let out: any = { ok: false }

  out.why = 'no-code-yet' 

  return out
}

As a convention, our microservices accept messages that are JSON documents and also respond with messages that are JSON documents. There may not be a response (if the message is an "event"), but if there is, we use the property ok: boolean to indicate the success or failure of the message, and use the why: string property to provide an error reason to the sender.

Why use this convention? You can't throw exceptions over the network (easily or naturally). But you also don't want to use exceptions for business logic failures. The system itself hasn't failed, just some business logic process.

Our initial implementation does nothing except fail, but we can still test it by using the REPL:

boqr/pdm-local> aim:ingest,subscribe:podcast
{
  ok: false,
  why: 'no-code-yet'
}

Now we can start to write code to fill out the implementation, which will get hot-reloaded as we go, and we can keep using the REPL to test it. This is what they call Developer Experience folks.

Let's pause for a minute before writing any more code. We want our system to handle more than one podcast, and we know that we will need to process each podcast episode separately (download the audio, transcribe it, create a vector "embedding" etc.). So in this message action, all we should do is download the RSS, and then send another message to start the actual ingestion process. That way we separate obtaining the podcast RSS feed, from operating on that feed. This will make development and testing easier because once we have the feed, we can run ingestion multiple times without downloading the RSS each time. And trust me, when you build an AI chatbot, you need to rerun your pipeline a lot.

Here is the basic functionality:

import type { RSS } from './ingest-types'

async function subscribe_podcast(this: any, msg: any, meta: any) {
  // The current seneca instance.
  const seneca = this

  let out: any = { ok: false }

  // RSS URL
  let feed = out.feed = '' + msg.feed

  // Processing controls
  let doUpdate = out.doUpdate = !!msg.doUpdate
  let doIngest = out.doIngest = !!msg.doIngest

  // Load the podcast by feed URL too see if we are already subscribed
  let podcastEnt = await seneca.entity('pdm/podcast').load$({ feed })

  // Download the RSS feed if new or updating
  if (null == podcastEnt || doUpdate) {
    let rssRes = await seneca.shared.getRSS(feed)

    let rss = rssRes.rss as RSS

    podcastEnt = podcastEnt || seneca.entity('pdm/podcast').make$()
    podcastEnt = await podcastEnt.save$({
      feed,
      title: rss.title,
      desc: rss.description,
    })
  }

  if (null != podcastEnt) {
    out.ok = true
    out.podcast = podcastEnt.data$(false)

    if (doIngest) {
      await seneca.post('aim:ingest,ingest:podcast', {
        podcast_id: podcastEnt.id,
        doUpdate,
        doIngest,
      })
    }
  }
  else {
    out.why = 'podcast-not-found'
  }

  return out
}

Typescript types: so there's a lot of any types in this code. There will be a future refactoring to improve this situation. For now, remember that network messages are not function calls, and the messages are validated in other ways.

Yoda conditions: (null == foo) Safer, this is, young Padawan

I've removed most of the debugging, tracing, and control code, but what you see above is the real implementation. Let's unpack what it does.

This message action expects a message that gives us the feed URL in the feed property. But it also looks for optional doUpdate and doIngest boolean properties. These are used to control how far we progress along the ingestion pipeline.

The doUpdate property must be true to download the RSS feed and "update" the podcast.

The doIngest property must be true to send the ingestion message to start ingesting individual episodes.

You can use these properties in the REPL to switch off parts of the pipeline to concentrate on validating and developing the parts you want to work on.

Note that we also add these properties to the out variable and send them back with the response. This makes debugging easier, especially when looking at logs.

New or Existing Podcast?

The first real work happens when we try to load the podcast from our database using the feed URL. If it already exists, we use the existing database row. The Seneca framework has a fundamental design principle: everything is a message. That includes database access (or is there a database at all? Who knows...).

As a convenience, database messages are wrapped in a traditional Object Relational Mapper interface. Seneca also supports name-spacing data entities. I'll explain more about Seneca's persistence system as we go, so you'll have to take things a little on faith at the start.

Let's try to load a podcast from the database:

let podcastEnt = await seneca.entity('pdm/podcast').load$({ feed })

This is an asynchronous operation (we have to wait for the database), hence we have to await a promise. The entity method creates a new entity object for us, loading data from whatever table or data source is mapped to the pdm/podcast entity. The Seneca entity ORM provides standard methods that all end in $ to avoid clashing with your own database column names. The load$ method takes a query object and returns the first entity that matches the field values in the query.

In this system, the pdm namespace is used for tables relating to our podcast chatbot business logic. We will also make use of Seneca plugins that provide standard functionality (such as user accounts) that use the sys namespace. By using our own namespace, we ensure our own tables never conflict with any of the Seneca plugins we might want to use later. I did say this was production code. This is the sort of thing you have to worry about in production applications.

If the podcast does not exist in our system, or if doUpdate is true, then we proceed to download the RSS feed, set some fields on the pdm/podcast entity, and save it to the database.

Cool. So now, if we do have a podcast (null != podcastEnt), then we can proceed to ingest it if doIngest is true. We send a new message but do not expect a reply. We are emitting the message as an event. Locally, the message gets routed inside our single process monolith. When deployed to AWS, our message will get sent out over a Simple Queue Service topic to whoever is interested in it for further processing. Either way, we do have to wait for the message to be posted.

Notice that the mental model of "it's all just JSON messages all the way down" means we don't have to (in this code) think about message routing architectures, microservice design patterns, or what service SDK we need to use. We can concentrate on our own business logic.

The Voxgig Podcast Chatbot is Production Code

Richard Rodger — Fri, 23 Feb 2024 11:45:27 +0000

This post is part of a series.

This is the second post in a series I'm writing about a new Minimal Viable Product we've released at Voxgig that turns your podcast into a chatbot. Your visitors can now ask your guests questions directly!

The first post is here: Building a Podcast Chatbot for Voxgig

In this series, I'll dive into the code that implements the chatbot. It's all open source so you can cut and paste to have your own one.

Since it is production-grade code, not just an example for the sake of some "content", we'll navigate through the code in baby steps by focusing on specific tasks, rather than trying to go top-to-bottom. Production code has to do a lot of things, so rather than trying to start at the start, we'll start in a reasonable place so you can see useful code right away.

We have a podcast RSS feed URL, and we want to trigger ingestion of the episodes into the chatbot. Where do we begin?

Well, we have to download and parse the RSS feed. There's a great RSS parser package that we can use: rss-parser - thank you to Robert Brennan! To get the RSS feed we need one line of code:

await Parser.parseURL(feed)

That returns the contents of the feed (which is XML), as a JSON document so it's nice and easy to work with. We'll loop through all the episodes, download the audio, get it transcribed with a speech-to-text service (Deepgram in the first version), and then sprinkle some Retrieval Augmented Generation magic pixie dust over everything to make the chatbot work.

But first, let's do some software architecture. This is not meant to be a toy. The system deploys to AWS, and uses Lambda Functions, DynamoDB, S3, SQS, and other fun AWS stuff.

Also, the system is a microservice system when deployed, but a local monolith when developing locally. How that all works is something we'll come back to in later posts.

Since we are downloading RSS feeds, we are effectively an RSS reader, and thus we can have the concept of "subscribing" to a feed in our system. That means we need to have a microservice that can accept an instruction to subscribe to a podcast.

The microservice is called ingest and the message that we send to the microservice to subscribe to a podcast (and optionally trigger ingestion) looks like this:

{
  aim:       'ingest',   // The "aim" of the message is the `ingest` service.
  subscribe: 'podcast',  // The instruction to the `ingest` service.
  feed:      'https://feeds.resonaterecordings.com/voxgig-fireside-podcast'
}

There are more properties, but we'll come back to those later. Also, I'm a big fan of the Three Virtues: Laziness, Impatience, and Hubris. The string "aim" is three characters and one syllable, whereas "service" is seven characters and two syllables. Also "service" is a term that is going to be horrendously overloaded and over-used in any codebase. Using short non-technical Anglo-Saxon words to stand for project-specific concepts is a great way to reduce overall confusion in a large code base that you will have to maintain for a long time. My favorite programming tool has always been a thesaurus.

I'm completely ignoring, for now, all questions about how this message, which is a JSON document, gets routed to the ingest microservice. But let's look at the two main ways that we can send this message.

In code, you can send this message (perhaps as a result of a new user filling out a form with their RSS feed URL) by using the Seneca Framework microservices library:

const result = await seneca.post({
  aim:       'ingest',
  subscribe: 'podcast',
  feed:      'https://feeds.resonaterecordings.com/voxgig-fireside-podcast'
})

Oh wait, we forgot to be lazy. Let's try that again:

const result = await seneca.post('aim:ingest,subscribe:podcast',
  feed: 'https://feeds.resonaterecordings.com/voxgig-fireside-podcast'
})

Notice that there is no indication of how this message is transported to the ingest service. No HTTP calling code, no message topic, nothing. That's important. Because the shortest path to the dreaded distributed monolith is not to use a message abstraction layer.

The other way to send this message, which you'll see quite a bit in this series, is to use a REPL:

$ npm run repl-dev

> @voxgig/podmind-backend@0.3.3 repl-dev
> seneca-repl aws://lambda/pdm01-backend01-dev-monitor?region=us-east-1

Connected to Seneca: {
  version: '3.34.1',
  id: '6ejf/monitor-pdm01-dev@0.3.3',
  when: 1708685110275
}
6ejf/monitor-pdm01-dev@0.3.3> aim:ingest,subscribe:podcast,doIngest:false,feed:'https://feeds.resonaterecordings.com/voxgig-fireside-podcast'
{
  ok: true,
  why: '',
  batch: 'B2024022310460874',
  mark: 'Ml032jm',
  feed: 'https://feeds.resonaterecordings.com/voxgig-fireside-podcast',
  doUpdate: false,
  doIngest: false,
  doAudio: false,
  doTranscribe: false,
  episodeStart: 0,
  episodeEnd: -1,
  chunkEnd: -1,
  podcast: Entity {
    'entity$': '-/pdm/podcast',
    feed: 'https://feeds.resonaterecordings.com/voxgig-fireside-podcast',
    t_mh: 2024020722371877,
    t_m: 1707345438776,
    t_ch: 2024012801013692,
    t_c: 1706403696926,
    id: 'pdfxc5',
    batch: 'B2024020722371839',
    title: 'Fireside with Voxgig',
    desc: '\n' +
      '            This DevRel focused podcast allows entrepreneur, author and coder Richard Rodger introduce you to interesting leaders and experienced professionals in the tech community. Richard and his guests chat not just about their current work or latest trend, but also about their experiences, good and bad, throughout their career. DevRel requires so many different skills and you can come to it from so many routes, that this podcast has featured conference creators, entrepreneurs, open source maintainers, developer advocates and community managers. Join us to learn about just how varied DevRel can be and get ideas to expand your work, impact and community.\n' +
      '        '
  }
}

I literally just REPL'd into my live system and sent that message so I could cut and paste that example for you.

That doIngest parameter lets me turn off ingestion. No point ingesting a podcast I'm already up-to-date with, just for the sake of an example. The message action, implemented in the ingest service, is synchronous, so I get a response back with the details of the podcast stored in my database.

I can trigger almost any behavior in my system, at any point in the ingestion pipeline, using the REPL. I can do this live on AWS, or I can do it locally (with hot-reloading), so the debugging experience is just lovely.

In the next post, we'll look at the implementation of the aim:ingest,subscribe:podcast message action in detail. That will set us up to understand the other messages that focus more on the RAG implementation.

But you don't have to wait for me: ingest source code on github.

Building a Podcast Chatbot for Voxgig

Richard Rodger — Tue, 20 Feb 2024 15:25:20 +0000

This post is part of a series.

At Voxgig we have now recorded 140 podcast episodes (as of Feb 2024). That's a lot of chatting about developer relations. It's great if you are a regular listener (thanks!), or have the time to listen to our back catalogue. But what is frustrating is that there is so much wonderful knowledge about developer relations locked up in an audio format that, while enjoyable to listen to, is not accessible or usable in an efficient manner.

When you need to know something about developer relations, and you want to tap into the wisdom of one our guests, you are out of luck. Or at least, you were. We decided to turn our coding skills to this little problem and write our own AI chatbot to let you ask our guests your questions directly.

If you go to the Voxgig podcast page, you can now see a little chatbot widget, and when you type in a question, you get a reasonably good answer!

Now this is a very crude initial version, and we are actively improving it. The chatbot uses Retrieval Augmented Generation (RAG) to ingest all our podcast episodes, and let you ask questions of our guests directly.

We decided to open-source the entire project. Building a RAG ingestor and query engine is a natural fit for microservices (there are lots of little asynchronous tasks). So if you'd like your own podcast chatbot, just cut and paste our code: github.com/voxgig/podmind.

We've had such an enthusiastic reaction to this fun little project we've decided to take it further and extend the chatbot to handle all sorts of developer content. We're also adding the ability to easily experiment with the prompt and the "chunker" (stay tuned if you want to know all about that stuff).

We're going to "build in public" with this project. You'll be able to follow along at a source-code level as we make it more and more useful.

To get started, here are the microservices and what each one does:

audio - Transcribes audio to dialogue text (using Deepgram).
auth - User identity - for the user analytics dashboard.
chat - Perform chat queries.
chunk - The "chunker" that splits the transcript into chunks for the vector db.
embed - Embed the chunks as vectors so you can do similarity searches.
entity - General purpose persistent entity operations.
ingest - Ingestor to orchestrate the ingestion process.
monitor - Monitoring and debugging of the system, provides a REPL.
store - Store audio and RSS files.
user - User management.
widget - The API for the embeddable chat widget (which is a proper web component).

We run this system as a "local monolith", but deploy it as Lambdas on AWS.

A lot of our development is done in a REPL with hot reloading, so we can alter business logic and experiment on the fly, even with our deployed Lambdas. We'll get into this in later posts, but here is an example of submitting a question to the live chat service Lambda:

$ seneca-repl aws://lambda/pdm01-backend01-dev-monitor?region=us-east-1

> @voxgig/podmind-backend@0.3.3 repl-dev
> seneca-repl aws://lambda/pdm01-backend01-dev-monitor?region=us-east-1

Connected to Seneca: {
  version: '3.34.1',
  id: 'axjz/monitor-pdm01-dev@0.3.3',
  when: 1708436725968
}
axjz/monitor-pdm01-dev@0.3.3> aim:chat,chat:query,query:'what is devrel?'
{
  ok: true,
  why: '',
  answer: "DevRel, short for Developer Relations, is a field within the tech industry that primarily focuses on building and maintaining relationships between companies and developers. DevRel professionals often act as intermediaries between developers and the companies they serve, bridging the gap between technical know-how and the business needs of a product. They help developers understand how a company's product works and are always on the lookout for opportunities to improve the overall developer experience.",
  context: {
    hits: [ ... ]
  }
}

We're a TypeScript/JavaScript shop, so that's what we're using right now. I would not be surprised if a Python microservice or two appears in the future, but for now, we'll stick to what we know best. This is the start of a series - I'll add links to more posts as we go.

Why you should be using a REPL all the way to production

Richard Rodger — Mon, 04 Sep 2023 09:34:22 +0000

Let’s talk about one of the most productive tools a coder can use: the REPL! The first half of this article gives you a short introduction to the subject if you’re not a coder. If you are a coder, skip ahead to the second half where I talk about using REPLs in production (“with great power comes great responsibility”).

I recently released a long overdue update to the Seneca REPL plugin (A Node.js microservices framework). Since I’m also a fan of fastify (a web framework for Node.js) I couldn’t resist submitting a plugin proposal for a fastify REPL too, since we’ve been having far too much fun with our Seneca REPL, and fastify users are missing out!

Why this renewed focus on a feature I first built years ago? What’s so great about REPLs anyway?

A REPL (Read-Execute-Print-Loop) is a coding tool that lets you interact directly with a running system. The browser console is easily the most used REPL, and essential to the development of any complex web application. You can type in code directly, and see results right away. Rather than waiting for your code to compile, deploy or load, you can make changes immediately. You get direct feedback with no delays that lets you stay focused on the work.

Programming systems provide a REPL as an interactive text interface that lets you write code, usually in a more lenient or shorthand form. You don’t have to formally declare variables. You get access to predefined useful state objects (in the browser, you can access the document object, among many others). The essential thing is that you are given access to the live running system, whether that is a web browser, or a local development server, or something else.

Coders use a REPL to debug problems directly, to inspect data to figure out what is going on, to experiment with new code to see what works, to get a feel for a new API (Application Programming Interface), to manipulate data, and even to administer deployed systems.

If a new system doesn’t yet have a REPL, coders will always build one for it. Why does this happen? In more ordinary software development, you write some code, save it, and then run the code, often as part of a test suite. If you’re lucky, your test suite is pretty fast, on the order of seconds. More usually, it can take minutes to run for a production system.

There’s a huge amount of friction in writing or changing some code, then switching to another place to run the code, and then waiting for results. Programming is a lot like the spinning plates performance act. The coder has to spin up multiple mental states and keep them all in their head, so that they can tie the pieces together to solve the problem. Get interrupted while you’re waiting for your tests to complete? Good luck fixing that bug–now you have to rebuild your mental state again, which can easily take a few minutes of focused thought.

As an aside, this is why “collaborative” open plan offices make your developers super unhappy and unproductive–they need to concentrate, not chit chat!

The coding tool that makes unnecessary wait time go away is the REPL. You don’t have to context-switch half as much. The code is run and you see the results right away, so you can stay focused and get the solution. Often you’d be able to copy and paste the rough code out of the REPL into your editor, and fix it up to be production grade.

Not all code is the same. REPLs work best with “business logic” code, and user interface code. When you’re writing code that’s more about computer science algorithms, or core library code, you spend much more time just thinking rather than smashing the keyboard. But when you have a documented business requirement, the work is more about gluing the parts together than inventing new algorithms. What counts is being able to ensure your data has the right structure, and you are calling the right APIs in the right way. A REPL makes this work much easier.

Where (and when) does the idea of a REPL come from? Probably in 1958, from John McCarthy, the inventor of the Lisp language. Interactive evaluation of expressions was an essential feature of Lisp right from the start. If you had had to code using punch cards up to that point, you’d probably be pretty keen on having a REPL too.

The Smalltalk programming environment took the idea of a REPL to the next level, by making the entire system “live”. You coded by manipulating the state of the live system using an editor that was basically a REPL. Saving your project was the same as serializing the current state.

Although it didn’t really work out, the database query language SQL was supposed to be a way to “talk” to databases using human-like language. But us programmers still get the benefit of an interactive language to manipulate data.

The REPL environment that most non-coders would be familiar with is a spreadsheet. You edit formulas and you see immediate results. This is very gratifying! Programmers enjoy the same serotonin kick. And it gets the work done faster. Enjoy the cheese!

With machine learning finally starting to deliver, many folks are getting exposed to another form of the REPL: the interactive notebook. First seen in the realm of statistical analytics and mathematics, with systems like Mathematica, notebooks such as Jupyter now provide a comfortable interface to complex underlying systems.

And we are not far off the holy grail of REPLs–the fully competent natural language interface provided by the new generation of artificial intelligence LLMs (Large Language Models). Just playing around with a service like ChatGPT will give you an instant feel for why coders are so keen on having a REPL available.

There’s even a marked-for-unicorn status tech startup called repl.it that provides software developers with, you guessed it, an online REPL-like environment for coding.

OK, let’s get a bit more technical. If you’re a coder and have used REPLs occasionally but have not found them to be that useful, let’s dive into how you can use them to get some super powers.

First, find a REPL! Here’s a list. The fact that almost every serious programming environment has one should tell you REPLs are a serious tool you need to master.

Python:
- Default Python shell
- IPython
- bpython
Ruby:
- irb (Interactive Ruby)
JavaScript:
- Node.js
- Browsers' Console
Lisp/Scheme/Clojure:
- GNU Clisp, Scheme, and other implementations
- Clojure
Scala:
- Scala REPL
Haskell:
- ghci
Java:
- jshell
Groovy:
- groovysh
Erlang:
- erl
Elixir:
- iex
R:
- R documentation
Perl:
- perl debug mode
- re.pl
Lua:
- Lua documentation
Swift:
- Swift REPL
Go:
- gore (third-party)
Julia:
- Julia REPL
OCaml:
- utop
- Default OCaml REPL
F#:
- fsi
C#:
- C# Interactive Window in Visual Studio
- csi command-line tool
Kotlin:
- Kotlin REPL
Crystal:
- icr
Rust:
- evcxr (third-party)
PHP:
- php -a
MATLAB:
- MATLAB documentation
Shell scripting languages:
- Bash, Zsh, and other Unix shells
Racket:
- Racket documentation
Prolog:
- SWI-Prolog and other implementations

Let’s talk about what you can do with REPLs and why you would want one. I’ll start with the ordinary stuff and work my way up to the definitely controversial radioactive-spider use cases.

You write some code. You compile it (for most languages). You run your unit tests. Or you run a small script to check things. Or you start up your system and then manually interact with it to check if your code works.

This cycle takes time. You spend a lot of time just waiting for compilation to complete, or the unit testing framework to boot up and then run your tests. Each individual delay might take a few seconds, but it all adds up. Pretty soon you’re on Twitter or Youtube or Discord and you lose twenty minutes before you drag yourself back to work. There are decades worth of articles all over the web about the productivity horrors of “context switching” and how it kills your concentration. You work best as a developer when you can get into that magical “flow state” where the code just pours out of your brain.

You know what makes flow state easy to maintain? A REPL! You write, you see results. Right here, right now. If that REPL is exposed by a local version of the system you’re working on, it gets even better. You stay in flow working on your system. Out pours the code!

Now let’s take it to the next level. If you rig up some sort of hot-reloading into your system (there’s always a way to do this in your language of choice), you can debug in flow state without those restart wait times that break your concentration. Instant, instant feedback. That’s the drug we need.

You run your code in the REPL. It’s broken. Alt-Tab back to your editor, make a change. Alt-Tab back to the REPL. Hit the up-arrow key, hit return, see results. Still broken? Repeat. Fixed? Alt-Tab, run the unit tests and bask in the glow of green. Go grab a coffee. You’re a 10X dev.

The feeling of intellectual reward you get from not wasting any time waiting for the machine is not something you’ll ever want to give up on again once you’ve tasted it.

And up another level. Take your REPL and start adding custom commands and shortcuts. Your command history will have lots of code that you keep using again and again. Turn that repeated code into shortcuts. Apart from the righteous satisfaction every dev gets from building their own tools, you just made yourself and your team a lot faster and a lot happier.

At this point it’s time to invoke the eternal three virtues of programming:

According to Larry Wall, the original author of the Perl programming language, there are three great virtues of a programmer; Laziness, Impatience and Hubris

Laziness: The quality that makes you go to great effort to reduce overall energy expenditure. It makes you write labor-saving programs that other people will find useful and document what you wrote so you don't have to answer so many questions about it.
Impatience: The anger you feel when the computer is being lazy. This makes you write programs that don't just react to your needs, but actually anticipate them. Or at least pretend to.
Hubris: The quality that makes you write (and maintain) programs that other people won't want to say bad things about.

[“Programming Perl", 2nd Edition, O'Reilly & Associates, 1996]

Nothing is more faithful to the Three Virtues than a REPL (or perhaps Perl, or Raku nowadays, I guess).

Your shortcuts can go beyond mere abbreviations. All large systems define conceptual abstractions for the system itself. How do you represent data? Operations on that data? Components? User interactions? API requests? All these things and more can be “reified” in the REPL. Given form and substance, so that you can use them directly.

Here’s an example. Let’s say you use an Object-Relation-Mapper to talk to your database. Maybe your code looks like this:


let alice = new User({ name: “Alice”, role: “admin” })

alice.save()

You could type that into the REPL directly. Or you could lift up (“reify”) the “idea” of data entities into a system of shortcuts:


> save user {name:alice,role:admin}

Here’s another example. Let’s say you’re working on a REST API. Sure you’ve got curl and postman to help with testing and debugging, but again, they do require a context-switch. Not much, but it adds up. Instead, use the REPL:


> GET /api/users

[ …list of user objects… ]

Once you start to build up a set of shortcuts, they become a vernacular that makes you and your team very happy developers. And fast.

Where can you go next? I’m sure you’ll be very familiar with code working perfectly on your local machine, and breaking once deployed to a build server, or to staging or production. This is normal. No amount of testing will save you. Indeed the purpose of build or staging servers is to find exactly these fracture points where different environments break your code.

That’s nice. What isn’t nice is the pain of debugging code that is broken on staging. Welcome to log file hell. We won’t wade into the Big Fight about print-versus-debugger debugging, but when it comes to staging servers, all you often get is a log file.

So you add some print statements and you redeploy. Then you wait for the remote system to churn. Then you load up your log files (probably in a clunky web interface), and try to make sense of the output, most of which is utterly irrelevant. You thought flow state was hard to maintain for local dev work? There ain’t no flow state here. This work is grueling and slow and very not fun.

Wouldn’t you like a REPL into staging? With a REPL you just look at things directly and muck about debugging right on the system, with the staging data. This is the way to debug! And if your REPL also includes some commands for data querying and modification, then you don’t even need to Alt-Tab over the database interface. Bugs and issues on staging can be resolved in nearly the same way, and nearly as fast as local issues. Happy happy developer!

When you implement a REPL to give you access to staging, you may not always just be able to expose an open network port, which is the normal mode of connection to a REPL. For things like serverless functions, you’ll need to co-opt the cloud infrastructure to let you submit commands and get responses. Use things like AWS Lambda invocation to get your commands up into the cloud. What matters is the developer experience of an interactive terminal. What happens behind the curtain is an implementation detail.

At this point you will need to start thinking more seriously about access control. Staging servers, build servers, and test servers are shared infrastructure. Who gets REPL access and what can they do? That has to be context dependent, but here’s some things that have worked well for me.

If your system has a concept of user accounts with permissions, reuse that abstraction. Each REPL is operated as a user in the system. You’ll need to add some way to login or provide an access token. If you have an external API that provides this already, you’re nearly home and dry.

If you don’t have a concept of a user, then you’ll need to at least provide a set of access levels, and restrictions for those levels. This is no different from other kinds of infrastructure work that you do to make your systems easy to monitor and maintain.

You will also want to control access at a high level such as network layers and application domains. This is again very much context dependent, but things like kubectl take you most of the way there.

And here’s the big one: TURN OFF THE REPL IN PRODUCTION. You really don’t want an active REPL to go live (well…unless you do–see below!). The safest way to do this is to treat the REPL as being under feature flag control, switched off by default. Add some failsafes by also checking general environment characteristics, like system variables. A REPL falls into the same bucket as system monitoring and control, so you can apply the same DevOps control principles.

And finally, you can run a REPL in production. It’s not as crazy as it sounds. A REPL lets you resolve emergency production issues. It gives you a fine-grained administration interface. It lets you perform maintenance you have not yet automated. These are all massive time savers, especially in the early days of a startup, before you have full systems in place.

Access to the production REPL should be very tightly controlled. And I like to add in additional layers of security, such as time-based one-time password verification. You could even start using SSL certs if you’re really paranoid.

I have found that the operational benefits for power users are worth the risk.

There is one important thing you’ll need to add to your REPL implementation if you go all the way to production. An audit log. You’ll need to keep a log of all the commands, and who ran them. And you’ll need to add some scrubbing logic to remove sensitive data. Your REPL is now a serious systems management tool, so will require more effort to maintain.

I’ve been using REPLs in all these ways for more than a decade. A REPL makes me faster and happier as a developer. It will make you faster and happier too - use a REPL!

@seneca/repl Version 6 Released!

Richard Rodger — Wed, 02 Aug 2023 17:04:09 +0000

I've released a substantial update to the @seneca/repl plugin!

The @seneca/repl plugin provides a REPL for the Seneca microservices framework. As one of the earliest plugins, it has proven to be incredibly useful. A REPL (Read-Execute-Print-Loop) offers an interactive space to write code and execute it instantly. If you've ever used the browser console or run the command node in Node.js, you've used a REPL.

To learn more about the plan for this release, refer to my previous post: @seneca/repl version 2.x plan (Yes, I did say 2.x - that was a brain glitch!). Read that post to understand what the Seneca REPL can do for you.

New Feature: Entity Commands

The Seneca REPL allows you to send messages directly to a running Seneca instance, just by entering the JSON source of the message. In fact, it's even simpler than that, as the REPL accepts a relaxed form of JSON called Jsonic, which lets you avoid most strict JSON syntax rules.

Here's an example of using the REPL to get the status of a Seneca instance:

$ seneca-repl
> role:seneca,stats:true
{
  start: '2023-08-01T19:23:23.274Z',
  act: { calls: 105, done: 104, fails: 0, cache: 0 },
  actmap: undefined,
  now: '2023-08-01T19:29:00.108Z',
  uptime: 336834
}

In this interaction, the full JSON message was submitted:

{ "role":"seneca", "stats":true }

This is equivalent to the Seneca API call:

seneca.act({ "role":"seneca", "stats":true })

Also, since Seneca accepts Jsonic too:

seneca.act('role:seneca,stats:true')

When working with Seneca data entities that provide a simple ORM to access your database, you can interact with them using standard Seneca messages. For example:

seneca.entity('foo').list$() // lists all rows in the table "foo"

In the REPL, this would be the equivalent message:

> sys:entity,cmd:load,name:foo

The REPL itself also provides a Seneca instance, allowing you to write standard Seneca code.

However, adhering to Larry Wall's programming virtues: Laziness, Impatience, and Hubris, I've introduced a REPL shortcut for data entities, as they are empirically the most common use case for the REPL.

Each entity operation (list$, load$, save$, remove$) gets its own REPL command, all following the same syntax:

CMD$ CANON [QUERY]

Here are some examples:

> list$ sys/user
> list$ sys/user group:foo
> save$ sys/user id:aaa,group:bar
> load$ sys/user aaa

The REPL commands provide various functions to manage data entities, and details and examples can be found in the article.

New Feature: Auto Reconnection

To enhance developer experience, the REPL client now automatically reconnects to the server if disconnected, using a backoff mechanism. You can also hit the return key to reconnect instantly.

New Feature: REPL Tunnels

Configuring REPL tunnels has been simplified, and it's now possible to drive the REPL using Seneca messages. We've introduced HTTP and AWS Lambda tunnels, and detailed instructions are provided in the article.

WARNING: This feature is a security risk! Don't expose your production systems without additional access controls.

New Feature: Isolated History

This release improves command history storage, now saved locally in a hidden .seneca folder in your home folder. It keeps a separate history for each unique connection URL, and histories are not truncated.

$ seneca-repl localhost?project=foo # unique command history for the "foo" project
$ seneca-repl localhost?project=bar # unique command history for the "bar" project

Done!

With this new, eagerly anticipated release of the @seneca/repl plugin, there are many features to explore and enjoy. As it's open-source, you can find it on GitHub, where you're welcome to submit bugs, issues, and feature requests. Enjoy!

@seneca/repl version 2.x plan

Richard Rodger — Mon, 24 Jul 2023 12:07:41 +0000

I'm updating the @seneca/repl plugin! Here is the basic plan.

NOTE: There's a @seneca/repl dev Github Project to track this work.

The @seneca/repl plugin provides a REPL for the seneca microservices framework. It is one of the earliest plugins, and has proven to be one of the most useful. A REPL (Read-Execute-Print-Loop) is an interactive space to write code and get it executed right away. If you've used the browser console, you've used a REPL. With Node.js, you also get a REPL, just run the command node by itself and off you go!

The big thing about a REPL is the speed boost it gives your development process. You just type stuff in and it works right away. You can directly inspect objects to see what they are made of. Debugging is much easier.

The Seneca REPL provides you with the standard Node.js REPL -- you can execute arbitrary JavaScript. But it also provides you with access to the root Seneca instance, and with shortcuts to post Seneca messages, and examine the running Seneca system.

For example, if you have a message foo:bar,zed:qaz, then you can post that message directly in the REPL:

> foo:bar,zed:qaz

The REPL accepts any valid JSON (or the equivalent Jsonic form of abbreviated JSON) as an input and attempts to post the input as a message, printing the result. Combine this with hot-reloading from the @seneca/reload plugin and you have a lovely little high speed local development environment for your microservice system.

An update for @seneca/repl is long overdue. I've created a Github project to track the main tasks. The most important new feature is the ability to interact with the REPL when your microservice is running in a serverless context.

A traditional REPL exposes a network port and you cannot do that over the network. This is fine for local development, but it is not supported in a serverless context. However most serverless environments provide an invocation mechanism so you can send messages to your deployed function. I'm extending @seneca/repl so that it can support this interaction pathway by providing a special message: sys:repl,send:cmd that can run REPL commands and collect their output.

To implement this some refactoring is required. The old codebase is pretty old. As in, callback-hell old. It also assumes a network socket. So this all has to be pulled apart and abstracted a little. The code is very much streams-based, and that also makes it fun, as the streams have to be marshalled to operate via a request/response interaction.

One issue in the existing code is the lack of a delimiter for the command result. It all sort of works by accident! I'm going to use NUL as the delimiter to mark the end of command response text. This should also clear up some bizarro bugs.

The other new feature is more direct support for Seneca entities with an extended set of special commands: list$, save$, load$, etc. that mirror the Seneca entity operations. This is a pretty big use case and we've been putting up with the kludgy workaround of using entity messages directly for ... too many years, sigh.

On the command-line side, the REPL client needs to be extended to perform serverless invocations. A stream overlay will be used for this to preserve streams as the basic abstraction for REPL communication.

The other tasks are housekeeping to move to the new Seneca standard test runner, Jest, and convert the codebase to Typescript.

Once this release is out, the @seneca/gateway plugins will need to be updated to handle security more generally when exposing a REPL. At the moment we tend to use a monitor serverless function that has no external exposure, and can only be called by invocation with appropriate credentials. This monitor function also uses our little trick of embedding all the other microservices as a modular monolith so that you can use the REPL to post any message. While this is mostly sufficient in practice, it would be nice to also be able to invoke any function directly via the REPL.

DEV Community: Richard Rodger

The Voxgig Podcast Chatbot: Triggering Ingestion, and some Debugging DX

Other Posts in this Series

The Voxgig Podcast Chatbot: First, Subscribe!

New or Existing Podcast?

Other Posts in this Series

The Voxgig Podcast Chatbot is Production Code

Other Posts in this Series

Building a Podcast Chatbot for Voxgig

Other posts in this series

Why you should be using a REPL all the way to production

@seneca/repl Version 6 Released!

New Feature: Entity Commands

New Feature: Auto Reconnection

New Feature: REPL Tunnels

New Feature: Isolated History

Done!

@seneca/repl version 2.x plan