DEV Community: Slobodan Stojanović

Employees, AI, and AI employees

Slobodan Stojanović — Wed, 17 Sep 2025 15:00:00 +0000

Two and a half years ago, my cofounder, Lav, and I tried to create an AI cofounder (CofounderGPT, as we called it), had a lot of fun, and failed. That’s not the only thing we’ve tried using AI; far from it. We use it daily for a variety of tasks, including (but not limited to) research, coding, marketing, managing and improving our product, creating new products, and for our hobbies. However, the AI cofounder project still runs somewhere in the back of our heads. Can we, or anyone, create it at some point in the future? And, what would be the impact on our products, employees, friends, and everyone if it gets created?

This is not another AGI discussion (well, it’s a monologue). I do not believe the evil artificial intelligence will threaten us all in the near future. However, if you continue reading past this point, expect a long article with diagrams and (mostly wrong) opinions on AI, employees, and cofounders.

A tale of an impossible AI cofounder

I do not understand things very well until I can visualize the way they work. In tech, the easiest way to understand how things work is to try to recreate them or use them to build something.

From the first moment I tried using LLMs, it was clear that they are amazing and have huge potential. Lav and I kept sending each other amazing demos we saw on Twitter. Then we tried recreating them just for fun and learning purposes. Most ended up with facepalming and “of course, it does not work except for this specific case in the demo!” But we learned a lot.

Then, one day, we decided to try building a product using AI (like building one product is not hard enough). And to make things even more interesting, we decided to use AI as a cofounder and create something without any employees or external help (while still running and working on Vacation Tracker full-time). This decision set us on a journey of 36 working days with CofounderGPT (over 5 months), which was fully documented here: https://knowlo.co/blog/day-1-startup-adventure-with-cofoundergpt/. And then, we just stopped. From the very beginning, it was clear that this experiment had failed. It was impossible to have a real AI cofounder. However, we continued because it was fun and we learned a lot.

But why was it impossible? Were we too early? Would it be different today with GPT 5 Pro, Claude Opus 4.1, Cursor, reasoning models, and all the other amazing tools we have access to? It would be different for sure. And we would have an illusion that our experiment might succeed for a long time.

There were and still are many problems with the potential AI cofounder. Some are very obvious, such as the context window (the amount of data an AI can keep in its memory), reasoning (the ability to think through and analyze problems), and available tools (the things an AI cofounder can use to do its job). All these things are getting better and better. Just look at Cursor, Claude Code, OpenAI Codex, and other coding AI tools today. The improvement of AI coding abilities over the past few years is so big that it’s not even measurable anymore.

However, a cofounder is more than a set of skills. Cofounders have visions, motivation, passion, and many other qualities that an AI cannot replicate. They make decisions (which is easy for an AI) and accept the potential consequences of these decisions (which is not even close to possible with LLMs and the current state of AI). And they have emotions and gut feelings.

To be honest, a true AI cofounder would not be possible from a legal perspective at the moment. However, the focus was on cofounder skills, not legal matters (they are not fun anyway).

Ok, let’s say that cofounders are a bit out of reach of the AI in its current stage. But what about employees? Will AI replace developers and other employees in the near future?

To understand that, let’s rewind a bit and try to understand how people do things.

A case of AI employees

I enjoy using AI for coding, research, and product management. Writing code or exploring and building a feature prototype has never been easier. I feel a lot more productive. Everything moves so fast! New tools, new models, better practices. Does that mean that we’ll have full-time AI developers replacing actual people soon?

If you compare the number of data center vs office space construction in the US, it seems so. The trend of building space for GPUs and our future digital employees is rising rapidly.

Source: https://x.com/JosephPolitano/status/1951740903925715126.

But, of course, it’s not that simple. To understand why, we need to go back to the core goal of each company: to make money. Whatever a company does, it needs to generate revenue to survive. Even hot new startups that focus on market capture instead of profits need to make money through investments.

How do companies make money? By selling products or services (providing value) to entities that can pay. So, companies hire employees to do things that lead to creating and selling products or services (directly, such as sales, programming, or indirectly, such as management, HR, etc.).

So, our oversimplified value chain could look similar to the following.

Companies need to make money.
Companies make money by creating and selling products or services (value) to entities capable of purchasing them.
Companies need employees to create and sell products or services.

Obviously, it’s more complex than that. But if we focus on the employees, we can simplify even more and say:

Companies need to make money.
Companies make money by doing something (creating, selling, etc.).
Companies need employees to be able to do that “something.”

But what type of employees are we talking about here? It’s quite different when we talk about factory workers or product managers and software developers. Or is it? In a timeline long enough, all jobs will either become obsolete and disappear or become fully automated.

But let’s focus on what we call knowledge workers. “Knowledge workers are professionals whose primary asset is their mental skills and expertise, used for critical thinking, problem-solving, and generating new information rather than just performing manual tasks.”

How do knowledge workers do their jobs? The answer is obvious: it depends on the actual job. Their main asset are their mental skills and expertise, but they use various tools to accomplish more and amplify their skills.

Initially, knowledge workers did everything manually, but eventually, they or someone else created a variety of tools to automate and simplify parts of their jobs, enabling them to complete more tasks and reduce the number of mistakes. For example, software developers have frameworks, linters, and many other tools. But we now have AI, too! Where do Cursor, Claude Code, OpenAI Codex, and other tools fit?

Let’s try to visualize this evolution. First, we’ll take our company value chain and add another axis to it, which represents an evolution of all these components (an X-axis). We’ll get something similar to the diagram below. Actually, this diagram is a special type of map, known as a Wardley Map. The Y-axis represents a value chain with a user (or an anchor) at the top (Company) and all of its needs (company needs, interaction channels, activities, etc.) below. Needs higher on the vertical axis are more visible and important in the context of the map than those lower in the value chain.

Wardley Maps define the following stages of the evolution:

Genesis - Novel, uncertain, constantly changing
Custom Built - Growing, becoming more defined
Product - Stable, well-defined, widely available
Commodity - Ubiquitous, standardized, utility-like

It’s a bit hard to fit employees in these evolution stages, but we can fit the work they do in something similar.

Initially, we had knowledge workers doing everything manually, which was the genesis stage, because it was novel. Then knowledge workers started using tools to amplify their skills, which was (and still is) a custom build stage. This may be a bit confusing because the actual tools they (or we) use are often in the product or even commodity stage, but the way we use them is still custom, and it’s slowly becoming more defined. Finally, we see these amazing AI tools that are still not stable enough, but they are slowly moving knowledge workers to the product stage: an employee + AI automation will become well-defined and widely available over time. But what’s next? What would be the actual commodity version of a knowledge worker? Fully automated (AI) employee, I guess.

Scary, right?

It is! Luckily, it’s as true as the Windows updater telling you it’s 99% done.

So, is AI coming for our jobs? And if it does, when?

Is AI coming for our jobs?

The only proper way to answer this question is to define what our job actually is. As I already said, if we look far enough in the future, all currently known jobs will eventually become either obsolete or fully automated. But we don’t really care about thousands of years from now. We care about the relatively near future (decades, I guess).

Let’s take software developers as an example. If you are a software developer and you think your job is to write code, then I have bad news.

Luckily, businesses rarely care about code (yet some care even about GitHub stars, I know). Ok, but what are the actual responsibilities of a software developer, then? Companies need to make money, so it must be something related to earning more money or spending less money. They need to use their mental skills and expertise to:

Intent & prioritization : Decide which problems are worth solving, define success metrics (revenue, retention, service levels), identify constraints, and set sequencing.
Boundaries & contracts : Shape the domain model and API/data contracts to keep change cheap and risk contained.
Code & implementation : Turn intent into working software: write code and tests, integrate with existing systems, and document expected behavior.
Verify & operate (reliability & maintenance): Verify changes using tests and controlled rollouts, run the system in production with monitoring, alerting, error tracking, incident response, rollbacks, etc.
Security : Define trust boundaries, access controls, and data‑handling rules; manage vulnerabilities, exceptions, and risk acceptance.
Delivery & pipelines : Automate build/test/deploy/rollback, with quality and compliance checks, ensuring changes ship safely and predictably.
Long‑term strategy & evolution : Choose a target architecture and evolve it in alignment with the business strategy; manage tech debt and total cost of ownership.

Obviously, these depend on the software engineer’s expertise. It’s not the same for frontend and backend engineers, juniors and seniors, etc. It also depends on the person who is preparing the list. Finally, it depends on the LLM and model we use to analyze the list of essential software engineer activities. Yes, of course, we use LLMs to assist us with almost everything today.

AI affects most of these activities, but some have a high potential to be fully (or mostly) replaced by AI, and other activities will be augmented by AI. Let’s see activity by activity:

Intent & prioritization : AI can help you find relevant information faster and help you evaluate everything you need to know (you can run potential scenarios through brainstorming-like sessions, etc.). It could help with KPIs, ROI, documentation, and many other things. But, it can’t replace your judgment, among other important things. AI could help, but it’s, again, up to you to use and apply that help and come up with something that makes sense in each specific case (so, it’s still in the custom built phase).
Boundaries & contracts : API schemas, payloads, examples, scaffolds, and similar things are (or soon will be) a commodity with AI. These will be mostly automated. However, you’ll still need to define domain boundaries and understand the domain. However, this will get more well-defined over time, so I think it’s closer to the product phase.
Code & implementation : This one seems obvious, AI already writes code well enough. Over time, you’ll write code only in specific cases, such as very novel logic, examples of tricky integrations, or in cases when you need to gain a very high level of performance in your area of expertise (and when I say expertise, I mean you are one of the best, not just good). So, this goes to the commodity stage.
Verify & operate (reliability & maintenance): You still need to make decisions, but everything gets easier to decide (AI does triage, correlates logs and traces, sends PRs, suggests decisions). I would say that this goes to the product phase and gradually shifts to the commodity phase over the years.
Security : Something similar might happen to security. It’s still crucial for you to make a decision and have the final word, but applications are continually improving. You make decisions, and the app provides what you need. Well-defined, widely available => product phase.
Delivery & pipelines : This is another area that we can see easily dominated by AI in the relatively near future. You might still need to think about the release strategy and to give final approvals in some cases, but CI/CD, versioning, changelogs, canaries, rollbacks, and similar are strong candidates for the full AI automation (shifts to the commodity stage).
Long‑term strategy & evolution : Similarly to the intent and prioritization, AI will be able to give you all the critical information and assistance you might need. For example, codebase analytics, refactor opportunities (or even PRs), potential impact analysis, and different potential paths. However, it’s still up to you, your understanding of the goal, judgment, and a team to use these and come up with something that makes sense for the current and desired state of the business you are working on (or for). This stays in the custom built phase.

Here’s the updated map:

So, what do you say, is AI coming for software engineer jobs? AI has high potential to eliminate the typing aspect of the job really fast, but a software developer’s job is not typing code. It’s problem-solving through thinking, judgment, communication, and writing instructions for computers to follow.

We are going in circles. Now that we can get the code faster, we started talking about better specifications, then we’ll focus on tests, and so on. The only important difference is that, as my good friend said recently, these circles are not actually circles; they are a big spiral, and everything goes faster and faster.

Focused and recharged super employee

If AI automates most of the typing aspect of the software engineering job, we can expect something similar for other knowledge workers sooner or later. However, that does not sound so scary if you remember that the main asset of knowledge workers is their mental skills and expertise, used for critical thinking, problem-solving, and generating new information rather than just performing manual tasks.

We will have more time to devote to the essential things, such as thinking and judgment. Ok, that sounds a bit scary.

However, there’s one problem with that: you can write code 14 hours per day, but you can’t really make good decisions and process large amounts of high-quality information that fast.

You need to be focused and rested.

Are you a cofounder, software developer, marketing manager, product manager, or do you have some other role? It does not matter. What matters is that you have enough focus and mental power to review all generated code, prepare prototypes, write instructions (prompts), orchestrate all agents, and decide what matters. These activities burn your energy faster than AI burns tokens. You can’t do that and have 10 meetings a day. Or work long hours and weekends. You need your battery at 100%.

How do you recharge and get focused? That’s different for each of us. However, skipping meetings that do not provide value, replying to non-urgent Slack messages or emails the next working day, closing your laptop after you finish your workday, and taking PTO are definitely helpful.

AI Agents: how they work and how to build them

Slobodan Stojanović — Sat, 03 May 2025 16:57:40 +0000

Have you heard about AI Agents? Of course, you heard about them. These are the intelligent agents who will take our jobs in a few years!

I don’t want to scare you, but someone on Twitter said that “most jobs will become obsolete” in less than 10 years. McKinsey agrees (they say AI Agents will replace 70% of office work), and Goldman, too.

So, I guess our clock is ticking. We don’t have much time. It’s probably better to take a woodworking course or something similar.

But I am not that good at woodworking. So, let’s try to understand how AI Agents work and if they are that scary.

If you read Twitter or Linkedin, AI Agents look like special agents that can do everything. The demos that they share look amazing.

However, these agents don’t feel that special when you use them. They are helpful for specific cases, just like travel agents. If you give enough details about your desired journey and budget to travel agents, they can find you the vacation you want and plan the entire trip. Just like Cursor! If you give enough details and specific instructions, vibe coding feels like magic (parts of the application start assembling in front of you). In other cases, Cursor feels as smart and useful as Alexa or Siri.

So, AI Agents can be extremely helpful, especially if you understand how they work. But before we can understand AI Agents, we need to understand LLMs.

How LLMs work

Large Language Models or LLMs are very good at predicting the next best set of words based on your input (questions, part of the text they need to complete, or detailed instructions), their training data (all the text that creators of the LLM you use were able to use for training, such as books, websites, your private data [just kidding, or am I?], and other datasets), context (the previous conversation flow or documents you attached), and specific configurations (such as weights, which prioritize certain word patterns, and settings like temperature, which control the randomness of predictions).

Let’s use the same example I used in the “5 Prompt Engineering Tips for Developers” article! If you ask an LLM to finish the following sentence: “I am speaking at,” it’ll probably say something such as “a business conference,” “a tech meetup,“or “a community forum.” There’s almost zero chance it would say, “A Martian picnic?” Or “a space farmer’s market.”

However, if we add a few sentences to the beginning of the instructions (or prompt, as we call it when talking to LLMs) that tell an LLM that it is a playful, chatty cartoon character named “Space Bunny,” the LLM would not finish the sentence with “a tech meetup,” or similar, but with something more similar to a Martian picnic.

When you talk to an LLM, your question or a set of instructions is called a prompt. So, prompts are just instructions. You tell an LLM what you want, and it tries to reply based on your input and context, as well as its training set and configuration. If your instructions are clear, there’s a higher chance you’ll get a helpful reply. However, an LLM will reply even if your instructions do not make sense. In that case and in some other cases, its replies might not be based on truth (we call that hallucinations). Everything related to hallucinations is improving fast, so whatever I write here will probably not be true in a few months.

So, you give your instructions (or write your prompt if you want to sound smarter), LLM takes these instructions, spins up some GPUs, burns a small forest, “eats” some of your tokens, and you get an unexpected wisdom or a hallucination. In the world of LLMs, tokens = money. You burn them like the Monopoly money, but the key difference is that LLM tokens are connected to your credit card.

But how does an LLM know how to reply to your prompt?

Computers are not that great with words. They prefer numbers. So, an LLM will split your instructions into tokens (yeah, these are the tokens I mentioned above). A token is a set of characters that is sometimes equal to a word, sometimes to the part of the word, and sometimes to a set of letters and other characters such as spaces, dots, commas, etc. The exact number of tokens your instructions have depends on the algorithm the LLM uses. You can see the visualization of OpenAI’s tokenizer in the image below or here: https://platform.openai.com/tokenizer. You’ll get slightly different results based on the model you select.

But tokens are still words! A tokenizer represents each of these tokens as a set of numbers (so each token becomes an array of numbers). These numbers are vectors that can be placed in a multidimensional space. The entire training set of an LLM is also transformed into tokens and then vectors and put in the same multidimensional space. The major power of LLMs is their ability to put related words (based on their vast training sets) close to each other in this space.

For a quick visual example, imagine that each token converts to an array of two numbers (two-dimensional space is easy to visualize). Then, we would be able to put our dots in this space similar to the following image:

Now that an LLM converts your instructions to a set of vectors (an array of arrays of numbers!) and puts them in its multidimensional space, it can use its algorithm to find the closest vectors that might be a good answer to your instructions. LLMs are Large Language Models, meaning they are trained on a massive set of data, which helps them put these vectors in the correct places in the multidimensional space and offer a meaningful answer.

Luckily, LLMs are products, and like other products, they evolve with user feedback and misusage. So we got many useful features that were not initially available, such as system prompts (parts of the prompts that are more important than the rest of the conversation with an LLM), better coding and JSON skills, etc.

An undercover agent

Meet my friend Claude. I ask it many weird questions every day. Claude is nice, so it tries to answer each question in detail politely.

One day, I asked Claude what the weather was like in Belgrade. I ask way more weird questions to both Claude and ChatGPT. But this question is special!

It’s special because Claude can’t answer it. It told me politely that it had no access to real-time weather information. Ah, I forgot that ChatGPT can search the internet, but Claude can’t do it yet!

It makes sense that Claude does not have an answer to my question because it takes months to train an LLM model. I could ask another LLM to answer my question or simply check my phone. But I like Claude! Can I do something to help it to answer this kind of question?

Can I do a quick Google search for Claude when it needs some real-time data?

It’s a weird idea, but let’s try it! I’ll tell Claude that it should let me know when it needs me to search the internet. Claude can be a bit chatty, so I’ll make sure to tell it to provide an exact search phrase I should use. For example, telling me Google:weather in Belgrade today would be ideal.

It seems that my friend Claude likes this game. Let’s ask again, “What’s the weather like in Belgrade today?”

It worked! Claude provided an exact search query so I can do a Google search and provide a screenshot. The reply was more detailed than I needed it to be, but it did not matter; I understood my assignment.

I copied the search phrase, opened my browser, and googled it. Then, I took a screenshot of the result and sent it to Claude. And Claude replied with useful information about the current weather in Belgrade!

Claude definitely liked this game.

But while I did this just for fun, I accidentally did one more thing – I just created an AI agent!

I know it’s not a very useful agent, as I could just read the weather data from Google. But it’s still an agent.

I also know ChatGPT can search the internet, so I could use it instead of Claude. But ChatGPT is also an agent! It’s just an undercover agent that looks like a plain old LLM. To be fair, Claude is also an agent. Just ask it to draw a diagram or create a webpage for you, and you’ll see some superpowers that LLMs do not have.

How AI Agents work

LLMs are amazing! They really are. But like many other tools, they are good at some things but not so good at others.

For example, LLMs are excellent at picking the best set of tokens to continue the set of tokens we provided. Or, in a human-understandable language, they are very good at answering questions, completing sentences, writing text, etc.

You ask a question. LLM replies. Sometimes, it’s helpful information; sometimes, you must ask a follow-up question. But it always replies.

However, not all of these replies are based on truth. Sometimes, an LLM replies with false information that we call hallucinations. That’s because it tries to find the closest set of tokens to your tokenized instructions (or your question) and always finds something.

LLMs do not really care about the truth. They care about the closest tokens to your tokenized instructions, their training sets, their configuration, and some additional parameters.

But what makes an LLM an agent?

Agents are LLMs with something that provides missing information or capabilities to help LLMs answer our questions. If we call these things “tools,” agents are LLMs with tools.

However, to be an agent, LLM must orchestrate these tools and decide when it has enough information to answer our questions or complete our tasks. If we orchestrate the tools with predefined code, LLM is just a tool in our code, and our code is not an AI agent.

The diagram above looks familiar. It looks like a while loop!

So, I guess we can say that an AI agent is like a “while loop” that keeps asking available tools to provide additional information or capability until it has all it needs to complete the task or answer the question.

Anything can be a tool that provides missing capabilities or information to LLMs as long as LLMs have an easy way to use that tool.

For example, I was a tool that my friend Claude used to find information about the current weather in Belgrade! But that made our “while loop” expensive because it used both LLM tokens and my time.

These “while loops” are generally expensive. They are not expensive because of the big O notation and code complexity but because in each iteration, LLM evaluates whether it can complete the task and uses tokens (and our money).

Being expensive depends on the value it provides, but it’s always a good idea to be careful. You can be careful by setting the billing alarms and spending limits, making sure that the LLM does not iterate indefinitely (by limiting the number of iterations), picking the right model for your task (sometimes cheaper models can also complete your tasks), and configuring monitoring, error tracking, and alarms.

While loops and where to write them

So, if agents are while loops with LLMs and some additional tools, where do we write these while loops to create an agent?

The answer is almost anywhere. While creating an AI agent using pen and paper could be a thing, it’s not really a practical way of making an agent. Another cost-inefficient and unhelpful way of creating a while loop is using a person to act as one. However, you can write this “while loop” wherever you need it. For example, it can be inside the app you are working on, in your terminal, on a server (using any backend language you prefer), in a browser, etc. As long as you are careful and you do not leak your LLM secret key and other similar secrets.

To write an agent “while loop,” you need to do the following:

Choose an LLM model that fits your needs and your budget (which can be $0 or whatever other number).
Define a system prompt with a clear explanation of all the tools you want to provide (including when and how to use them).
Ask an LLM to reply in a strict JSON format or any other structure you prefer.
Make sure you parse and validate the reply correctly.
Handle errors and set the maximal number of iterations, billing budget, and alerts.

Remember, LLMs are good at talking to humans (actually imitating human interaction), but human language is not easy to parse in the code. If you worked with LLMs and tried to get a JSON reply and nothing else, I am sure you, at least once, got the reply similar to: “Here’s your JSON: {...}.” Yelling at LLM and telling it to reply with JSON works sometimes, but in some cases, even 3 exclamation points at the end do not help. Even a simple agent we built inside Claude.ai replied with a sentence in front of the search phrase:

You can either pick a format that is easier to parse or use a simple trick explained in my previous article: provide the beginning of the reply and let an LLM continue. You can see the code example here: 5 Prompt Engineering Tips for Developers.

But, while understanding how these LLM “while loops” work is good, you do not need to write your own while loops. There are many existing tools and frameworks you can use.

AI Agent tools and frameworks

AI Agent tools and frameworks are like JavaScript frameworks – we get many new ones every day. Pick any word that comes to your mind. The chance to find a JavaScript package with that name in NPM and a .ai domain with that name is higher than the latest US-to-China tariff percentage.

For example, LangChain was the AI Agent framework a while back. Today, we have LlamaIndex and many other popular tools besides it. Big players like Microsoft have their own open-source takes, such as AutoGen. And, of course, services such as Amazon Bedrock Agents. There are many other examples, from tools for non-coders and open-source tools to enterprise-grade tools.

It’s hard to pick the best one. If you want to check just one that works with JavaScript or TypeScript, you can start with LlamaIndex.

LlamaIndex sounds similar to the Meta Llama models. But it’s not the same. Actually, LlamaIndex supports the Meta Llama model and many others (including OpenAI models, Anthropic models, open source models, Amazon Bedrock, Azure OpenAI, etc.).

Another interesting thing about LlamaIndex is that they focused on the AI Agent memory issue as an important problem to focus on. If you have worked with AI Agents, you know what I am talking about. If not, read on.

Remember, remember… the conversation we had yesterday

As I already mentioned, LLMs are limited by their training set, configuration, your instructions, and a few other things. One of their most important limitations is their context size.

The context size represents the maximum number of tokens an LLM can have in a single conversation (through the API, UI, or any other way you interact with it). It’s a hard stop. Once you fill the context with tokens, an LLM will explode. Well, not literally. But it’ll stop working. If you have used LLMs from the early days, you might remember that after a certain number of messages, LLM seems to forget what you were talking about. That’s because the context was filled, and an LLM removed the initial messages to make space for your new messages. Luckily, LLM then made system prompts, sticky parts of the conversation that always stay in context and allow you to provide the instructions.

If you manage to fill the context, LLMs will most likely do one of the following:

Remove some messages from the beginning of your conversation, but keep the system prompt so it still follows the instructions). This can cause an LLM to forget some parts of your conversation.
Summarize some parts of the conversation and replace N messages with the summary (well, LLMs are good with summarization). The quality of the remaining conversation depends on the way an LLM summarizes the conversation.
Block you from sending more messages (most likely if you are using an API).

Luckily, the context size is increasing fast (Claude has a 200k token context, Gemini 1M context, and Llama now has a context size with up to 10M tokens). However, a larger context size can decrease the ability of an LLM to find specific items in it. Also, we want to fit larger items in the context. We started with simple spreadsheets and PDFs, and now we want to embed whole knowledge bases, books, project documentation, etc.

Again, luckily, many smart people work with LLMs, so they quickly came up with an effective way to make the most of the (at that time very) limited LLM context size. However, naming things is hard (ask OpenAI and Anthropic or simply read the names of their models), so they called this approach Retrieval-Augmented Generation (RAG).

While RAG sounds complicated and is still one of the most misunderstood terms related to LLM, it is quite a simple but powerful concept.

In short, instead of putting all documents in the system prompt, you can wait for the user question, then tokenize it before replying and do a vector search against your knowledge base to find a few closest matches. Then, you take these pieces and tell an LLM to respond to the user’s question in the context of the provided pieces of your knowledge base.

Before you do a vector search, you need to split your knowledge base into reasonable chunks (i.e., articles, sections of the articles, or even paragraphs in some cases) and create vectors from each piece.

And when I say vector search, I mean something similar to the vector search that LLMs use under the hood. Remember the following image?

You can use a vector DB to do a vector search, but that’s not required, as you can do vector searches in some of the popular databases (such as PostgreSQL, ElasticSearch, etc.) or store vectors almost anywhere and create your own vector search function (Amazon S3, for example).

Writing your own vector search (or actually vector similarity) function also sounds complicated, but luckily, you can ask an LLM to write that function for you, and it can look similar to the following one:

// Calculate cosine similarity between two vectors
function cosineSimilarity(vector1, vector2) {
  // Calculate the dot product of the two vectors
  const dotProduct = vector1.reduce((sum, a, i) => sum + a * vector2[i], 0)

  // Calculate the magnitude of both vectors
  const magnitude =
        Math.sqrt(vector1.reduce((sum, val) => sum + val * val, 0))
        * Math.sqrt(vector2.reduce((sum, val) => sum + val * val, 0))

  // Return the cosine similarity
  return dotProduct / magnitude
}

This function returns a number that represents the similarity percentage. You can run the same function against each of the knowledge base article vectors and pick the two top ones with more than 80% similarity or something similar.

A simple function like the one I showed above would work fine for smaller databases. However, you should use a more efficient search for large data sets.

AI Agents use many tokens, and often need vast knowledge bases. LlamaIndex helps with more efficient vector search and allows you to create agents that are not like they came from the Memento movie.

However, explaining RAG and LLM memory in detail requires more than a few paragraphs. All these explanations would convert this article into a short book. So, let’s leave that for another article and get back to AI Agents.

Let’s build an AI Agent!

LLMs are products. Products evolve and add features based on users’ feedback and usage patterns. Well, using tools is one of the important features that most LLMs now support natively. Some LLMs call this feature tools (for example, tools in Claude), some call it functions (for example, OpenAI functions), but it’s the same thing that allows us to build AI Agents.

Built-in tools have a few clear benefits, such as replies in a strict JSON format, a well-defined format, and an easier ability to stream responses via HTTP. They also have less surface area for errors because they are now built into the LLM itself. But, as always, there are many different standards, and if you want to switch to some other LLM, you’ll probably need to define tools in a slightly different format. However, a simple abstraction (or, even better, a hexagonal architecture) makes this problem easier to manage.

Let’s build a simple agent using built-in tools! You can pick any LLM you like. I’ll use Claude Sonnet 3.7 on Amazon Bedrock. The example below would work fine with other models. I use Amazon Web Services (AWS) every day, so Bedrock is a natural choice (despite its limits, especially in European data centers).

So, where do we start? With our “while loop,” of course! Remember this?

I want to build a simple agent for my product, Vacation Tracker. It will be very simple because otherwise, I would need to write a book to show all the details. I want my AI Agent to be able to do the following 3 things only:

Help users to request leave, such as PTO.
Let users see which coworkers are not working today and who will be off this or next week.
Answer some basic questions about our product using our knowledge base.

With these 3 features, my “while loop” would look like the following diagram.

There are so many different ways to build this AI Agent using AWS. For example, we could create a simple serverless solution like the diagram below, with the following components:

I would use an Amazon API Gateway to expose the API endpoint for my AI Agent.
My AI Agent "while loop," or business logic, would be in a Lambda function that defines the specification of the tool, invokes the LLM, and talks to the Vacation Tracker API and storage for our knowledge base and vectors.
I would use the Claude Sonnet 3.7 model on Amazon Bedrock.
I could store the vectors and parts of our knowledge base in the S3 bucket. This is not an ideal long-term solution, but it would work fine for the MVP version.

By using API Gateway, we get all the benefits this service offers, including easy setup for rate limits, Web Application Firewall (WAF), etc. However, as Austen Collins suggested, API Gateway has some significant downsides when building AI Agents. For example, the API Gateway timeout is limited to 29 seconds (AWS allows us to change the timeout now, but changing the timeout can affect account-level throttle limits, etc.), which could be a serious limitation for more complex production-ready agents that can do some longer tasks. Also, we can't stream the response from our Lambda function, so we need to wait for our agent to generate the whole long reply before we can start showing it to the user. Streaming would allow us to show the response as LLM generates it. This is especially helpful for long responses, as our agent starts responding to the user faster and keeps adding text as the user reads (the effect is similar to typing).

Luckily, there's an alternative! AWS Lambda supports Lambda function URLs. It's basically a simple HTTP endpoint in front of your Lambda function. The main benefits of the function URL over API Gateway are that it offers timeouts of up to 15 minutes (it's a Lambda timeout, not an API Gateway timeout anymore) and support for streaming responses. Just what we needed!

However, it comes with many downsides, too. You do not get all the features of the API Gateway, such as built-in rate limits, authorization support, etc. It does not support WAF, either. There are no custom domains for Lambda function URLs. However, you can mitigate some of these downsides by putting a CloudFront in front of the Lambda function, as shown in the diagram below.

Is this an ideal setup? It depends on your use case. It's a good start. There are many other alternatives. For example, we could keep the initial setup, and instead of waiting for the reply with an open HTTP connection, we could send a message to a background job and tell the frontend application that the message is received and that we'll send a reply via WebSockets. There's no out-of-the-box streaming for this setup, too, but it can give you more flexibility and some benefits from both approaches.

In production, we would need to think about our use case, WAF, rate limiting (for our app, for LLM we are using, and for other services we are using), securing the API endpoint (auth token, API key, etc.), monitoring, error logging and handling, storage for the conversation (we do not want to send the whole conversation from the frontend when a user sends a new message), and many other things.

But let's keep things simple.

Show me some code!

The initial idea of this article was to show the code. But here we are, 4000 words later, and I barely explained how agents work. I'll show the most important parts of the code here. I might do a "part 2" article with a code deep dive if anyone reads the article past this point and thinks a complete coding example might be beneficial.

The most important part of the code is a definition of the available tools. The format depends on a model or a service you use, but for Claude Sonnet 3.7 on AWS Bedrock using the Converse API, you can do something similar to the following:

const toolConfig = {
  tools: [
    // We need a similar object for each of our tools
    {
      // The name and the description of our tool
      // A clear description is important because it helps an LLM to select the right tool
      name: 'request_leave',
      description: 'Request leave (such as PTO, sick day, etc.). The leave request will be submitted to the Vacation Tracker application, and sent to your manager (approver).',
      inputSchema: {
        // It accepts JSON, I told you developers love JSON!
        json: {
          type: 'object',
          properties: {
            // And it expects the following properties (see the type and the description for each).
            // For simplicity, I'll show the most important parts only
            startDate: {
              type: 'string',
              description: 'The date when the leave starts. Format: YYYY-MM-DD',
            },
            endDate: {
              type: 'string',
              description: 'The date when the leave ends. Format: YYYY-MM-DD',
            },
            leaveType: {
              type: 'string',
              description: 'The type of leave. For example, "vacation", "sick", etc.',
            },
            reason: {
              type: 'string',
              description: 'The reason for the leave request. For example, "Vacation in Greece", "I am not feeling well today," etc.',
            },
          },
          required: ['startDate', 'endDate', 'leaveType'],
        },
      },
    },
    // Define the other 2 tools here
  ],
}

It's important to provide clear descriptions as that would help an LLM pick the right tool when needed.

I am a big fan of hexagonal architecture (or ports and adapters), and I would use it in a production code. Our business logic does not care about the Claude Sonnet 3.7 model. It does not care about Amazon Bedrock, either. So, I would put the LLM logic in some kind of repository, initialize it, and use it to send a message when needed. That would make my code cleaner to read, easier to test, and allow me to try other models (i.e., Open AI models, which are not available in Amazon Bedrock) without changing the business logic.

However, I'll just show the simplest code example without hexagonal architecture to keep things simple.

// Stripped down example of using the AWS Bedrock SDK to create a simple AI Agent
import { BedrockRuntimeClient, ConverseCommand } from '@aws-sdk/client-bedrock-runtime'
import { toolConfig } from './tool-config.js'

// Initialize the Bedrock client and specify the region and a model ID
const client = new BedrockRuntimeClient({ region: 'us-east-1' })

const modelId = 'anthropic.claude-3-7-sonnet-20250219-v1:0'

// Start a conversation with the user message.
const conversation = [
  {
    role: 'user',
    content: [{
      // The user messsage is hardcoded for this example, but in a real application, you would get it from the user input
      // Also, in practice, users would probably tell us that they want to go on vacation in the first week of July, and the agent would need to ask for the start date
      text: 'I want to go on vacation on the first week of July. Full week, starting June 30th.',
    }],
  },
]

// Create a command with the model ID, the message, and a basic configuration
const command = new ConverseCommand({
  modelId,
  messages: conversation,
  system: [{
    // We would need to provide a more detailed system message in a real application
    text: 'You are Vacation Tracker assistant and you help users to request leaves, see who else from their team is off, check their leave balance and learn about the Vacation Tracker app functionalities. When a user asks to go off for a longer period, assume whole week. Here is the list of available leave types: `PTO`, `SickDay`. Try assuming the correct leave type from the input.',
  }],
  // We pass our tool configuration to Bedrock
  toolConfig: toolConfig,
})


async function run() {
  try {
    // Send the command to the model and wait for the response
    const response = await client.send(command)

    console.log('Response:', JSON.stringify(response))
  } catch (err) {
    console.log('ERROR', err)
  }
}

// We wrap the call in the run function just to be able to use it in a terminal without the deployment
// The actual code would define a Lambda function handler
run().then(console.log).catch(console.error)

To run this code example, you'd need an AWS account with Claude Sonnet 3.7 enabled in the Amazon Bedrock (in the us-east-1 region). When you run it, the response should be similar to the following JSON (inline comments make this JSON invalid, but I added them for easier understanding):

{
  // Just metadata, you can ignore this part at the moment
  "$metadata": {
    "httpStatusCode": 200,
    "requestId": "bc806712-b1b2-40eb-8488-f0085237ebcf",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  // Metrics can be useful, but let's ignore that too at the moment
  "metrics": {
    "latencyMs": 4191
  },
  // Claude Sonnet 3.7 response
  "output": {
    "message": {
      "content": [
        {
          // A message that we can show to our users if we want to
          "text": "Certainly! I'd be happy to help you request leave for your vacation during the first week of July. Based on the information you've provided, I'll submit a leave request for you using the Vacation Tracker application. Let me go ahead and process that for you."
        },
        {
          // Claude Sonnet 3.7 tells us to use a tool!
          "toolUse": {
            // Request params, as defined in the tool configuration
            "input": {
              "startDate": "2023-06-30" ,
              "endDate": "2023-07-07",
              "leaveType": "PTO",
              "reason": "Vacation for the first week of July"
            },
            // The name and ID of a tool we should use
            "name": "request_leave",
            "toolUseId": "tooluse_JaORLwrHSSGifTRMApUrGA"
          }
        }
      ],
      // This is a response from an agent, we need to pass this with the response
      // from a tool to continue the conversation
      "role": "assistant"
    }
  },
  // Claude tells us that it stopped because it needs a tool to continue
  "stopReason": "tool_use" ,
  // Useful metrics for the number of used tokens
  "usage": {
    "inputTokens": 630,
    "outputTokens" : 183,
    "totalTokens": 813
  }
}

As you can see in the JSON above, Claude tells us that it needs a tool to be able to reply (stopReason: 'tool_use'). It gives us a nice message that we can display to our users if we want to, but keep in mind that the agent still works at this point, so users should not send new messages yet. Finally, it gives us the details about the tool we need to use and the parameters we should send to our tool to get the response.

After receiving this response, we should send the API request to the Vacation Tracker API with the provided start and end dates, leave type, and reason. But before that, we probably need to validate the data and get the authentication token for the API.

Before we continue, let's talk about one more thing: request duration. Remember the 29-second timeout in the API Gateway? Well, this request took 4.2 seconds out of these 29. If we add a few hundred milliseconds for the Lambda overhead and a request to a database to get the previous messages in this conversation, we are probably around 5 seconds.

But that's just the first part of this request. Before we reply, we need to call the Vacation Tracker API, parse the response, and call the Amazon Bedrock again. If we are lucky, our agent will need just one tool to be able to reply, so we'll be at 10 or 15 seconds, including saving the conversation to the DynamoDB.

What happens if we have a complex agent that could use multiple tools in one request? We can easily get close or over the 29-second timeout, which could break our agent.

Ok, so, we got the ID of the tool we need to use and the parameters to send to the tool. In this case, an agent wants us to request a leave. In production, we would ask the user to confirm the request details first, but for this simple example, we can send the request straight to the Vacation Tracker API.

Once the Vacation Tracker API responds, we probably want to process the response to make it clear to our LLM because most API responses are not 100% LLM or human friendly. For example, we can add a description, change the field names to be more descriptive, etc.

We could send a new request to Amazon Bedrock with the following conversation:

[
  // The initial user message
  {
    "role": "user",
    "content": [
      {
        "text": "I want to go on vacation on the first week of July. Full week, starting June 30th."
      }
    ]
  },
  // The unchanged assistant message from the previous example
  {
    "role": "assistant",
    "content": [
      {
        "text": "Certainly! I'd be happy to help you request leave for your vacation during the first week of July. Based on the information you've provided, I'll submit a leave request for you using the Vacation Tracker application. Let me go ahead and process that for you."
      },
      {
        "toolUse": {
          "input": {
            "startDate": "2025-06-30",
            "endDate": "2025-07-06",
            "leaveType": "PTO",
            "reason": "Vacation for the first week of July"
          },
          "name": "request_leave",
          "toolUseId": "tooluse_JaORLwrHSSGifTRMApUrGA"
        }
      }  
    ]
  },
  // Tool response
  {
    // Role needs to be "user"
    "role": "user",
    "content": [
      {
        // We provide the tool result object
        "toolResult": {
          // Tool ID
          "toolUseId": "tooluse_JaORLwrHSSGifTRMApUrGA",
          "content": [
            {
              // Response is a JSON object with a custom payload
              "json": {
                // For example, the API tells us that the request was successfully sent
                "success": true,
                // We have the request details
                "request": {
                  "id": "request_12345",
                  "status": "pending",
                  "startDate": "2025-06-30",
                  "endDate": "2025-07-06",
                  "leaveType": "PTO",
                  "reason": "Vacation for the first week of July"
                },
                // And also the remaining quota
                "quota": {
                  "remaining": 10,
                  "used": 5,
                  "total": 15,
                  "unit": "days"
                }
              }
            }
          ]
        }
      }
    ]
  }
]

When we send this request to Claude, the reply will probably be similar to the following JSON:

{
  // Metadata, ignore
  "$metadata": {
    "httpStatusCode": 200,
    "requestId" : "de4dad99-1d57-46bd-ad71-c69ebe853756",
    "attempts": 1,
    "totalRetryDelay": 0
  },
  // It took almost 9 seconds to reply!
  "metrics": {
    "latencyMs": 8985
  },
  // The actual response object
  "output": {
    "message": {
      "content": [
        {
          // The text we want to show to the user
          "text": "Great news! I've successfully submitted your leave request for your vacation. Here are the details of your request:\n\n- Leave Type: PTO (Paid Time Off)\n- Start Date: June 30, 2025 (Monday)\n- End Date: July 6, 2025 (Sunday)\n- Reason: Vacation for the first week of July\n- Status: Pending approval\n \nYour request has been submitted to the Vacation Tracker application and sent to your manager for approval. The request ID is request_12345.\n\nAdditionally, I can see from the response that your current leave balance is as follows:\n- Total PTO: 15 days\n- Used: 5 days\n- Remaining: 10 days\n\nThis request will use 6 working days of your PTO (assuming a standard Monday to Friday work week). Please note that your request is currently pending approval from your manager. You should receive a notification once it's been reviewed. \n\nIs there anything else you'd like to know about your leave request or the Vacation Tracker application?"
        }
      ],
      "role": "assistant",
    }
  },
  // Claude says that the request was successfully fulfilled
  "stopReason": "end_turn",
  // Usage data
  "usage": {
    "inputTokens": 901,
    "outputTokens": 236,
    "totalTokens": 1137
  }
}

To make it a bit more readable, here's the AI Agent response:

Great news! I've successfully submitted your leave request for your vacation. Here are the details of your request:

Leave Type: PTO (Paid Time Off)
Start Date: June 30, 2025 (Monday)
End Date: July 7, 2025 (Friday)
Reason: Vacation for the first week of July
Status: Pending approval

Your request has been submitted to the Vacation Tracker application and sent to your manager for approval. The request ID is request_12345.

Additionally, I can see from the response that your current leave balance is as follows:

Total PTO: 15 days
Used: 5 days
Remaining: 10 days

This request will use 5 working days of your PTO (assuming a standard Monday to Friday work week). Please note that your request is currently pending approval from your manager. You should receive a notification once it's been reviewed.

Is there anything else you'd like to know about your leave request or the Vacation Tracker application?

As you can see, our AI Agent is a bit more chatty and friendly. It made up some parts of the reply (for example, we never mentioned the notification), but the response generally looks good.

In production, we probably want an LLM to reply in a structured way so we can display a predefined UI for the leave request along with a short, friendly message. Also, we need to be careful with dates and other numbers, as LLMs do not care about numbers. Numbers are close in the vector space. For an LLM, 4 is similar to 5 or even 42, but 4 or 5 used PTO days make a big difference for our users.

And that's it! We built an AI Agent again. This time with code. It's a simple agent, more like a proof of concept. But it's still an agent.

Summary

After a long article, I'll keep the summary short.

Before we do a quick summary, you should check 2 relatively fresh things related to AI Agents.

First, my Twitter feed has been buzzing about MCP servers in the past few weeks. Model Context Protocol (MCP) is an open-source protocol that standardizes how applications provide context (and tool specification) to LLMs, and you can read more about it here: https://docs.anthropic.com/en/docs/agents-and-tools/mcp.

Second, OpenAI has some new tools for building agents. Read more about these tools here: https://openai.com/index/new-tools-for-building-agents/.

To end this long article, I just want to remind you that AI Agents sound very complicated, but they are actually like LLM "while loops" with tools.

You have all the skills you need to build these tools. So, go, build tools and agents, and have fun!

Serverless: a backend thing that gives superpowers to frontend developers

Slobodan Stojanović — Wed, 08 Jul 2020 09:47:14 +0000

Learning web development is hard. Most of the time, you start with HTML and spend much time to learn all of its tags. However, your web page looks like it's early 1990. You have to learn CSS to make it beautiful. It seems simple until you try to align two elements the way you want.

Finally, you move on to JavaScript to make your new web page interactive. Before you even grasp into its dynamic world of frameworks, you try to do something simple, such as calculating a sum of 0.1 and 0.2, and you don't get the result you expected. If you spent a few days trying to learn JavaScript, you most likely met its weird side.

However, if you are tenacious enough, you'll manage to build your first web application. And when you do that, you'll feel like you have superpowers. And you do have superpowers!

If you are like me, your first app will probably not be an award-winning web application, but the moment you finish it, it'll look like the best and most complex application ever. You should be proud of it, and show it to your friends.

You finally show your app to your friends, and they are happy for you. Then you want to show it to your good friend that lives far away. Wait, how do you send your masterpiece to your friend Ben from Australia?

How to make your web app available

After a quick online search, it's obvious; you need a server!

That's easy. You quickly find and rent one. Now you can just put your app in the cloud and make it available to your friend Ben and the rest of the world.

Well, not so fast. Servers are not boxes where you can just put your app and make it globally available. Servers are like pets; they require more of your attention and knowledge. You probably got an operating system with your cloud server, but you need to install and configure a few other libraries to host your application. Apache, Nginx, domains, SSL, and other weird words you barely understand. Do you need to learn all these things to be able to show your app to your friend Ben? You learned so much, and now it seems it's not enough to finish a simple real-world application.

Huh, is there a better way? Wouldn't it be great if you could simply click on some magic button and make your app publicly available?

How should hosting static web apps work

In an ideal world, once you created a web application, you should be able to click on some button or run a command from your terminal to make your app available to everyone. It should also be cheap. Even better, it should be free if almost no one uses your app. Why would you need to pay $10 per month to show your app to your friend Ben? That's not much money, but it's enough to buy that excellent video course that teaches you new skills.

Also, what if your app suddenly becomes popular? That happens too. Servers are not good at handling the instant popularity of the web applications they host. They can support many visitors, but at some point, they start being shy and slow, until they crash. Then you have to deal with a new set of problems.

In an ideal world, your app should be able to handle instant popularity by auto-scaling and caching, without your help.

After a few more minutes searching the web, you can find several services with a similar set of features, such as Netlify, Github Pages, or Amazon Simple Storage Service (S3).

However, there's another thing with a similar set of features and a weird name: serverless.

What is serverless, anyway.

So, what is serverless? Is it some magic where you host your app without servers, similar to peer-to-peer networks? Or you need fewer servers? You saw a thread where someone claims that there are even more servers with serverless, and you are probably confused.

To understand serverless, let's go a step back and see what do we need to do to make our apps function correctly.

To have a fully functional web application, you need to buy or rent a server and make sure it has an operating system. Then you need to set up your server and install the necessary tools and libraries, such as Nginx or Apache. Then you often need some frameworks, etc. At that moment, you can finally think about your application's business logic and code. Once your code is ready, you need to make it available by deploying it to your server. However, that's not all; you need to make sure your application always works by monitoring it. Also, from time to time, you need to manage both your server (software updates and security patches) and your application (new features and bug fixes).

Long to-do list just to make sure your app is delivered the way it should. As an exercise, if you are building that app for a non-technical client, show that list to your clients and ask them what's important to them. I am sure that the list of essential things is much shorter, and it'll probably include business logic, making sure that the application works (monitoring), and making sure the app is well maintained (regular new features and bug fixes).

As a developer, you should focus on the things that are important to your client and application end-users. The cloud helps you with taking care of some of the less critical elements from your list: renting a server and managing the operating system is easier than ever, and you don't need to care about that anymore.

Serverless is a natural next step of cloud evolution. Its idea is to take care of other non-essential things from your to-do list. With serverless, you don't need to set up servers anymore, you write your business logic, and the platform manages the operating system and all necessary libraries and frameworks on top of it. The platform also takes care of updates and security patches of your OS, libraries, and frameworks, and gives you an easy way to deploy your application.

Serverless is adding another layer of abstraction and helps you to focus on the things that are important for your client and end-users: business logic. You still need to be on top of monitoring and deployment procedures, but there are many tools to support you.

If you still can't get over the name of this next step of cloud evolution, I understand. It's not the best name ever. As you remember, there are two hard things in computer science, and naming things is one of them.

The best explanation I found is the one Gojko Adzic gave in one of his excellent serverless articles:

It is serverless the same way WiFi is wireless. At some point, the e-mail I send over WiFi will hit a wire, of course, but I don’t have to pull along a cord with my phone.

Benefits of serverless

Serverless has many benefits, besides managed infrastructure. It has auto-scaling, auto-failover, and per function isolation. It's easy to start with, and it's cheap most of the time. You'll see some arguments that serverless can be expensive on a large scale, but most of these arguments directly compare the cost of infrastructure and ignore all the things that you don't need to do anymore.

While we are talking about the cost of serverless, it's important to note that the essential benefit of serverless is its pricing model. Why? Because you pay per usage, and if no one uses your app, you'll pay $0. Besides that, most of the vendors have low prices with generous free tiers. For example, the AWS Lambda function costs you $0.2 per million function executions, and the first million are free. Other services and other vendors have similar pricing.

Explaining why the serverless pricing model is essential would require much more than a few paragraphs of text. However, the pricing model doesn't give superpowers to front end developers, at least not directly. If you want to learn more about serverless economics, you can read about it in many articles, including the following:

FinDev and Serverless Microeconomics by Aleksandar Simović
Micro-Optimization: Activity-Based Costing for Digital Services? by Mark Schwartz
Serverless architectures: game-changer or a recycled fad? by Gojko Adzic

Superpowers

The pricing model doesn't give superpowers to front end developers. But what does?

What would be the perfect superpower for a front end developer, besides an ability to float the elements in CSS, and understanding what "this" is in JavaScript?

There are many potential candidates, but one of the top picks is a mythical full-stack developer.

What is a full-stack developer? In theory, it's a developer capable of building and delivering an application from scratch. However, as Carl Sagan said:

If you wish to make an apple pie from scratch, you must first invent the universe.

In practice, a full-stack web developer is often a back end developer that knows jQuery or basics of a popular front end framework, or a front end developer capable of creating a web API using Express.js or some other popular web framework. That's impressive, but in many cases, these skills are far from the skills required for delivering a production-ready web application. It takes time to learn both front end and back end. However, being both front end and backend developer doesn't make you a full-stack developer anymore. If you want to create and provide a production-ready web app, you also need DevOps skills.

Fortunately, that's where the serverless jump in and help a front end developer to gain superpowers. How? It's storytime!

CodePen

If you are a front end developer, there's a big chance you heard about CodePen. In case you didn't, CodePen is an online community for showcasing user-created HTML, CSS, and JavaScript code snippets. It functions as an online code editor and open-source learning environment, where developers can create code snippets, creatively named "pens."

In CodePen, you can write your code snippets using TypeScript, SASS, LESS, and many other popular libraries and tools. As your browser doesn't understand most of these libraries out-of-the-box, the platform does some magic in the background and converts your code snippet to plain HTML, CSS, and JavaScript that your browser understands.

CodePen is awesome. However, they don't have an unlimited budget and a big team. At the moment we interviewed their team for our book, they had a single DevOps person in their team, and more than 250 million requests to their preprocessor API each month.

A preprocessor is a service that translates tools and libraries that browser doesn't understand to plain HTML, CSS, and JavaScript. For example, TypeScript to JavaScript, or SCSS to CSS.

Their original architecture was based on two monolithic Ruby on Rails applications—the main website and another application dedicated to preprocessors—and a single, relatively small database service. After implementing their initial idea, they realized some significant downsides of their plan. First, some pens go viral fast, and they need to scale quickly, but to keep the infrastructure cost as small as possible. Another, even more, the critical downside was the isolation of their preprocessors, or to be more precise lack of it. Some of their users were creative and managed to run SASS and LESS functions that had access to the file system and interfere with other preprocessors.

They started investigating how to separate the users' code execution for security purposes. That was the first time they heard about AWS Lambda: their DevOps engineer suggested it as a possible solution. Initially, their developers rejected the idea because they didn't see the point, thinking it would be a hassle to set up and configure a new environment.

Then, one day they wanted to add a new preprocessor and decided to try out this "Lambda concept." One of their front end engineers used Claudia.js to create and deploy a new preprocessor. And then they fell in love with serverless.

Soon after, they migrated all of their preprocessors to AWS Lambda and Amazon API Gateway. Now each of their preprocessors lives in a single AWS Lambda function, and they are fully isolated. If two users run the same pen at the same time, each request spins a single Lambda function, and both services run in parallel but fully isolated by design.

CodePen preprocessors handle more than 250 million API requests per month, and at the time of the interview, they had more than 200,000 requests per hour in peak.

How many DevOps team members you need to be able to handle 250 million preprocessor API requests per month?

If you are CodePen, the answer is zero. Yes, you heard that correctly — zero.

Preprocessors are developed, deployed, and maintained by their front end team. They still have one DevOps person that maintains their application server and the database. Their monthly bill for AWS was a bit more than $1000 per month, which seems high, but it's still a fraction of the cost of the DevOps engineer and infrastructure with virtual servers or containers. Also, they could reduce their bill by half if they apply a few optimizations.

Serverless gave their front end team real superpowers.

Vacation Tracker

Another good example is a startup I am working on, Vacation Tracker.

Vacation Tracker is a leave management system where employees can request leaves and manage their days off with easily within Slack. With our tool, you can check your remaining vacation days and request a new vacation from Slack. You can also receive a confirmation without leaving Slack. No need to remember another password, learn a new tool, or manage your team from another tool.

The idea started with a company hackathon, then we created a simple serverless prototype, published a landing page, and forgot about it. However, a lot of teams signed up for the private beta, and we decided to build a tool.

The initial team wasn't a team, as it had just a single full-time front end developer who never worked with serverless before. Our developer quickly learned how to create an API using Claudia API Builder, Amazon API Gateway, and AWS Lambda.

After the initial fight with the learning curve, we gained a decent velocity with fast development cycles. Our app is fully scalable out-of-the-box, and our AWS bill is less than $100 per month, despite having almost 500 paying teams and another a lot of other organizations waiting for the MS Teams beta (which will be available in the next few weeks).

Our team has grown, but the core of the app is still maintained and developed by front end developers with superpowers. As our team is learning fast, we can quickly apply things we learned to our existing services, as everything is developed in isolation using AWS Lambda and other AWS services.

We also ended up with some reusable parts for our next products.

Where should I start?

If you like these stories, you probably want to know where to start. But to keep this article reasonably short, I'll leave that for the next post in this series.

Handling webhooks with EventBridge, SAM and SAR

Slobodan Stojanović — Tue, 30 Jun 2020 16:29:25 +0000

Applications I worked on in the last decade were rarely isolated from the rest of the world. Most of the time, they had many interactions with other applications out there. From time to time, some of these integrations are using WebSockets, which makes our integration realtime. But much more common integration is using webhooks to send us new changes, and give us some API or SDK to allow us to communicate in the other direction. There’s a big chance that you worked with many similar integrations, such as Stripe, Slack, Github, and many others. A typical integration looks similar to the diagram below.

A quest to a cleanest webhook integration

In Vacation Tracker, the leave tracking application I am working on, we have a lot of external integrations. We integrate with Slack for user management, and we use Slack chatbot as one of the entry points to our app, and we are expanding to other platforms. We outsourced payments to Stripe, emails to MailChimp and Customer.io, and so forth. Many of these integrations require webhook integration, and from the very beginning, we are on a quest to the clean and simple way to manage our webhooks.

From its early days, Serverless Application Repository (SAR) sounds like an excellent tool for isolation of the common patterns in our serverless applications. If we do a similar payment integration to multiple applications, why don’t we move that set of functions and services to a place that allows us to reuse it quickly, both privately and publicly?

Our initial idea was to put all of our integrations as separate SAR apps, open-source some of them, and keep the rest of them privately. Something similar to the following diagram.

Not a bad for an initial idea, but we quickly realized that there is a common thing in a lot of our potential apps. As you can guess: a webhook.

What's an easy way to handle a webhook in a serverless application? We need some API; we can start with an API Gateway. And we need some integration point with the rest of our business logic. One of the logical picks would be Amazon Simple Notification Service (SNS). And we need a Lambda in between.

Wait, do we need that Lamdba function?

It seems that we do not need it, because API Gateway can talk directly to multiple services, including SNS, using a service integration. You need to write a "simple" template using the Velocity Template Language (VTL).

What's VTL? I would say it's an alien language (well, its Java-based 🤷‍♂️) insanely hard to test in isolation in a serverless application, especially in AWS CloudForamation and AWS Serverless Application Model (SAM) templates.

Our webhook would look similar to the following diagram.

API Gateway gives us a REST API, with a lot of awesome integrations and tricks. However, an API required for a common webhook is quite simple. We can use Application Load Balancer instead, but that requires a few more modifications of our app, and time spent on these modifications is time we wasted for working on our business logic.

Fortunately, AWS announced a new API Gateway service on re:Invent 2019 conference, called HTTP APIs for API Gateway. HTTP APIs are a lighter, cheaper and slightly faster version of API Gateway's REST APIs. HTTP APIs don't support VTL templates and service integrations at the moment, and we need our Lambda function back. At least until AWS implements service integrations, or add Lambda Destinations for synchronous invocations. Back to the drawing board! Our SAR app should look similar to the following diagram.

The new architecture looks good. But after integrating many webhooks, we'll end up with a lot of SNS topics. SNS topics are serverless, we pay for used capacity only, but each of them come with a custom event structure, which makes documenting and integrating all event schemas harder down the road.

It would be great if AWS had an event bus that would make this easier, right?

Meet Amazon EventBridge, a serverless event bus that connects application data from your apps, SaaS, and AWS services. Yes, something like an enterprise service bus.

Why EventBridge

Events are the core of the common serverless application. We use events to trigger our functions; we send them to queues and notification services, we stream them. But events are also the core of almost any application.

Let's take Vacation Tracker as an example. When you request a leave or a vacation in your company, that's an event that requires some action. Response to your request is another event. When your leave starts, that's an event, too.

EventBridge represents a new home for your events. We can use it to integrate with some of the third-party services or build our integrations.

Here are a few reasons why we would pick EventBridge instead of SNS:

We can connect Amazon SNS with a few other services directly. At the moment, EventBridge supports 20 different targets, including Lambda functions, SQS, SNS, Kinesis and others.
It gives us a single place to see and handle all of our event subscriptions.
For unsuccessful deliveries, SNS retries up to three times. EventBridge does retries out of the box for 24 hours. Both SNS and EventBridge support Lambda Destinations.
EventBridge has Schema Registry for events. It supports versioning, and it has an auto-discovery and can generate code bindings.

Enough to give it a chance.

The solution

Our SAR app should look similar to the one we already have, with one crucial difference: we don't want to create an EventBridge event bus in the SAR app. We'll use the same event bus for multiple events, so it's better to keep it outside of the SAR app and pass the reference to it to the SAR app.

As you can see in the following diagram, we'll have the API Gateway's HTTP API and a Lambda function in our SAR app. That app receives webhook events from any external source and passes it to our event bus. We'll route the events from our event bus to functions or other services.

Let's implement it.

EventBridge integration with AWS SAM

We are using AWS SAM for our serverless apps. Until SAM documentation gets some support from Amazon Kendra, searching for EventBridge support can take some time.

After a few minutes of digging through the documentation and Github issues and pull requests, we can see that SAM doesn't have support for EventBridge out of the box. Fortunately, CloudFormation got support for EventBridge resources a few months ago.

CloudFormation has support for the following EventBridge resource types:

The AWS::Events::EventBus resource creates or updates a custom or partner event bus.
The AWS::Events::EventBusPolicy resource creates an event bus policy for Amazon EventBridge, that enables your account to receive events from other AWS accounts.
The AWS::Events::Rule resource creates a rule that matches incoming events and routes them to one or more targets for processing.

We'll need AWS::Events::EventBus to create a new event bus for our app.

But before we add an event bus, make sure that you have AWS SAM installed, and then run the sam init -n stripe-webhook -r nodejs12.x --app-template hello-world command from your terminal to create a new SAM app. This command creates the "stripe-webhook" folder with the "template.yaml" file and the "hello-world" function.

Open the "template.yaml" file in your favorite code editor, and add the following resource at the top of the Resources section:

PaymentEventBus: 
  Type: AWS::Events::EventBus
  Properties: 
    Name: paymentEventBus

The resource above creates an EventBridge event bus named "paymentEventBus". Besides the "Name" property, the AWS::Events::EventBus accepts the "EventSourceName" property, required when we are creating a partner event bus. Since we are creating a custom event bus, we do not need it.

Then we want to add a subscription for our event bus to the Lambda function. We can do that using the CloudFormation AWS::Events::Rule resource, however, the more natural way is using the SAM's CloudWatchEvent event. To add a subscription, replace the "HelloWorld" resource with the following one:

ChargeHandlerFunction:
  Type: AWS::Serverless::Function
  Properties:
    CodeUri: hello-world/
    Handler: app.lambdaHandler
    Runtime: nodejs12.x
    Events:
      OnChargeSucceeded:
        Type: CloudWatchEvent
        Properties:
          EventBusName: paymentEventBus
          Pattern:
            detail:
              body:
                type:
                - charge.succeeded

This resource triggers our HelloWorld function when our event bus receives the "charge.succeeded" event from a Stripe webhook, or any other event that contains the following:

{
  "body": {
    "type": "charge.succeeded"
  }
}

The powerful thing about EventBridge is that we can easily subscribe to all events that contain a specific pattern in the request body or headers. For example, to subscribe to both "charge.succeeded" and "invoice.upcoming" events, modify the subscription pattern to look like the following one:

Pattern:
  detail:
    body:
      type:
      - charge.succeeded
      - invoice.upcoming

As we don't use an API Gateway anymore, we need to update the HelloWorld function to log the event. To do so, open the "hello-world/app.js" file in your code editor, and replace its content with the following code snippet:

exports.lambdaHandler = async (event) => {
  console.log('RECEIVED EVENT', JSON.stringify(event));
  return true;
};

We also want to add our webhook endpoint SAR application. To do so, add the following resource to the Resources section of the "template.yaml" file:

StripeWebhook:
  Type: AWS::Serverless::Application
  Properties:
    Location:
      ApplicationId: arn:aws:serverlessrepo:us-east-1:721177882564:applications/generic-webhook-to-eventbridge
      SemanticVersion: 1.3.3
    Parameters:
      EventBusName: paymentEventBus
      EventSource: stripe-webhook

Before deploying the application, we need to modify the output to print the webhook URL. To do so, replace the Outputs section of the "template.yaml" file with the following:

Outputs:
  WebhookUrl:
    Description: "The URL of the Stripe webhook"
    Value: !GetAtt StripeWebhook.Outputs.WebhookApiUrl

To deploy the application, open your terminal, navigate to the project folder, and run the sam deploy --guided command to deploy the application. Once you follow the instructions, SAM deploys your app, and prints the webhook URL in the output.

Testing the webhook

To test this webhook, you can navigate to your Stripe dashboard, switch it to the test mode, then click on the "Developers" link in the sidebar, and select the "Webhooks" from the sub-menu. Click the "Add endpoint" button. Paste the webhook URL you copied from the sam deploy output in the "Endpoint URL" field, and select the "charge.succeeded" event from the "Events to send" dropdown. Finally, click the "Add endpoint" button to add a new webhook, and the "Send test webhook" button to test your webhook.

You can confirm that your event was successfully received by listing the CloudWatch logs for the "ChargeHandlerFunction" function. To do so, navigate to the CloudWatch logs in the AWS Web Console, or use the sam logs command.

If you do not have the Stripe account, you can send the POST request to the webhook URL using CURL or Postman. Just make sure you send the Content-Type: application/json header and the body similar to the following code snippet:

{
  "body": {
    "type": "charge.succeeded"
  }
}

SAR application

As you can see in the Github repository, our SAR app is simple. It receives the event bus name through the parameters, defines a Lambda function and an API Gateway's HTTP API, and outputs the webhook URL.

To be able to send events to the event bus, the Lambda function requires the following policy:

Policies:
  -
    Version: 2012-10-17
    Statement:
      -
        Effect: Allow
        Action:
          - events:PutEvents
        Resource: '*'

This policy allows our function to send the events to the EventBridge event buses. This policy does not allow us to add the "events:PutEvents" action to a specific EventBus, so we need to pass '*' as a Resource value.

To send an event, we use the "PutEvents" property from the EventBridge class of the AWS SDK for JavaScript.

That's all folks

EventBridge promises an easy but powerful way to organize both internal and external events in our serverless applications. In combination with SAR, we can create reusable parts of the application and potentially save much time.

However, EventBridge is not a silver bullet. By using it and its Schema Registry, we give all of our event structure to Amazon. With its current velocity, Amazon can sooner or later come after any of our businesses, and the Schema Registry could make that easier. Fortunately, EventBridge upsides and promises are way higher than those risks. Also, avoiding the particular service or choosing another cloud vendor doesn't help you a lot anyway.

There are a few other downsides of the EventBridge at the moment. The main one is the debugging, but I am sure AWS will improve that significantly in the coming months.

Build something awesome using the EventBrigde, and let us know once you do it! Just make sure you check the service limits (which are quite high) before you lock you in a solution not made for your problem.

What problems does serverless solve?

Slobodan Stojanović — Wed, 24 Jun 2020 10:02:41 +0000

The problem serverless is solving is not a technical problem (unless you work for a serverless vendor, such as AWS, Azure, and others). It's way more important than that because it directly affects the business value of the product you are building.

How? Let me try to tell you a story. But before we begin, I want to warn you that serverless is not the main character in this story, but it is one of the heroes of the story, I promise!

In 2018, we finally decided to build a simple product that will allow us to manage leaves in our company without having a complicated and expensive HR system. We were a team of 15 people, and we needed a simple tool that will do just one thing.

We tried to find something simple, but I guess we didn't try hard enough because everything was too complex and expensive for us at that moment. While exploring an idea to build that tool, we talked to our friends from other small and medium businesses. They had the same issues, and some of them told us that they would love to use a tool similar to something tried to build. And the idea sparked in our minds -- this could be a real product!

But to be able to solve the problem, you need to understand it well enough. For the sake of simplicity, I'll show only the basics of a single user journey.

Meet Anna; she is an excellent manager in the fast-growing smart house maintenance startup Sonic Screwdriver. Besides many other jobs, Anna needs to manage leaves for her team. That includes knowing who is not working, tracking the number of remaining and used PTO days, and managing leave requests. Besides that, she needs to plan future leaves and team capacity, manage leave policies in their intergalactic team, and many more responsibilities.

To be able to stay on top of the user leaves, Anna is using a bunch of products, such as calendars, spreadsheets, different communication channels, etc. This process includes a lot of manual work. Also, calendar and spreadsheets don't always give you the information just-in-time. You often realize that your coworker is ooo at the moment you need that final PSD file. At that moment, they are already enjoying a mojito at the beach half the world away.

Well, this defines a problem, but to solve it, we need to analyze it and act on it. To do so, we use Wardley maps, maps that help us with the representation of the landscape in which our business operates. If we put the previous value chain in the Wardley map, it looks similar to the following image.

The Y-axis represents a value chain, on top are the things that our customer sees, and on the bottom are things invisible to them. The X-axis represents an evolution of components. Everything starts with genesis; for example, an idea of electricity exists from ancient times. Then people start building custom solutions, and at some point, convert to a product. Finally, things end up as commodity or utility, for example, electricity is everywhere today, and we pay only for what we used.

If we want to build a leave management app, that app becomes a product for Anna, and the map starts looking at something similar to the following image.

Our app becomes just a dot on the map for Anna, and everything below it is entirely invisible to her and her team. But that dot is another massive map for us, that has its components, value chain, dependencies, etc. And we need to manage things below it, in the value chain.

When you work on a startup, your capacity and budget are limited (especially if you bootstrap your business as we did). You need to pick your battles very carefully. Each solution is the source of the next problem.

Is it more relevant to Anna to be able to get just-in-time notifications and synchronize team calendar, or to know that your app has a beautiful data center? That was the obvious question. But we can ask less obvious ones, for example, does Anna care about our excellent framework or Kubernetes? I don't think so. I don't care about Dev.to tech stack, and I am a programmer.

So what can we do to be able to free up our time for the things Anna cares about? We'll we can outsource some of our responsibilities. But what should we outsource? Leave tracking? Our business logic? Nope, that's the core of our business. However, we can try to outsource everything else, such as user management (we do that through Slack or Microsoft Teams), sending emails (MailChimp, or even better customer.io), infrastructure, etc.

Wait, did I just said infrastructure?

Yes, and here comes the hero of the story -- serverless. Let's see the map first.

As anything else, compute evolves and moves to the far right of our evolution axis. With compute as a commodity, servers became a commodity, then OS too. Which OS is running in your infrastructure? Who cares while it works. We don't install things manually anymore. One thing sliding to commodity pulls many other things and unlocks new opportunities.

How do serverless fit that image?

Serverless is all the things that you'll not need to take care of in the future. Do you need to send a leave request? Slack or web app will send it to some API endpoint, and your provider will trigger a function in the background that will execute your business logic. That function will save the leave request data to some database and then send some notification. However, all of these things will be fully managed and mostly hidden from us, even if we are developers.

You probably don't assemble your new computer hardware anymore unless you are nostalgic, and you want to have fun. Why would you assemble your infrastructure? Other people do that better than us (or at least me), and in most cases, the business logic of products we build don't care about infrastructure.
But if we want serverless to fulfill that goal, we need to adopt new coding best practices for the new evolving ecosystem.

Does that mean that serverless is not ready?

Not at all! We run our product on serverless from the beginning. Last month we served more than 13M requests, and we paid around $100 for our infrastructure (I am lying, we paid $0 because we have credits). With almost 500 happy teams using our product, I would say serverless is ready. And I am sure many others would agree, including products such are Codepen, MindMup, and many, many others.

If we don't need to babysit our infrastructure, we can focus on other things essential for Anna and other awesome people using our app. And again, we can take a look at our map, and focus on things that are still custom-built for them.

But the map is just the beginning, and as the creator of Wardley maps, Simon Wardley says, all maps are imperfect. You need to evolve it and also to communicate with people using your product, because in the end, without them you don't have a product.

By the way, if you have the same problem we had, or you want to see the result, check our product: Vacation Tracker.

Before we finish, you might ask what about containers? Are they also trying to solve the same problem? Is serverless better then containers, and if yes, why?

I think that both containers and serverless are leading us toward the future of app development. What's that future? I don't know; it depends on all of us.

Maybe we'll build our apps by talking to the computer, or something completely different, we'll see. But I am 100% sure it will not require us to manage the infrastructure.

This evolution will have significant inertia, mostly by you, me, and other people that were early adopters of the things that are now standards. But like in all other cases, the technology will evolve regardless of our opinions.

If you pick containers or serverless, you'll be a step closer to that bright tech future. Serverless might not be perfect for all use cases at the moment, but I saw the incredible evolution of that ecosystem in the past five years. I also met many people way smarter than me working on taking it to the next level. Also, remember that with serverless, you'll be slightly more on the right side of that Wardley map up there, probably just behind that big inertia wall. And all things are slowly leaning that right side. Pick your battles very carefully, and build excellent products.

Writing testable serverless apps and preventing vendor lock-in using hexagonal architecture

Slobodan Stojanović — Thu, 18 Jun 2020 15:27:08 +0000

What's the scariest thing about serverless? The answer to that question depends on the person you ask and timing.

In the early days of serverless, people would mention long-running tasks. However, with AWS Lambda 15 minute timeout, AWS Fargate, and many other announcements, that problem is already solved. People who work with medical apps, and other industries that are dealing with sensitive data, would probably mention compliance, but serverless providers are adding support for many different compliances regularly to their platforms. What about binaries and large dependencies? That was an annoying problem, but then AWS introduced Lambda Layers. Cold starts? If you still have cold start issues, you either use VPC, something peculiar, or we are using serverless in a completely different way. If you are running your Lambda functions in VPC, AWS has some good news for you.

Maybe Node.js? Just kidding, I love Node.js!

However, it doesn't matter if you are talking to the back end developer, system architect, or a business person, there's one thing that always pops up, and it's often followed by a few seconds of silence.

What about big bad vendor lock-in?

What's vendor lock-in?

If a few seconds of scary silence doesn't scare you away, you can ask yourself what's that mysterious vendor lock-in, anyway?

If you check Wikipedia, you'll see the following definition:

In economics, vendor lock-in, makes a customer dependent on a vendor for products and services, unable to use another vendor without substantial switching costs.

As every definition, this sentence is either too boring and makes you fall asleep, or it opens a lot of other questions. One of the follow-up questions can be "how does vendor lock-in in cloud computing works?"

Let's say that you need a server. I don't know why, you are weird, but that's not important at the moment.

So you need a server. You can either buy it or rent it. You can probably try to build it from scratch, but to do that you must first invent the universe, and that's far beyond the scope of this article.

Let's say that you are reasonable enough, and you decide to rent a server. You find the guy with many servers, let's call him Jeff, and you rent a server from him. Because you are weird, you call your server "the cloud," but in reality, it's just a regular server somewhere in Jeff's basement.

Jeff is smart, and he knows how you and some other weird people use his servers. As most of you have some database, he introduces a cloud database service. Same with storage, computing, and even with machine learning services.

As Jeff has enough clients, he decides to charge you for real usage of his services. That means that you pay only for the services that you use. Should I even mention that you love his offering?

But, what if Jeff is a villain?

Maybe he is just waiting until you fully embrace his cheap and beautiful cloud services, and implement them deep into the business logic of your web application. Then; a few seconds of scary silence; he drastically increases the price of his services.

If that happens, I guess you'll not be in love with Jeff's cloud anymore. Your wallet will not be happy, either.

Luckily, it's not all over! There's another guy with many servers, let's call him Bill. He also has some cloud database, compute, storage, and other similar services. Bill is also pricing his services per usage, and it seems that your app could work fine in Bill's basement and that you would be happy again.

New hope arises. Can you move your app to Bill's basement, pardon, cloud?

Well, you can. However, that's not easy, as Bill's database service doesn't work the same way Jeff's database service work. Same with other managed services. To migrate your app to Bill's servers, you'll need to adjust it.

Do you remember the moment you decided that testing is tedious and that you can live without automated tests? Also, all those moments when you cut corners in the application architecture, to be able to finish some non-crucial feature a few hours faster?

All of your bad decisions now make the migration even more expensive than keeping your app in Jeff's basement.

Well, that's the moment you realize what the true meaning of cloud vendor lock-in is.

How to fight vendor lock-in?

So, how do you fight vendor lock-in? In most of the cases, you fight it the same way you fight monsters under your bed.

The first step is to face your fears, and give them the right name. In the cloud, the right name for vendor lock-in is switching cost. As Mark Schwartz, Enterprise Strategist at AWS, says in his excellent article "Switching Costs and Lock-In":

My train of thought went like this: the term “lock-in” is misleading. We are really talking about switching costs.

Is it? I'll answer with another quote from Mark's article:

Switching costs have existed throughout the history of IT. As soon as you commit yourself to a platform or a vendor you will have switching costs if you later decide to change. If you choose Java and then migrate to Node.js, you will have a cost.

My experience is similar. In our product, Vacation Tracker, we switched many things. We migrated most of our app from MongoDB to DynamoDB, migrated some Express.js parts of the app to serverless on AWS, entirely changed some services. Do we have issues with vendor lock-in? Sometimes we do! However, it's our choice, and it's probably not a vendor lock-in you would expect.

Even though our app is almost 100% serverless on AWS, we don't have any issues with cloud vendor lock-in. However, our app integrates deeply with Slack, and sometimes, even the smallest change on Slack's platform can affect our product.

So, how can we fight cloud vendor lock-in? First, we can fight it by asking ourselves the right question — how do we keep our switching costs reasonably low?

To keep our switching costs low, we need to start with better planning. How low should our switching costs be? That depends on how likely we need to switch to some other platform. So far, AWS lowered the prices for their cloud services more than 15 times, and they never raised the price of any of the products. I don't think the risk that they'll increase the prices significantly is high. Even if they do, and our infrastructure cost rise 100 times, we'll pay less than $100 per month. Should we even care about that?

If the risk is high enough, so it needs to be planned for, how much would switching cost? The cost depends on your architecture, but in our case, spending a few weeks on migration would not have a significant impact on our business, so I think our switching cost is reasonably low.

Once you finish initial planning, it's time to consider some good architecture practices and deployment procedures that allow you to evolve your application, and to make necessary migrations in the future less painful and expensive. Deployment procedures are beyond the scope of this article, and we'll probably discuss them in some of the future ones (you can always subscribe to our newsletter on Vacation Tracker website), but even the deployment procedures often depend on a decent app architecture and how testable is your app is.

Designing testable serverless apps using hexagonal architecture

I mentioned testing, but why do you need to test your application if it's serverless and it scales automagically? Your infrastructure might be fully managed, but your business logic and code are not. You can, and will have bugs in your serverless application. The difference is that your bugs will not crash your infrastructure, but they can scale automatically.

Most of the time serverless applications are not entirely isolated monoliths without integrations. Instead, they contain many services interacting with each other and with external dependencies. For example, our application is deeply integrated with Slack, and the central part of our integration looks similar to the following diagram. Slack sends webhook events to the API Gateway. Then we route them to different Lambda functions that handle different scenarios, for example, Slack slash command handler for slash commands, or message action handlers for responses to the button actions in Slack. Lambda functions process the event, push the event to the Amazon Simple Notification Service (SNS) topic, and reply to Slack. Then our business logic gets the message from the SNS topic and does something with it.

If just one of these pieces fails, our business logic will not be able to function correctly. Also, when you have many small services in your app, any integration can change any moment, either on the next deployment or when the external dependency change. Tests will not prevent these changes, but they'll at least ensure that your changes are not accidental.

But how do you know what should you test in a serverless app? That's a big topic, and you can read a bit more about it in my previous article about testing serverless applications.

The testing pyramid is a good start. Does it still apply? Yes, it does. Is it the same? Not really, because automated tests are cheaper than they were before. You can now create a new DynamoDB table in seconds, and then delete it after running your tests. Alternatively, you can leave it there, because you will pay only for the real usage (unless you store some vast amount of data during your tests). It's the same with other parts of your app. You can create an exact copy of your production app in minutes, and it will probably cost you less than a few cents to run a full end-to-end test suite.

However, faster and cheaper tests are not the only differences. Integration tests are cheaper, but also more critical because a common serverless app is split into many small pieces.

What makes a serverless app testable

Every application is testable, but some applications are written in a way that makes automated testing incredibly hard and expensive. That's precisely what you want to avoid in your application because lack of automated tests can make your switching process way more complicated.

That's where your app architecture jumps in and saves the day. You shouldn't reinvent the wheel; many excellent app architectures are around for years or decades. Which one is the right one for your serverless app?

Any architecture that will let you test your app easily and keep your switching costs low will be perfect. Because sooner or later you'll need to migrate pieces of your app. Not to another cloud vendor, but a new service, or some new or changed integration.

Like any other application, your serverless app has certain risks you'll need to consider. As my friend and co-author Aleksandar Simovic explained in our book Serverless Applications with Node.js, there are the following four risks that you should consider when architecting your app:

Configuration risks; for example, is DynamoDB table correct and do you have access rights?
Technical workflow risks; for example, are you parsing and using the incoming request properly?
Business logic risks; or is your app logic working the way it should?
Integration risks; for example, are you storing the data to your DynamodB table correctly?

You can test most of these risks with your end-to-end tests. But, imagine if testing a new car was done that way, and that you need to assemble the whole car to test if windshield wipers are working correctly.

Ports, adapters and architecture

If you are often traveling, you know the pain with power plugs. If you go from Europe to North America, you can't just plug your laptop in the power supply socket. They are incompatible.

However, buying a new cable whenever you travel to another country would be too expensive and pointless. Fortunately, you can buy a small adapter to make your power cable compatible with power sockets all around the world.

Your application should work the same way. Does your business logic care if it stores the data to MongoDB or DynamoDB? Not really. However, your database adapter should care about that.

This leads us to my favorite architecture for serverless apps: hexagonal architecture, alternatively called ports and adapters. As it's creator, Alistair Cockburn, explains, the hexagonal architecture allows an application to equally be driven by users, programs, automated test or batch scripts, and to be developed and tested in isolation from its eventual run-time devices and databases.

By definition, this architecture seems like a perfect fit for serverless, as it's almost impossible to simulate your entire eventual runtime locally during the development.

Let's say you have a service that receives some notification, parse it, save the data to the database, and then sends another notification to SNS topic. For us, this is a typical pattern. For example, when we save a vacation request to the database, we also send a message to the SNS topic that then triggers another service to send a request to the manager via Slack.

Instead of bundling everything together, the business logic of this service is simple, and it just coordinates other services. The core of the service exposes three ports:

A port for an incoming event
A port for saving data to the database
A port for sending a notification

Then we have different adapters, for example, one for sending the notification to the Amazon SNS topic for production, and another that fits the same port for sending a local notification during testing.

As explained in my other article about hexagonal architecture, our minimal code example is split into the following two files:

The lambda.js file wires the dependencies and has no tests.
The main.js file holds the business logic.

Remember our Slack flow? Let's see this in practice for one of the functions, for example, Slack slash command handler.

The lambda.js file from this function takes the SNS notification repository as a dependency and invokes the main.js function with it, and the received event. We have unit and integration tests for the function from the main.js, but it's never tested against Amazon SNS. Why? Because we send messages to multiple SNS topics from many different services. If we test each of them against Amazon SNS, it will take much time, and most of our tests would be redundant, as we'll repeatedly check if our SNS repository and it's a dependency, AWS SDK, are working as they should.

Instead, we test our main.js function against a local notification adapter that talks fit the same notification port. However, in the SNS notification repository's integration tests, we test the integration with Amazon SNS to make sure it works as intended.

But, how do we write ports and adapters in our code? It's simple!

Our main.js function receives an instance of notification repository as a parameter. That instance can be any notification repository compatible to the notification port, not just Amazon SNS adapter.

Also, what is the notification port? It's just a .send method of that notification repository. Our main.js file will try to send a message by invoking the following function: notification.send(message). Anything that can fulfill this request is a compatible adapter.

Our SNS notification is a class that exposes send method that sends a message. That's the adapter.

The lambda.js file looks similar to the following code snippet:

// Dependencies
const {
  parseApiEvent,
  SnsRepository
} = require('../common')
const httpResponse = require('@vacationtracker/api-gateway-http-response')
// Business logic
const main = require('./main')

async function handler(event) {
  // Parse API event
  const { body, headers } = parseApiEvent(event)
  // Create an instance of SNS notification repository
  const notification = new SnsRepository(
    process.env.topic
  )
  // Invoke the main function with all dependencies
  await main(body, headers, notification)
  // Return status an empty 204 response
  return httpResponse()
}

The most important part of our lambda.js file is the following line:

await main(body, headers, notification)

With this approach, it's easy to write automated tests for our business logic. In unit tests, we invoke our main function with some static values for the body and headers and mock the notification adapter. Then we check if the mock is invoked with the correct data.

In integration tests, we invoke the main function with some a static body and headers, and the instance of local notification repository. Local notification repository can be a simple wrapper around native JavaScript events.

If you want to read more about testing serverless apps and see code examples, checkout our new course on Senzo Homeschool platform (first one starts on Monday, June 22, 2020):

What about Vendor lock-in?

Yes, what about it? Ah, yes, we were talking about vendor lock-in! So you can ask how does hexagonal architecture help fighting a vendor lock-in.

Choosing a wrong database seems like significant vendor lock-in, right? We migrated most of our app from MongoDB to DynamoDB easily.

Our app is integrated with databases the same way it is integrated with Amazon SNS: using the database repository. At some point, our app was using MongoDB repository, that had unit and integration tests.

Once we decided to migrate to DynamoDB, we created another adapter for DynamoDB and called it dynamodb-repository.js. This repository has the same interface as MongoDB one, for example, if you want to delete a vacation you need to invoke the following function: db.deleteVacation(params). MongoDB repository will delete the vacation in MongoDB, and DynamoDB repository will delete it in DynamoDB.

During the migration, we connected our services to two repositories at the same time and started switching integrations, one by one. When the migration was finished, we removed the MongoDB integration from the service.

Beyond testing

There are always some things that can't be tested. For example, you integrated with Google, and they changed their API without proper notice. We saw Slack changing their app behavior many times, and it wasn't even documented.

Some of these integration changes are hard to detect, for example, when Slack decide to show only 5 attachments in the mobile layout our calendar is screwed, but our app still works properly. However, most of these changes start causing many errors in your app.

You can't fight against unexpected changes of third-party dependencies, they'll happen, but you can and should monitor both front end and back end of your app, and react fast when the changes break parts of your app.

If your serverless app is on AWS, there is a variety of excellent services that helps you monitor it. You can use built-in tools such as Amazon CloudWatch and AWS X-Ray, or some of the third-party apps, such as IOpipe, Epsagon, Thundra, Lumigo, and many others.

Single command deployment for single page apps

Slobodan Stojanović — Mon, 02 Apr 2018 06:28:32 +0000

Single command deployment for single page apps

Developing a single page app is hard. From the very beginning, you’ll need to make many decisions — decisions like picking a framework, setting the folder structure, configuring linter, and many others.
Some of those tasks are easier because of the ecosystem of the tools surrounding your favorite framework and web development in general. For example, tools like Create React App, Angular CLI and Create Choo App will help you to setup your favorite framework in a few seconds.

Often, you don’t have enough time to even think about the deployment when you start your new project. And at some point, you need your app to be publicly accessible because you want to show it to your client, friends, or to add it to your portfolio while you are looking for your first job.

But how can you pick the best place to deploy the app fast? There are many tools for deployment, too. If you go with some new shiny thing, will it scale for production, or will you be forced to make another decision about changing it soon? You can go with Github pages, but what about the HTTPS you need for service workers?

Amazon offers something that can scale, a combination of Simple Storage Service (S3) for static website hosting and CloudFront as a CDN is a cheap but scalable way to deliver your single page app. Although it takes some time to prepare both of those too, even more if you are not familiar with Amazon Web Services.

There is an easier way, though — introducing Scotty.js, a simple CLI tool that helps you deploy your website or single page app to Amazon S3 and CloudFront with a single command.

Beam me up, Scotty

The main idea behind Scotty is to deploy your static website or single page app to Amazon ecosystem with a single command.

It will deploy your static website, set up CDN with HTTPS, and even copy the website URL to your clipboard in a minute or so, depending on your internet speed and the website/app size.

For single page applications, it will also configure redirections, so pushState can work out of the box.

Let’s see it in action with a simple React application.

Create React App

Before the deployment, we need the app, so let’s create a simple one using Create React App.

First, create a sample app by running create-react-app command from your terminal:

create-react-app scotty-cra-example

If you do not have the create-react-app command installed, you can get it from NPM here: https://www.npmjs.com/package/create-react-app.

Or if you are using NPM v5, you can run Create React App command without installing it globally with the new npx command:

npx create-react-app -- scotty-cra-example

Learn more about npx here: https://medium.com/@maybekatz/introducing-npx-an-npm-package-runner-55f7d4bd282b.

Let’s add React Router to demonstrate how pushState support works. To do so, enter your new project and install React Router as a dependency:

cd scotty-cra-example

npm install react-router-dom --save

Now that everything is installed, let’s add React Router to the project — open “src/App.js” file in your favorite editor and update it to look like a basic example of React Router (https://reacttraining.com/react-router/web/example/basic):

import React from 'react'
import {
  BrowserRouter as Router,
  Route,
  Link
} from 'react-router-dom'
import logo from './logo.svg'
import './App.css'

const BasicExample = () => (
  <div className="App">
    <div className="App-header">
      <img src={logo} className="App-logo" alt="logo" />
      <h2>Welcome to React</h2>
    </div>
    <p className="App-intro">
      <Router>
        <div>
          <ul>
            <li><Link to="/">Home</Link></li>
            <li><Link to="/about">About</Link></li>
            <li><Link to="/topics">Topics</Link></li>
          </ul>

          <hr/>

          <Route exact path="/" component={Home}/>
          <Route path="/about" component={About}/>
          <Route path="/topics" component={Topics}/>
        </div>
      </Router>
    </p>
  </div>
)

const Home = () => (
  <div>
    <h2>Home</h2>
  </div>
)

const About = () => (
  <div>
    <h2>About</h2>
  </div>
)

const Topics = ({ match }) => (
  <div>
    <h2>Topics</h2>
    <ul>
      <li>
        <Link to={`${match.url}/rendering`}>
          Rendering with React
        </Link>
      </li>
      <li>
        <Link to={`${match.url}/components`}>
          Components
        </Link>
      </li>
      <li>
        <Link to={`${match.url}/props-v-state`}>
          Props v. State
        </Link>
      </li>
    </ul>

    <Route path={`${match.url}/:topicId`} component={Topic}/>
    <Route exact path={match.url} render={() => (
      <h3>Please select a topic.</h3>
    )}/>
  </div>
)

const Topic = ({ match }) => (
  <div>
    <h3>{match.params.topicId}</h3>
  </div>
)

export default BasicExample

Now, if you start your app using npm start it should work and look similar to the one from this screenshot:

It’s time to build your app using npm run build node script. This will create a folder called “build” in root of your project.

Deploy the app

First install Scotty.js from NPM as a global package by running:

npm install scottyjs -g

Prerequisites for Scotty are:

Node.js (v4+) with NPM
AWS account
AWS credentials — see setup tutorial here

Then just run following command from your terminal (make sure you navigate to project folder first):

scotty --spa --source ./build

This command tells Scotty that your app is single page app (SPA) and that the source of your project is in “build” folder.

Bucket names are global for all users, which means that you need to come up with a unique name for your app — reusing “scotty-cra-example” will not work.

Running this command from your terminal will deploy the app and give you 2 URLs as shown here:

First one, which is also added to your clipboard, is an HTTP link to AWS S3. The second one is a CloudFront URL that also supports HTTPS.

CDN and HTTPS

Scotty will set up your project on CloudFront CDN, which means it will be cached and distributed to different regions to decrease latency.

It will also set up HTTPS for free, so your app will be ready to use with service workers or anything else that requires a secure connection.

Live app: https://d1reyqfbyftmjg.cloudfront.net

How does it work

There’s no magic behind Scotty. It uses AWS SDK for Node.js behind the scene.
First, it checks if you already have a default region. Unfortunately, AWS doesn’t give us a default region via AWS SDK. Scotty has a small LevelDB database to store that info. If the region doesn’t exist and is not provided, Scotty will ask you to select it.

Next step is to create a bucket if bucket name is not provided, Scotty will use the name of your current folder. Keep in mind that bucket names are global for all users, hence, you need to come up with a unique name for your bucket.

After bucket is created, Scotty will upload your project to AWS S3 using AWS SDK. If a source flag is not provided, the current folder will be used as a source.

As the last step, if your project is a website or a single page app, Scotty will set up CloudFront CDN with HTTPS support. The difference between SPA and website is that Scotty redirects all of the non-existing pages back to index.html, which allows pushState to work out-of-the-box.

What are the next steps?

Try Scotty and let me know if something can be improved. Happy to receive pull requests as new features and improvements are welcome.

Github repository: https://github.com/stojanovic/scottyjs

The current idea for Scotty is to stay a small library for AWS only and offer an easy way to deploy frontend apps and sites in a serverless manner.

However, there are a few missing things, such as setting up custom domain names and config file for easier collaboration.

Hope you’ll enjoy it 👽

If you want to learn more about serverless architecture on AWS, check out my new book published by Manning Publications: Serverless Apps with Node and Claudia.js.