DEV Community: Sergey Bolshchikov

Open Deep Research Internals: A Step-by-Step Architecture Guide

Sergey Bolshchikov — Sun, 02 Nov 2025 09:49:38 +0000

This blog post is a different kind. It's a deep dive explanation of how Open Deep Research works under the hood and which design patterns are applied to make it one of the best open source deep research agents.

How is this post different from other resources?

The LangChain GitHub repo and blog posts provide only high-level explanations of how things work.
LangSmith and LangGraph Studio don't expose all the details—it's difficult to capture the state at each step and the dynamic invocation of the graph. Therefore, it's hard to understand the full picture at each step of execution.
A solid grasp of reflection agents, tool-use design patterns, and basic recursion is required to fully understand the process.

This is a long post by design, so brace yourself. We'll start by aligning on the high-level design of Open Deep Research. We'll then cover several design patterns used in the implementation that are crucial for understanding. Finally, we'll take a step-by-step deep dive through an example to see how the Open Deep Research graph and state evolve at each step.

How It's Designed

I assume you've read the official LangChain blog post that explains how Open Deep Research is built. However, to ensure we're all on the same page, let's have a quick look at the architecture.

The agent conceptually consists of three main parts: scoping, research, and final report. Any design variation you might want to build will likely contain these three components.

The goal of the scoping part is to build the input for the research phase. In Open Deep Research, this consists of a user clarification loop—the LLM determines whether it requires any clarification from the user or not—and then brief generation.

If you design your own agent, this is where you can perform user prompt optimizations or apply other techniques to improve the quality of user input.

When the brief is generated, it proceeds to the research phase. This is where the heavy work happens. We'll go into much more detail later, but in a nutshell, it consists of two stages: supervisor and research sub-agents. The supervisor, given the brief and using reflection, spawns multiple research sub-agents on demand, each with a dedicated sub-task. Each sub-agent (sub-graph) receives a dedicated topic, performs research on it, and returns a summary to the supervisor. When the supervisor reflects on the results and decides that it has gathered enough data, everything moves to the Reporter part.

The Reporter takes all the collected information and generates the final result. If your result is too large, this is where you can generate an artifact (like in Claude) instead of text results.

Prior Knowledge

To understand how Open Deep Research really works, we need to discuss several patterns in isolation that are used throughout. It's not the classic ReACT pattern that's now standard.

Reflection Pattern

The reflection pattern enables agents to evaluate their own output and use that feedback to refine their responses iteratively. In this pattern, an LLM generates an initial response, then acts as its own critic to assess the quality of that output. Based on this self-critique, the agent produces an improved version, repeating this cycle until it meets quality standards or reaches a stopping condition. This self-correction loop allows agents to avoid getting stuck in purely reactive thinking patterns and move toward more deliberate, methodical problem-solving.

Tool Use Pattern

The Tool Use pattern is fundamental to understanding Open Deep Research's architecture. While it might seem similar to standard tool calling, there are crucial differences that enable more sophisticated agent behaviors.

Standard Tool Calling (ReACT)

In classical ReACT architecture, tool calling is straightforward:

@tool
def search_web(query: str) -> str:
    """Search the web for information."""
    return tavily_search(query)

# LangGraph automatically:
# 1. Detects the tool call in LLM response
# 2. Executes the function
# 3. Adds result to message history
# 4. Continues to next LLM call

This works great for simple tools, but has limitations when dealing with complex operations.

The Manual Tool Use Pattern

Open Deep Research uses a different approach—manual tool orchestration. Here's why and how:

Why is this needed?

Complex operations: When a "tool" is actually spawning an entire subgraph (like a research sub-agent), you need more control
Memory management: Tool results can be massive (e.g., full research reports). Adding everything to message history bloats the context window
Custom routing logic: You might need to handle tool execution differently based on business logic
Parallel execution: Spawning multiple sub-agents simultaneously requires manual coordination

How it works:

Define tool schemas without implementations:

class ConductResearch(BaseModel):
    """Tool definition for spawning a research sub-agent."""
    topic: str = Field(description="The research topic to investigate")

# Note: This is just a schema - no actual function implementation

Bind schemas to the LLM:

llm_with_tools = llm.bind_tools([ConductResearch, ThinkTool, ResearchComplete])

LLM returns structured tool calls: When the LLM decides to "use" a tool, it returns:

AIMessage(
    content="I'll research this topic now",
    tool_calls=[
        {
            "name": "ConductResearch",
            "args": {"topic": "machine learning frameworks"},
            "id": "call_abc123"
        }
    ]
)

You manually handle the tool execution:

# Check if LLM wants to call a tool
if message.tool_calls:
    for tool_call in message.tool_calls:
        if tool_call["name"] == "ConductResearch":
            # Spawn a research sub-agent (a whole subgraph!)
            result = await research_subgraph.ainvoke({
                "research_topic": tool_call["args"]["topic"]
            })

            # Return a compact confirmation, not the full result
            return ToolMessage(
                content=f"Research completed on {tool_call['args']['topic']}",
                tool_call_id=tool_call["id"]
            )

Key Benefits in Open Deep Research:

Subgraph invocation: The supervisor can spawn entire research sub-agents as "tools"
Context efficiency: Instead of adding a 10,000-token research report to the message history, you return a simple "Research completed" message
Flexible routing: You can route different tool calls to different subgraphs or handlers
Parallel coordination: Spawn multiple research sub-agents simultaneously, each with isolated context

The Trade-off:
You lose automatic tool execution but gain fine-grained control over when, how, and what gets executed. This control is essential for sophisticated multi-agent architectures like Open Deep Research, where "tools" are actually entire reasoning subgraphs with their own state management.

Deep Step-by-Step Guide

Now that we have all the necessary knowledge and a high-level understanding of how Open Deep Research is designed, it's time for a deep dive.

The best way to describe it is to walk through an example step-by-step and see how the overall graph and state of the deep research agent change at each step.

Each step contains a corresponding image showing how the graph looks and the LangGraph state object (in green) at that step.

Step 1: User Question

It all starts with the user question. The LLM decides, using structured output, whether it requires any clarification from the user. If so, it returns the corresponding boolean value and a follow-up question that is sent to the user. The state at this stage is simple: an array of messages.

Step 2: User Responds to a Clarifying Question

When a user replies, Open Deep Research performs another call to the LLM with the array of messages. It might ask for clarification again or return structured output with:

{
  need_verification: false,
  verification: "Thank you…"
}

where need_verification indicates whether we should proceed to the brief writer or not.

response = await clarification_model.ainvoke([HumanMessage(content=prompt_content)])
if response.need_clarification:
    # End with clarifying question for user
    return Command(
        goto=END,
        update={"messages": [AIMessage(content=response.question)]}
    )
else:
    # Proceed to research with verification message
    return Command(
        goto="write_research_brief",
        update={"messages": [AIMessage(content=response.verification)]}
    )

Step 3: Generate the Brief and Pass to Supervisor

Now we make another call to the LLM with a different prompt to generate the research brief. The result we get back is stored in the state.

We also prepare the initial state for the supervisor subgraph. It's stored in supervisor_messages inside the state and starts with the system prompt and the brief that was just generated.

Step 4: Supervisor Reflects on the Brief

Now the complex part begins. The supervisor has its own system prompt and three tool definitions: think_tool, conduct_research, and research_complete. It's important to point out the pattern we described earlier—these are just definitions without actual implementations. The supervisor node listens to LLM calls and performs tool calls itself.

It starts with a call to the think_tool (reflection pattern) to understand what it should do, and the result is stored in supervisor_messages. The supervisor also checks the number of calls (research_iterations) it has made and stops if it exceeds the predefined maximum; otherwise, the research could continue indefinitely.

Step 5: Supervisor Initiates the Research

When the first reflection is recorded, the supervisor makes a call to the LLM, which returns a call to the conduct_research tool with the topic. It might return several conduct_research tools; in that case, the supervisor will spawn multiple sub-agents in parallel, each with a dedicated topic.

Step 6: Initiate Research Sub-Agent

The research sub-agent is a subgraph that is invoked dynamically based on the LLM response from the conduct_research tool; therefore, it's not visible in LangGraph Studio. It consists of three main nodes: researcher, research_tools, and compress_research. Research tools can be configured but generally include think_tool with the same logic as the supervisor tool, search, MCP servers, and the research_complete tool. As a subgraph, it has its own state with messages and research_topic that it received from the supervisor. Similar to the supervisor, each tool call is tracked in the state and capped by the maximum number of iterations.

Step 7: Research Agent Initiates the Search

This step is straightforward. The research node makes an LLM call with the system prompt and the research topic it received from the supervisor. It receives back a tool call to web_search with an array of queries that it should perform.

Step 8: Research Agent Performs Multiple Searches

Open Deep Research uses Tavily search and spawns several searches in parallel, one for each query it received from the researcher node. Given that search results can be large, before the search tool returns the results, they are summarized and then stored in researchers_messages in the state.

Step 9: Reflect on Research Results

Similar to the supervisor, the research node invokes the think_tool to reflect on received search results and make a decision about whether the results are sufficient or whether to continue.

Step 10: Research Completed

If the result of the think_tool indicates that the research has sufficient information, the research node calls the research_complete tool and goes to the compress_research node. Since the result of the search can be large, we compress the results at this point and return them back to the supervisor.

Step 11: Re-evaluate Research Plan

When results from research sub-agents are received, it's like finishing a tool call. We store the results in the state and specify to the LLM that conduct_research has finished. The supervisor, as per its system prompt, calls the think_tool to understand what to do next.

Step 12: Research Completed

At this stage, reflection indicates that the supervisor has gathered sufficient information for the research the user requested, and the research_complete tool is called to finish executing this subgraph.

Step 13: Generate Final Report

When all subgraphs are complete, the current state contains the message history, supervisor_messages, the brief, and the research results. The last step is to make a call to the LLM to generate the final report and return it to the user.

References

How to Make OpenAI API to Return JSON

Sergey Bolshchikov — Wed, 15 Nov 2023 18:02:06 +0000

During OpenAI's dev day, one of the major announcements was the ability to receive a JSON from the chat completion API. However, there aren't a few clear examples of how to do this as most examples focus on function calls.

Our objective is straightforward: given a query, we want to receive an answer in JSON format.

How can we achieve this?

There are three crucial steps.

Modify your prompt

Your prompt must explicitly specify that the response should be in JSON format and you need to define the structure of the JSON object.

Given this work history, what's the overall years of experience?
Work history includes start and end date (or present), title, and company name.
If the end date is "Present", use the current date. Today is November 2023.
Return the answer in JSON format with the field "experienceInMonths"
and value as a number.

Pay attention to the last sentence of the prompt.

Pass response_format

When calling the API, specify the response_format.

const res = await this.openAI.chat.completions.create({
  model: 'gpt-3.5-turbo-1106',
  temperature: 0.0,
  top_p: 1,
  frequency_penalty: 0,
  presence_penalty: 0,
  response_format: {
    type: 'json_object', // specify the format
  },
  messages: [
    { role: 'system', content: systemPrompt },
    { role: 'user', content: workHistory },
  ],
});

It's crucial to modify the prompt as well. Just changing the response type to JSON might result in a JSON of an arbitrary structure.

See this comment from the OpenAI API:

**Important:** when using JSON mode, you **must** also instruct the model to
produce JSON yourself via a system or user message. Without this, the model may
generate an unending stream of whitespace until the generation reaches the token
limit, resulting in increased latency and the appearance of a "stuck" request.
Also note that the message content may be partially cut off if
`finish_reason="length"`, which indicates the generation exceeded `max_tokens`
or the conversation exceeded the max context length.

Parse JSON response

Once we receive the response, the content is still text (string type), but we can now parse it as JSON.

const content: string = get(res, 'choices[0].message.content');
try {
  return JSON.parse(content)['experienceInMonths'];
} catch (error) {
  this.logger.warn('Calculating total experience for a member did not work');
  return -1;
}

It's good practice to wrap JSON.parse in a try...catch statement in case we receive an invalid JSON structure.

You can find a playground example here.

🧹🧹 Sanitizing user input with OpenAI under $1

Sergey Bolshchikov — Sat, 04 Nov 2023 08:03:38 +0000

The objective of this task is to extract a person's name correctly as it appears on LinkedIn. For example, if the input is John, Smith, the desired output should be John,Smith. Here's a slightly more complex example: if the input is 🌯 John,Smith, the output should be John,Smith.

The Simplistic Solution

The most straightforward solution in this scenario is to utilize a library that effectively removes unwanted characters from the input. The npm package known as String-sanitizer is adept at performing this task.

const { sanitize } = require("string-sanitizer");

const names = [
  " John,Smith ",
  "🌯John,Smith",
  "John,Smith ✔️",
  "John,Smith  🇺🇦"
];

names
  .map(name => name.split(',')) // split because sanitize removes ','
  .map(([firstName, lastName]) => ([sanitize(firstName), sanitize(lastName)]))
  .map(parts => parts.join(','));

// ["John,Smith", "John,Smith", "John,Smith", "John,Smith"]

Initially, this solution may appear effective until one encounters less predictable instances of names.

Dealing with Unpredictable Input

It is important to note that LinkedIn users often get creative with their naming conventions. The following examples illustrate how this variation can disrupt the efficiency of the code.

const { sanitize } = require("string-sanitizer");

const names = [
  "John (Johnny),Smith",
  "Joghn,\"Smith, CPA\"",
  "John,Smith-Perry",
  "John,Smith Jr.",
  "John,\"Smith, Ph.D\"",
  "John,Smith ✰ I'm Hiring ✰",
];

names
  .map(name => name.split(','))
  .map(([firstName, lastName]) => ([sanitize(firstName), sanitize(lastName)]))
  .map(parts => parts.join(','))

// ["JohnJohnny,Smith", "Joghn,Smith", "John,SmithPerry", "John,SmithJr", "John,Smith", "John,SmithImHiring"]

One might attempt to address each of these instances by crafting complex regex. However, this approach presents two major challenges:

It's plausible that there will always be a use case where the code will return an incorrect result.
The maintenance cost of the code, which includes testing and improving, is quite high. For each new use case encountered, the function must be altered to accommodate it.

Probabilistic Approach

For this use case, probabilistic models can yield significantly superior results than any possible code written by developers.

An example of this is using the OpenAI API with a simple prompt to return the person’s name. This method excels in more complex use cases, such as "I am hiring". However, it may overlook various suffixes.

OpenAI provides a mechanism to refine results according to specific needs via fine-tuning the model.

Fine-tuning the model has three key advantages:

The model is trained based on specific data, which yields more precise results.
It is cost-effective due to the use of a pre-trained model, necessitating a smaller system prompt.
The process is expedited.

Although it may seem excessive to employ machine learning for a seemingly simple task, this task is not as simple as it appears. Solutions like OpenAI offer fine-tuning capabilities that are easy to implement, providing superior results in less time than traditional coding approaches.

Fine-Tuning the Model

Here's how we can prepare a fine-tuned model in three steps:

Prepare training and validation datasets.
Train the model.
Implement the pre-trained model in the code.

Training a model, in simplest terms, involves supplying the OpenAI model with a file containing examples that include user input and the correct answer that the model should return.

OpenAI recommends providing 10-100 such examples.

The training file in .jsonl format might look like this:

{"messages": [{"role": "system", "content": "A given phrase contains a name. Your task is to extract it."}, {"role": "user", "content": "Smith 🇮🇱"}, {"role": "assistant", "content": "Smith"}]}
{"messages": [{"role": "system", "content": "A given phrase contains a name. Your task is to extract it."}, {"role": "user", "content": "John גיון סמיט"}, {"role": "assistant", "content": "John"}]}
{"messages": [{"role": "system", "content": "A given phrase contains a name. Your task is to extract it."}, {"role": "user", "content": "John-Perry"}, {"role": "assistant", "content": "John-Perry"}]}
{"messages": [{"role": "system", "content": "A given phrase contains a name. Your task is to extract it."}, {"role": "user", "content": "John Smith"}, {"role": "assistant", "content": "John Smith"}]}

Each line is a separate example. It contains a system prompt (what you want the model to do), user input, and assistant output (the correct answer that the model should provide).

Upon preparing this file, it can be uploaded to OpenAI's fine-tuning UI to pre-train the model. If your data preparation process is more complex, the OpenAI SDK can also be used for fine-tuning models.

The training duration will depend on the number of examples provided. Once complete, the model can be integrated into your code.

import { get } from 'lodash';

async sanitizeName(someName: string) {
    const res = await openai.chat.completions.create({
      model: 'ft:gpt-3.5-turbo-0613:personal::some-weird-code', // fine-tuned model
      temperature: 0.0,
      top_p: 1,
      frequency_penalty: 0,
      presence_penalty: 0,
      messages: [
        { role: 'system', content: systemPrompt },
        { role: 'user', content: someName },
      ],
    });

    const answer: string = get(res, 'choices[0].message.content');
    return answer;
  }

Evaluating the Cost

Finally, let us consider the cost. If you choose to use OpenAI, there may be costs involved, but this should not be a deterrent.

Instead, it would be beneficial to assess the efficiency perspective. This approach considers how much time you have expended to achieve the most optimal results.

The Simplistic Approach

The efficiency of the simplistic approach is rather low. For instance, you might spend approximately four hours writing and testing a function that covers all known cases. The challenge with this approach is the inability to predict all possible scenarios, resulting in a high likelihood of errors.

Fine-Tuning Approach

The cost of the fine-tuning approach consists of three components:

Your time to prepare training data + Cost to train the model + Cost to use it.

Although preparing the training data is the most time-consuming part, it would likely take less than half the time spent writing code using the simplistic approach.

Fine-tuning the model with OpenAI is a cost-effective solution. For instance, it took about 15 minutes to train a model with 151 examples at a cost of $0.13.

The final component is the cost of usage, which is also not substantial.

However, the fundamental question is whether the benefits outweigh the costs. Can you truly obtain better results for unpredictable input?

Consider this: fine-tune approach works not only with known scenarios but also with names that include mixed languages or are entirely in different languages.

Photo by PAN XIAOZHEN on Unsplash

How to maximize your product wins

Sergey Bolshchikov — Sat, 28 Oct 2023 18:20:01 +0000

Throughout my professional career, I've built many products for different companies. Each of them had a unique way of doing it.

I started to cultivate the feeling of what's THE right way to build the product and deliver value to customers consistently and sustainably. Those two criteria are important and have a significant impact on how to build a product.

I couldn't verbalize it myself but Itamar Gilad did and he did it very well - Stop Obsessing Over Development Velocity, Focus on This Instead.

😱 Sane front-end project architecture based on 12 years of experience

Sergey Bolshchikov — Wed, 27 Sep 2023 17:33:38 +0000

With 12 years of professional experience in front-end development, I have used 8 different frameworks in production. By any standards, that's a lot. While at the beginning of my career it was fun and exciting to learn something new and discover different flavors and concepts from various frameworks, now I am seeking a bulletproof solution that simply works.

The ideal architecture has a low cognitive load, is easy to maintain, scalable with the application's growth, and is customizable to accommodate the imagination of any UX designer. Unfortunately, such I could’t find a framework that meets all those needs, and, no, I am not on a mission to build one.

However, we do have many tools at our disposal. Can't we simply wire them all together? The short answer is no.

If you choose to wire different libraries yourself, various problems arise, such as overlapping functionality and different assumptions in their usage.It is frustrating to piece together different libraries in the React ecosystem.

Fortunately, I believe we have found a sane architecture that we are happy with.

Before we dive into the details, let's establish our requirements:

Clear Separation of Concerns Without Side Effects: As developers, we crave clarity, a map that shows us exactly where to lay down our code.
Scalability: Our architectural design should gracefully expand as our application grows, whether it's composed of 2 or 142 components.
Testability: Each layer should be testable in isolation, allowing for easy swaps if needed.
Simplicity: Our architecture should possess a low cognitive load, drawing from familiar concepts. This simplicity should make learning, maintenance, and modifications a breeze.

Implementation Blueprint

1. Project build foundation

The popular create-react-app is deprecated. Other recommended tools like Next.js or Remix have many assumptions and conventions that are imposed on developers. While following the recommended flow with these tools can make the development process a pure joy, it requires a strong commitment that we don't want to make from the beginning.

Instead, we have chosen Vite with the React TypeScript template. Vite is a fast, non-opinionated build library that provides the necessary structure for our application without imposing any strict conventions. It is also much easier to configure compared to Webpack, and can be easily extended if necessary.

2. Routing

Since we don't follow the conventions, we will handle the routing ourselves. In the React ecosystem, there is only one library for that - React Router. Specifically, we will use react-router-dom v6.

However, we will not be using loader functions and actions as recommended. They impose more limitations on client-side applications - you cannot use react hooks inside the loader function, they control the flow of fetch and render with function calls and the component, among other things.

Using these functions makes more sense if you are using Remix, the full-stack framework written by the same authors as React Router. It is very powerful in that context.

3. Authentication

Auth0 is a popular user management tool with a great developer experience library for React. It simplifies login configuration, user management, and more.

Auth0 also offers a powerful binding of their JavaScript library for React with the use of hooks.

To start, we need to configure the Auth0 provider, which is a straightforward process.

When we want certain routes to be accessible only for logged-in users, Auth0 provides a useful higher-order component. We can wrap it in a Protected component and use it later in the router configuration as follows:

const router = createBrowserRouter([
  {
    path: '/',
    element: <Root />,
    children: [
      {
        index: true,
        element: <Main />,
      },
      {
        path: 'protected',
        children: [
          {
            index: true,
            element: <Protected component={Internal} />,
          },
        ],
      },
    ],
  },
]);

Protected component is very simple too:

import { withAuthenticationRequired } from '@auth0/auth0-react';

export const Protected = ({
  component,
  ...args
}: {
  component: React.ComponentType;
}) => {
  const Component = withAuthenticationRequired(component, args);
  return <Component />;
};

When navigating to a specific route, auth0 checks if the user is logged in before rendering the component. If the user is not logged in, auth0 performs a full page redirect to the auth0 configuration page.

Auth0 is also used to handle the last concern, which is obtaining an access token for the logged in user. We can use this access token in the Authorization header when making API calls to our server.

Once the user is logged in, we extract the access token and store it for future use.

4. State management

When your application grows, you will eventually need a way to manage fetched data from a server and the application state.

There are many tools available for this purpose.

We have chosen Zustand, a small and simple library based on Flux concepts. I won't go into much detail comparing it to other libraries, as there is a dedicated page for that here.

I would like to specify the factors that were important to us in making this choice:

Small API surface: Zustand essentially has only two concepts - store and actions. This means there is not much boilerplate code that developers have to write, such as wiring actions or reducers.
Simplicity: It provides a hook and actions that mutate the initial state. That's it. There are no extra magical features like in Mobx.
The store is a React hook and can also be accessed as a plain JavaScript object if needed.
Easy scalability: As your application grows, you can split the store into separate slices with dedicated actions.

5. API calls

The final layer of our client-side architecture is the API. This is where we make calls to our server. The API layer interacts exclusively with the store (a vanilla object) and its actions. It is important to avoid any direct calls from components to maintain a clear separation between boundaries.

The API layer should also be stateless and focus on providing minimal logical data manipulation. Data management should be handled by the data management layer.

For making API calls, we create an Axios instance with an interceptor that takes an access token and adds it to every request sent to the server.

import axios from 'axios';
import { useStore } from '../store';

const axiosInstance = axios.create({
  baseURL: `${import.meta.env.VITE_SERVER_URL}/api`,
});

axiosInstance.interceptors.request.use(
  async (config) => {
    const token = useStore.getState().me.token;
    config.headers['Authorization'] = `Bearer ${token}`;
    return config;
  },
  (error) => {
    return Promise.reject(error);
  },
);

export default axiosInstance;

Conclusion

So, what have we achieved? We have created a layered client-side architecture for a scalable project with clear boundaries. Each layer is separately testable.

To get started, you can create a new GitHub repository using the sane-front-end-project-template. It already includes all the mentioned components wired together.

Why Software Engineers Aren't Going Anywhere, Even with AI

Sergey Bolshchikov — Wed, 23 Aug 2023 12:45:24 +0000

There's been a growing buzz about AI taking over software development in the next decade or so. But let's dig deeper and understand why this doesn't mean we're on the verge of being replaced. In this post, we'll explore the reasons why software engineers aren't going anywhere, even in the AI age.

AI's Influence and the Unchanged Essence of Software Engineering

The AI wave is indeed changing the game with tools like GitHub's Co-pilot. These tools boost our efficiency and save a ton of time and money. And there's cool stuff on the horizon, like self-healing code, that'll make our software better and quicker to produce. It's an exciting time, no doubt.

My Recent Experience: A Peek into Our World

I recently took on a project from scratch to finish, and it got me thinking. I compiled a list of technologies that we, as software engineers, need to be familiar with to create a simple app. Here's a snippet:

Creating the look (HTML)
Adding style (CSS)
Making it work (JavaScript)
Handling browsers (Browser stuff)
Navigating within the app (Routing)
Managing data flow (State management)
Designing the interface (UI libraries)
Linking with other apps and services (APIs)
Speeding things up (Caching)
Picking a backend language (Python, JavaScript, etc.)
Building the backend (Server framework)
Juggling tasks (Async programming)
Ensuring security (Security basics)
User access (User authentication)
Storing data (Database)
Team coordination (Version control)
Packaging the app (Docker)
Getting it out there (Deployment)
Checking it works (Testing)
Speeding up delivery for end users (CDN)
Handling addresses (DNS and networking)
Utilizing the cloud (Cloud architecture)
Domain management
Sending emails (Email service setup, reputation and warm-up)
Respecting privacy (Privacy concerns)
Monitoring performance (Monitoring)

It's a hefty list, right? Sure, AI tools like ChatGPT can help with some. Today's tools let us do more with a smaller team than before.

The Real Value We Bring

Here's the scoop: being a software engineer isn't just about knowing tools. It's about fitting these tools together seamlessly. It's like cooking up a dish with various ingredients. As we improve at this, we innovate fresh ways to solve issues.

A Bright Future Ahead

The idea that AI will replace us isn't the full picture. In the next 10 to 15 years, we'll still be here, doing our thing. Instead of fading away, we'll focus more on tackling tough challenges in smarter ways. As technology advances, so will our skills to make great things happen.

In a nutshell, the AI storm won't wipe us out. While AI tools are handy, the blend of skills, creativity, and problem-solving we possess is hard to replicate. So, brace yourselves – the future of software engineering is a thrilling ride!

As a software engineer, will I have a job in several years?

Sergey Bolshchikov — Wed, 07 Jun 2023 14:00:00 +0000

During a recent OpenAI roadshow at Tel Aviv University, a software engineering graduate student posed an important question to Sam Altman: "As a software engineer, will I have a job in several years?" It reflects the growing concerns about the impact of artificial intelligence (AI) on the job market. In this blog post, we will explore the evolving landscape of software engineering and shed light on how the role of software engineers may transform in the face of advancing AI technologies.

Code Implementation vs. Essence

To understand the potential influence of AI on software engineering, it's essential to distinguish between code implementation and its essence. While code implementation refers to the specific technical details and tools used to build software, the essence encompasses the broader concepts and architectural decisions behind it.

For example, message queue is an essence to support async communication between different services. Kafka including the usage of its API is the implementation detail. Due to different technical reasons, it’s possible to substitute Kafka with RabbitMQ. The implementation will change but the essence of message queue has remained.

AI's Growing Role in Code Implementation

AI has already started making its mark on code implementation. Tools like GitHub's Co-Pilot have demonstrated the ability to generate code snippets and assist developers in writing code more efficiently. As AI becomes increasingly proficient at writing code, developers may find themselves relying more on these AI-powered tools to handle implementation details.

Shifting Focus to the Essence

As AI takes over some aspects of code implementation, software engineers can redirect their focus towards the essence of software development. This entails delving into software architecture, designing bigger building blocks, and making critical decisions regarding the overall structure of the software. Instead of spending significant time on writing lines of code, developers will have the opportunity to engage in more strategic and creative thinking.

The Continued Need for Computer Science Knowledge

While AI may automate certain coding tasks, the fundamental knowledge of computer science remains crucial for software engineers. Understanding concepts such as complexity, DB schemas and their design, and security risks will continue to be essential. However, developers may no longer need to implement low-level algorithms from scratch, as AI can assist in generating optimized solutions. It is worth noting that many developers already rely on existing resources like Wikipedia and Stack Overflow for code snippets.

The Impact of AI on Software Engineering

The impact of AI on software engineering is expected to follow the path of abstraction. Much like the progression from Assembly language to high-level programming languages like Python, AI's role will elevate abstraction in the development process. Software engineers will still be in demand, but their responsibilities will shift towards higher-level tasks that leverage AI-generated code. This transition will enable developers to focus on designing sophisticated systems, ensuring robust software architecture, and effectively utilizing AI tools to enhance productivity.

Conclusion

As we contemplate the future of software engineering in the era of AI, one thing is clear: software engineers will continue to play a vital role in the industry. While AI may take over certain aspects of code implementation, the essence of software development and the need for strategic decision-making will remain essential. Software engineers must adapt to these changes by embracing AI-powered tools and leveraging their expertise to contribute to higher-level tasks. The future of software engineering holds exciting possibilities, and it's up to us to navigate this evolving landscape with enthusiasm and adaptability.

How software architecture changes in the era of AI

Sergey Bolshchikov — Mon, 05 Jun 2023 14:20:00 +0000

Introduction

In the world of software development, architecture plays a crucial role in shaping the overall structure and design of a system. Traditionally, architecture discussions have followed an outside-in approach, where architects and decision-makers envision the future architecture and communicate their ideas to the development team. However, this standard approach often falls short of delivering the desired results. In this blog post, we explore the limitations of the outside-in approach and propose alternative strategies to improve the architecture design process.

The Pitfalls of the Outside-In Approach

Starting from the blank canvas every time:
In reality, software development rarely involves starting from scratch. Instead, we build upon existing architectures with the goal of enhancing and improving them. When architects and decision-makers attempt to invent the future architecture without sufficient familiarity with the current codebase, their assumptions may be inaccurate or flawed. This lack of context can lead to suboptimal design decisions that hinder the development process.
Communication Gaps:
Effective communication between decision-makers and engineers is vital for successful architecture implementation. However, when architects pass down architecture requirements through documentation or verbal means, information can be lost or misinterpreted along the way. This breakdown in communication can result in a misalignment between the envisioned architecture and its actual implementation.
Lack of Feedback Loop:
One significant drawback of the outside-in approach is the absence of a proper feedback loop. Architects, team leads, and other stakeholders often lack a non-intrusive method to validate and review the implemented architecture. As a result, mistakes and issues may only be identified late in the development cycle, leading to costly and time-consuming rework.

Redefining the Architecture Design Process

Generating High Level Architecture from Code: To overcome the limitations of the outside-in approach, one alternative is to generate the architecture directly from the codebase. The code itself represents the ultimate source of truth, providing insights into the existing structure, dependencies, and patterns. By analyzing the codebase automatically, engineers can gain a deeper understanding of the architecture and make informed decisions for further enhancements. Some tools are getting close to doing so. For example, architecture-view-nestjs generates 2 levels of architecture for NestJs applications.

Leveraging AI for Architecture Generation:
Then, we can enhance the deduced architecture by leveraging artificial intelligence (AI). With the wealth of existing architectural patterns and best practices, training AI models like GPT on this knowledge can enable them to generate new and innovative architectural solutions. By tapping into AI's capabilities, architects can benefit from an augmented design process, expanding the scope of possibilities and ensuring comprehensive exploration of potential architectures.
Real-Time Architecture Updates:
To facilitate better collaboration and feedback, it is essential to establish a near real-time architecture view. This view would enable architects, team leads, and stakeholders to monitor the progress of architecture implementation throughout the development process. By having a clear and up-to-date understanding of the evolving architecture, early detection of implementation issues becomes possible, allowing for timely intervention and course correction.

Conclusion

The standard outside-in approach to architecture discussions often falls short due to its detachment from the existing codebase, communication gaps, and the lack of a feedback loop. By embracing alternative strategies, such as deducing architecture from code, leveraging AI for generating architectural solutions, and establishing real-time architecture updates, we can revolutionize the architecture design process. These new approaches encourage a more holistic understanding of the system, enable better collaboration between decision-makers and implementers, and empower teams to detect and rectify issues early on. Ultimately, by reimagining how we approach architecture discussions, we can unlock the potential for more robust, efficient, and successful software systems.

Why measuring experience in years is not good enough

Sergey Bolshchikov — Mon, 01 May 2023 14:14:55 +0000

Experience plays a tremendous role in any professional career and at the end of the day, every employer is looking for the best, usually the most experienced, people. A common way to measure experience is in years. However, I would argue that such an approach might be misleading.

Here I want to offer an alternative approach to evaluating experience and provide some practical questions that you can use to aid your understanding of what experience truly is.

Part 1. The definition.

Let’s start by looking at the definition. According to Merriam-Webster, the experience can be defined as:

a: direct observation of or participation in events as a basis of knowledge

b: the fact or state of having been affected by or gained knowledge through direct observation or participation

Simply, experience is gained from direct observation and/or participation. Therefore, the more we encounter different types of problems and their solutions, the more our experience grows.

Part 2. Real-life.

But what happens in real life? Most of the actions in our daily jobs are so abstract that we just repeat them over and over again. They are usually simplistic tasks, for example, workers at an assembly line, each one doing one part of an overall product but not able to act on the bigger picture. By dividing work into repetitive tasks, the overall business is efficient, but it may hinder attempts to gain broader experience.

What is true for those on industrial assembly lines is true for many software engineers. We are surrounded by infrastructure, libraries, and frameworks that assist us in carrying out complex tasks. This allows us to produce code quickly, and all we are left to do is technical design and its implementation.

Solving new problems and implementing innovative features can be extremely rewarding and stimulate our intellect. Encountering and dealing with the same time of problems can also create a depth of understanding and expertise. However, there is a threshold, that when crossed, means that repeated tasks become mind-numbing and fail to stimulate our creativity. At these times, we need to switch to new challenges. The problem is that many of us are stuck at that repetitive point and count this as additional experience. So while experience measured in time may be growing, experience measured in intellectual growth may have stalled.

Part 3. The valid alternative.

True experience comes from solving different types of problems in a variety of ways.

For example in a set of two candidates, from one perspective you could say that one who has worked for five years has more experience than the other who has worked for three years. However, if we assessed them more qualitatively, we might find that the candidate who has worked for three years has been exposed in that time to a wider diversity of problems.

This is the reason why software engineers who join startups in the early stages are more likely to gain significantly more familiarity with diverse areas while perhaps compromising on knowledge depth as a result.

Part 4. The practical way

I believe it’s fair to say that every position has a finite set of types of problems that one can encounter. For example, one way, but certainly not the only one, to categorize the problems of software engineering would be in the following way, in order of complexity:

Investigating and solving a bug
Implementing a feature within given specifications
Designing and implementing a feature according to a product specification
Architecting the solution across different boundaries (e.g. front-end, back-end, devops)

Thus, if we were evaluating experience in terms of diversity of experience rather than simply quantity of time, we might wish to understand the level of complexity a potential employee has been exposed to. This can be achieved by probing candidates more about the nature of their work experience. From my own experience, I have found that the majority of full-stack developers have experience working with the first and second categories above, while the best candidates have experience in three or more of the categories.

I also make sure to ask candidates a more subjective question, asking them to tell me what the most challenging problem they had to face in their career was or the problem-solving process that they are the most proud of. The answers to this question are a good indicator of the edge of a candidate’s experience and in ideal situations, their answers would fall in the third of the fourth category of problem-solving.

Part 5. Conclusion

If you are looking to optimize your professional growth, then seeking out work environments with fast-growth rates can help give you the space to gain a wider diversity of experience. Such places, for example, a startup that grows from hundreds to thousands of users within a short timeframe, is likely to experience many new and complex constraints and problems to solve. Thus, within these environments, you’ll be able to gain not just quantity of experience, but a true depth and range of experience too, which will make you a valuable asset for any company.

4 strategic choices that may boost your career in software engineering

Sergey Bolshchikov — Sat, 29 Apr 2023 19:33:13 +0000

Disclaimer: it's a very opinionated post!

Let's say you have 5 years of experience as a software engineer and you come to an interview for a senior position. To your surprise, you don't get the job because you lack experience as you were told. How come?

Honestly, professional experience has very little to do with the number of years you've been in business. Instead, it's the variety of experience you gain on the way.

I've interviewed over 100 people within just 2 years, and it's easy to spot candidates who planned their careers early on.

So what do successful candidates do differently?

1. Choose the right company over a title

Let's face it, landing your first job is hard. No pressure, but it also plays a significant role in your future career.

Usually, at that point, the main choice is between a small startup (maybe with a fancy title) or a big established company. I don't think that choosing your company by size is right.

Here are 3 signs the company is a good fit:

They are willing to invest in your professional growth;
They are a team of smart people who you will interact with daily;
They hold you accountable for your actions

Let's dig a little deeper into each of the above.

Investment in your professional growth

In a nutshell, it means the company has practices in place that contribute to the professional growth of employees. These practices don't have to be formal, such as courses or lectures. Simple things work well too: regular code review, involving junior engineers into design and architecture reviews. All these indicate a good engineering culture that aims to grow engineers and not only ship features or fix bugs.

Great people

A good team leader builds a team of engineers of different levels. A balanced team has a spectrum of engineers, e.g. 1 junior, 2 mid, and 2 senior engineers for a team of five.

Why should you care about it?

With experienced colleagues you get to read clean code and understand the logic behind these lines. It's like learning to cook by looking at our meal - you can guess some parts but it doesn't mean you can cook it.

Daily interactions with senior colleagues will teach you to analyze different approaches, reason about them, and make the right technical decisions. It's a skill that usually takes years to develop. Having experienced colleagues around you can significantly shorten this learning time.

Holding you accountable

We act differently when we know that our ass is on the line. On the one hand, such situations put you under pressure and intuitively we want to avoid them.

On the other hand, such situations allow tremendous growth because they force you to operate outside of your comfort zone. Good companies know that and nurture the culture of accountability (usually also backed up with the safety nets).

2. Become an expert in your domain

Software engineering has many specialties: front-end, back-end, DevOps, databases, and many others. Many companies are hiring a full-stack though - what should you do?

Young engineers face this dilemma quite often - should I know a little front-end, back-end, and one type of database, or focus on just one area, say, front-end?

As you might guess from the title, I believe it's better to be an expert in at least one area than being a surface learner. Every product has a period of weird behavior before the stack is mature enough. For example, NodeJS server fails, say, once every 3 days, your database gets random timeouts on connections, or your modal dialog keeps getting stuck. Fixing inconsistent behavior requires a solid understanding of underlying technology and experience to have a hunch where to look for.

When such moments come, experts are in the limelight and save the day. This is why established companies are interested in hiring more experts rather than generalists.

3. Ship a product

That feeling when you see customers using the product that you've built is extraordinary. It's also a great responsibility and requires you to be more than just a coder.

To feel this, you need to master the disciplines you’re not necessarily an expert in. These often include understanding product value, task prioritization, engineering, UX, QA, customer support, and feedback, you name it.

"Real artists ship"
-Steve Jobs

Having a proven record of a shipped product demonstrates your ability to get shit done and serves as a strong indicator of your abilities to the hiring manager.

4. Don't do more of the same

Your true experience comes from the types of problems you solve. You can have 7 years of experience, but it’s not so relevant if you solve the same problem over and over again, e.g. building the same type of application or the same type of website (landing pages). You’ll get exceptional at these, but it will be hard for you to change the job. Most likely your next position will be almost the same but in a different company. Why? Because you have 7 years of experience solving the same type of problem and you are damn good at that.

As an alternative, you can have just 3 years of experience, but these 3 years are a rollercoaster - you get to build an application, set testing infrastructure, improve monitoring, then build a different type of application, and so on. Combining your domain expertise with a handful of problems you solve on the way gives you the true experience.

Being an expert and solving different types of problems makes you experienced

You might notice, I didn't mention the number of years of experience at all. You may meet a person with 2-3 or even 4 years of experience and be amazed by their achievements. Most likely, these 2-3-4 years have been very rich with all sorts of engineering problems, solving which this person grew to be this outstanding professional you want to be.

Stick for a while but leave early if it’s not a good fit.

It all takes time. Understanding the culture, learning architecture, building your reputation - it doesn't happen overnight.

Depending on the size of the company, this time might vary. As a rule of thumb, for the mid-size company, it takes about 3-6 months to onboard an employee and up to a year for a new employee to bring some meaningful results.

Sometimes, a person switches jobs every year or two. If this is you - it’s okay, but be ready to give a good reason why this happened.

Sometimes, it just doesn't feel right. Try to understand what's wrong and try to work it out. Try to be objective and see the full picture. There's a very good chance things will improve.

Sometimes, things don’t work out. And that's okay too. Accept it and move on. No point in staying to collect "the years of experience".

Still have any doubts about career choice? Reach out to me. Seriously, I mean it. We can schedule a call and discuss it further.

Building a Successful Junior Software Engineering Career: A Path to Success

Sergey Bolshchikov — Thu, 27 Apr 2023 11:28:00 +0000

Throughout my career, I have conducted more than 700 professional interviews for various software engineering positions.

Successful candidates surprisingly share the same trait: the beginning of their careers was incredibly similar. I believe that when you are just starting, there should not be too much freedom, since tackling the right problems will define your path.

Stage 1: Learning to write production-grade code

Your main goal at the beginning of software engineer career is to learn how to transform your “writing code” abilities to “writing code for real users (production)”. This requires a lot more than just committing a code to a version control system. The list might vary between different companies, but in a nutshell, you should learn the following:

Code readability - how to write code for others to read
Teamwork - how to work in a team of other developers and QA
Code reviews
Pull requests
Development tasks/tickets/issues
Tests - how to write testable code and learn different types of tests
Deployments - how your code is deployed to production
Monitoring - how to monitor your code in production and make sure it works as intended
Coping with production problems (urgents) - how to handle problems in production that happen to real users

Countless books have been written on each of the topics above. While we won’t dive into them here, you must invest some time into learning more about all of them. This is the foundation to become a successful software engineer.

Stage 2: Learn how to build simple architecture

When you feel comfortable delivering code to production, we can go up a level: learning to design software architecture solutions. The best way to accomplish this is to start small and ask your team leader to give you less defined tasks. This creates an opportunity for you to brainstorm the potential solution, design it, discuss it with senior peers, receive feedback, and finally, to implement it. The emphasis at this stage is on suggesting solutions and receiving feedback.

Stage 3: Learn to design subsystems or their parts

This stage is a logical continuation of the previous one: we are moving from features to designing important parts of the system. This requires a deep understanding of the whole picture (system) and how the part in question, the one you are working on, fits into it.
The course of action is similar to the previous stage, in that you will provide solutions and receive feedback. However, now the scope of the project has grown. You should consider the overall architecture of the part, its API communication with other parts of the system, using industry standards solutions, different design patterns, etc.

Stage 4: Build service from zero to production

There’s no substitute for the experience of building a service from the ground up and delivering it to production. This is a rare opportunity, especially in well-established engineering companies. Thus, whenever you have the chance, be sure to jump on it!
What should you expect here? Ouch…. quite a lot.
• You should learn to evaluate different approaches to solve the problem you have, taking into account their advantages, disadvantages, and cost of implementation.
• You should also experience how to deploy a new service to production.
• If, in all previous cases, you could remain in one domain, e.g., front-end or back-end, at this stage, you should be able to cross them and start writing code in both of them, including DevOps for the deployment part.
• You have a chance to define and/or adopt coding standards for your project.
• Last but not least, you’ll need to understand how to monitor the behavior of the service in production and the load, as well as what metrics and KPIs to look at.

Stage 5: Build a product

Whether you are in a startup or your well-established company is discovering a new possibility, you should participate in building a product. You may be wondering, “How is it different from stage 1?”
We are always contributing code to some product. At this stage, you must shift your concentration from writing a good code to building a good product.
To achieve that, what do you pay attention to here?
You work with the product team is mainly where you will direct your energy. Why are you working on this or that feature? What customer problem does it solve? Have you considered all feasible edge cases? How will a certain feature perform under load?
Those are just a few among the many questions engineers should ask product people and themselves, too.
This is the step where engineering and business value finally interact.

Stage 6: From here to infinity

The easy part is now over. This journey usually takes around 3-5 years. This is when the ‘hard’ part begins. Nobody can tell you what your goals should be but you. You will have to decide for yourself whether you want to try the management path or go deeper into engineering craftsmanship.
Whatever you decide, the stage that starts here has no end.

A shortcut to become a go-to engineer in your team

Sergey Bolshchikov — Mon, 24 Apr 2023 14:19:08 +0000

There are very few people on the team who know, with the necessary depth, how things work. They are usually a “go-to” for the other team members and are often involved in strategic architectural decisions of the team. They are also usually the most seasoned team members and witnessed a lot of code being written. Does it mean that you have to wait for several years to get to that place? The answer is not necessarily if you invest in your knowledge early on.

Why understand software architecture as soon as you can?

Communication: Software architecture communicates the software system's design and structure to stakeholders, ensuring better collaboration and reducing the chances of misunderstandings.
Quality and Performance: Understanding software architecture is essential for ensuring the quality and performance of software systems by identifying potential issues and bottlenecks.
Planning and Design: Understanding software architecture helps to plan and design software systems, ensuring they are scalable and adaptable.

We don’t see it the same way

Having different levels of experience, knowledge, and seniority can lead to problems in software architecture as it can cause misunderstandings and communication issues between team members. For instance, those with less experience may not be able to understand the technical details provided by experienced team members, leading to confusion and misinterpretation of the software design. Similarly, senior team members may not be able to effectively communicate the impact of architectural decisions on the organization's goals to junior team members, leading to a lack of alignment and direction.

Visualize product architecture in real time

Having a tool that visualizes product architecture in almost real time, where everyone sees the same picture, can produce a significant impact on software quality and improve communication

Let's take the popular open-source project ToolJet as an example.

It's immediately obvious how complex the Tooljet project is. Analyzing its code and understanding its architecture can take several hours. However, you can use a VSCode extension that visualizes everything at once with the necessary level of abstraction. Seeing the high-level picture allows you to spot potential problems well before they become significant.

Credit: Visualized by Architecture View NestJS

What does it improve?

When entering a new or legacy code base, visualizing the product architecture can help you understand the system's structure quickly, making it easier to navigate and locate relevant sections of code.
When onboarding new employees, visualizing the product architecture can help them get up to speed with the system's design and structure, ensuring they are better equipped to contribute to the project.
Keeping up with changes in the system can be challenging, but visualizing the product architecture can help you identify potential impacts of any changes, ensuring you can make informed decisions and minimize the risk of introducing new issues.

Visualizing the architecture and seeing how code changes impact overall architecture can easily allow you to become another “go-to” person for your team.