DEV Community: epilot

We Made Coding Agents Actually Reliable By Fixing One Thing

Viljami Kuosmanen — Wed, 04 Feb 2026 15:55:45 +0000

Last week, Vercel published research showing that giving coding agents a compact index of your documentation dramatically outperforms letting them search for answers on demand. Their eval results: 100% task success rate with the map approach, versus only 79% when agents had to actively look things up.

Same agent, same tasks, different approach to context. The difference between working and not working.

The insight clicked immediately. If we could give Claude Code reliable access to this institutional knowledge without forcing it to decide when to look things up, it would fundamentally change how people work in our codebase.

So we built it.

The Context Problem

Coding agents have a token limit. A ceiling on how much information they can process at once. Think of it like working memory. You can't hand Claude Code your entire codebase and documentation library upfront. It's too much.

The traditional solution is skills: the coding agent decides when it needs information and actively looks it up. "I need to know about authentication, let me search for that." Sounds reasonable. In practice it creates three problems:

Decision paralysis - the agent has to decide when to look up docs, and it often guesses wrong
Async delay - every lookup is a round-trip, breaks flow
Sequencing conflicts - exploring code vs. consulting docs creates timing issues

Vercel's approach flips this: give Claude Code a compressed index of what documentation exists and where to find it, before it starts work. The index is small enough to fit in context every turn, so it always knows what's available. When it needs details, it reads the specific file directly. No decisions, no lookups, no conflicts.

Why Compression Matters

The key innovation is compression. A full documentation tree - all the folder structures, file names, categories - takes significant space. Too much to include in every conversation turn.

The compressed index uses a simple pipe-delimited format that shrinks this by ~80%:

[epilot Docs Index]|root: ./.epilot-docs|00-general:{tech-stack.md,business-context.md,ci-cd.md}|01-apis:{api-design.md,calling-apis.md}|02-epilot360-microfrontends:{env-vars.md,local-dev.md}|...

That single line tells the agent:

Where the docs live (.epilot-docs/)
What categories exist (00-general, 01-apis, etc.)
What files are in each category

It's a table of contents, not the full book. But it's enough. The agent sees the map, understands what's available, and pulls specific files when needed. The decision-making load disappears.

Why This Matters

The accuracy improvement is significant (100% vs 79% task success), but it's not the real story.

The real story is what happens when you lower the barrier to contribution. Proper context enables the person closest to the problem to fix it, regardless of job title. Your PM can fix bugs. Designers can adjust component behavior. Support engineers can patch data issues. You remove bottlenecks. Context and authority live in the same person.

Coding agents are the equalizer. But they're only as good as the context you give them.

Most companies will throw Claude Code at their codebase and wonder why results are inconsistent. The agent hallucinates patterns. Makes incorrect assumptions. Writes code that doesn't match conventions.

The difference is context. Structured, compressed, always-available context about how your codebase works.

How to Build Your Own

The pattern is straightforward. Here's what we did at epilot:

Curate agent-friendly documentation - organize your internal knowledge: conventions, APIs, architectural patterns, framework usage, code style
Structure by domain - group related docs (general, backend, frontend, infrastructure, etc.)
Use descriptive filenames - the agent sees filenames in the compressed index before opening files. api-design.md is better than guidelines.md. error-handling.md is better than errors.md. Make filenames searchable and specific.
Automate updates - pull live data where possible (OpenAPI specs, schema definitions, framework docs)
Generate compressed index - use a simple format (pipe-delimited works well) that reduces the doc tree by ~80%
Embed in agent context - add the index to your CLAUDE.md or AGENTS.md file (the context files Claude Code reads)

One thing worth highlighting: we don't just include our internal documentation. We also package docs for the frameworks and libraries we heavily use - single-spa, openapi-backend, openapi-client-axios, i18next, and Volt UI (our custom design system). When Claude Code needs to know how i18next pluralization works or how to register a single-spa parcel, it already has the answer. No hallucination, no outdated Stack Overflow posts, just accurate framework documentation.

What This Enables

We're already seeing daily usage across the team. Developers context-switch between services faster. Non-engineers contribute directly instead of filing tickets.

But the real potential is broader: if compressed context improves coding agents for technical documentation, why stop there?

Runbooks and incident response - on-call engineers with instant access to procedures
Customer domain knowledge - support teams with context on product behavior
Business logic - product decisions and their rationale, preserved and accessible

The pattern is the same: curate the knowledge, compress it, embed it in context, let the agent work.

The Constraints Are Disappearing

For decades, contributing to a codebase required deep technical knowledge. You needed to understand the language, the frameworks, the architectural patterns, the implicit conventions. The barrier was high.

Coding agents lower it. Claude Code, Cursor, and similar tools don't replace engineers. They make technical knowledge more accessible. With the right tooling and the right context, a PM can fix bugs. A designer can adjust styling logic. A support engineer can patch data issues.

The question isn't whether this is possible. It's how fast you can adapt.

Organizations which enable broader contribution will move faster than those that don't. The tools exist. The research is clear. What's missing is execution.

Start Simple

You don't need to document everything upfront. Start with the knowledge that causes the most friction:

Code style conventions - how you write TypeScript, naming patterns, file structure
Common patterns - how you handle authentication, API calls, error handling
Framework specifics - non-obvious usage of your frameworks and libraries
Internal APIs - if you have OpenAPI specs, even better

Create a simple doc structure:

docs/
  00-general/
    code-style.md
    tech-stack.md
  01-apis/
    api-design.md
    calling-apis.md
  02-backend/
    error-handling.md
    database-patterns.md

Generate the compressed index (pipe-delimited format, one line per directory). Add it to your CLAUDE.md or AGENTS.md file - the context files that Claude Code and other coding agents read on startup. Done.

The compressed index approach works! 🎉 Vercel's research proved it: 100% task success versus 79% without it. We've validated it internally. Now it's about whether you'll adopt it before your competitors do.

Security at Scale: Our npm Incident Response Story

João Pinho — Wed, 10 Sep 2025 16:20:24 +0000

On September 8th, the npm ecosystem saw what is now called the largest supply-chain compromise in its history. Packages like chalk, debug, and ansi-styles — together downloaded billions of times every week — were hijacked and malicious versions published.

For any SaaS company with Node.js in its stack, this was a moment to pause and act.

At epilot, where we build cloud software for the energy market, we recognised that this had the potential to impact our systems. Here's how we responded.

🚨 First, what happened?

A phishing attack compromised the maintainer of several widely used packages.
Malicious versions were published and quickly spread via transitive dependencies.
The injected code was designed to hijack crypto wallet transactions, but the bigger story is: if attackers could publish once, they could publish anything. Additionally, if attackers could read/intercept our network requests, they would have access to our tokens and thus our platform on behalf of our users.

⚡ Our response — measured in hours

As soon as the incident was announced, our engineering team moved fast. Within minutes of posting the alert in our #dev-security slack channel, we had a small response team come together.
172 repositories scanned: we used GitHub Codespaces to access our centralized codebase and systematically audit our product ecosystem.
451 Node.js projects analyzed: every package.json across our domain groups (auth, billing, analytics, 360-portal, and more) was checked for the compromised packages and versions.
Automation helped: we cross-checked with our internal tooling and monitoring alerts to confirm no production build had pulled malicious versions.

Result: ✅ zero exposure found across all 451 dependency trees.

🛡 Practices that kept us safe

The npm incident highlighted the value of practices we already had in place:

Dependency monitoring with Corge → ongoing visibility into package versions and vulnerabilities across our entire ecosystem.
Masked logs in Datadog → all PII automatically obfuscated, ensuring sensitive customer data never leaks, even if dependencies misbehave.
LLM anonymisation → using Microsoft Presidio, we automatically detect and anonymise emails, phone numbers, IBANs, credit cards, and other PII before any data reaches AI models for our AI-powered features.

These aren't "nice-to-haves." They are part of how we build trust with customers in a regulated industry like energy.

💡 What we learned

Supply chain attacks are inevitable — being able to respond fast is what matters.
Modern tooling makes a difference — GitHub Codespaces let us audit 172 repositories and 451 Node.js projects in hours, not days.
Good hygiene compounds — monitoring dependencies, masking logs, and anonymizing LLM data aren't glamorous, but when incidents like this happen, they prove their worth.

🔭 Looking forward

Incidents like the npm compromise will continue. At epilot, we're committed not just to reacting fast, but to building resilient, privacy-first systems that earn customer trust every day.

Because in the energy market — where trust and compliance are everything — resilience is the real competitive advantage.

From Chaos to Clarity: Fixing Our Monorepo

kate astrid — Tue, 19 Aug 2025 11:45:37 +0000

Introduction: Why This Story Might Save You Some Headaches

If you’ve worked in a large codebase, you’ve probably seen this: the code runs, but every change takes far longer than it should. Updating one feature often breaks another. Adding a dependency means worrying about conflicts. Old build tools still linger, and no one feels safe upgrading packages that half the system depends on.

That’s exactly what we faced in our journey-monorepo, the main repository for our team. Dependencies were wired together in inconsistent ways, different teams had introduced multiple overlapping build tools, and some libraries hadn’t been touched in years. Proposing a cleanup felt like asking to redesign a city while people were still living in it — risky, disruptive, and easy to delay indefinitely.

So why did we bother?

Development had slowed to a crawl. Builds were taking too long, and waiting minutes just to see a change in the browser was killing productivity.
Every change felt risky. The dependency spaghetti meant a small tweak in one package could break multiple apps — often in ways we wouldn’t notice until much later.
Onboarding new developers was painful. It took too long for new team members to get up to speed, mostly because they had to learn multiple bundlers, testing setups, and outdated tooling quirks.
Technical debt was blocking new features. We were spending more time patching, debugging, and wrestling with the repo than building the actual features our users cared about.
Costs were adding up. Longer build times meant higher CI costs, and outdated packages made security updates and maintenance more expensive in the long run.

In short, the longer we waited, the more time, money, and momentum we were losing.

And, hey, now we get a good story out of it.

Journey-Monorepo in a Nutshell

Journey-monorepo is the central place on the epilot platform for everything related to journeys. What you see in the screenshot above represents two main apps: journey-app on the left side and journey-builder on the right. Journey-builder is responsible for configuring journeys, and journey-app for rendering them.
Like in many startups, the repo started out clean and well-structured, but as the product grew quickly, the codebase expanded in unplanned ways — things were added wherever they seemed to fit at the time.

The “BEFORE” Setup

Apps:
- journey-builder.               
- journey-app
- entity-mapping
- journey-view

Packages:
- blocks-configurators
- blocks-renderers
- journey-elements
- journey-utils
- journey-logics-common

Let me show you the rough map of dependencies we had step by step. We are interested in two main apps: journey-builder and journey-app, so I am going to dive deeper into their connections. It is not important which app or package does what. What IS important is to see how many connections we had to keep in mind every time we needed to change anything.

Ok, this is fine.
This is too.
Yeah, cool.
Packages already become tangled, let's keep going.
Ok, even more connections.
Well, this is just a delinquent madness.

You might say: Kate, come on, this is what happens in every big project - things become deeply intertwined, and it is a normal thing to experience.
I will answer: yes, but I haven't finished.

Let's now add the build tools to this table to have the full picture.

Well, what we have here:

Webpack;
Craco;
Tsup;
Tsx;
Tsdx.

I don't know about you, but I hadn't worked with half of them before I started working on this repo.

In the end, it was a web of “if you change this, you might break that” — and “that” could be anywhere.

How We Tackled It

We kicked things off with an RFC — basically, “let’s write this down so we don’t lose the plot halfway through.”
Our two big goals:

Unify the tooling so we weren’t juggling five different build setups.
Untangle dependencies so teams could work without stepping on each other’s toes.

And this is exactly what we did.

Step 1: Cut the Cord on Unnecessary Dependencies
The first thing we had to do was face the dependency jungle. Over time, different teams had introduced overlapping packages, and no one was quite sure what depended on what anymore. Cleaning this up gave us a chance to regain control.

Untangled all the apps and packages as much as possible. We mapped out how everything connected and then started cutting back the unnecessary links. This meant reducing cross-dependencies that caused fragile builds.
Moved blocks-configurators into journey-builder. These configurators were never meant to be used outside, so pulling them closer to where they belong simplified the dependency graph.
Moved blocks-renderers into journey-app. Renderers were core to the application itself, so consolidating them in the main app made the architecture more intuitive.
Deleted journey-utils and journey-elements. Journey-utils was a very small package with some utilities that were moved to other packages. And journey-elements _ were later replaced by _concorde-elements. You can read more about it in this article written by my partner in crime.

This was a painful step, but once the dust settled, the repo felt lighter and easier to reason about. We no longer had to worry about accidentally breaking other parts of the system when making small changes.

Step 2: Standardize the Tooling
After trimming the dependencies, the next big challenge was our tooling. Different parts of the repo used different build systems, which slowed everyone down and made onboarding new developers a headache.

Phased out Craco, Tsup, and Tsdx by switching to Vite. These older tools had been added over time, each solving a narrow need. Vite gave us a unified, modern setup with faster builds and simpler configuration.
Swapped Jest for Vitest — faster tests, modern setup. Jest had served us well, but it was slow and required custom patching to keep working. Vitest gave us near-instant feedback and worked seamlessly with Vite, making the dev experience far smoother.

By unifying the tooling, we not only sped up day-to-day work but also reduced cognitive load. Developers no longer had to remember which tool applied to which package — everything just worked the same way everywhere.

This is how the "AFTER" setup looks:

As you can see, the monorepo is now far more streamlined — there are significantly fewer dependencies crisscrossing between packages, and we’ve eliminated most of the variations in build tools. This makes the structure easier to understand, faster to work with, and much less fragile when changes are introduced.

After the Restructuring: What We Learned

When the dust settled, it wasn’t just that the dependency graph was cleaner — we were thinking about the repo differently.

Immediate Wins

Changing things stopped feeling dangerous. Before, touching a shared package felt like pulling a brick from a Jenga tower. Now, fewer connections mean less risk.
Builds and tests stopped feeling like a punishment. Vite’s dev server = instant feedback. Vitest shaved minutes off test runs. Devs started running tests more often just because they could.
Onboarding went from “ugh” to “okay.” New folks could get set up without learning the quirks of four bundlers first.

Slower Payoffs

- Tooling swaps aren’t magic. Some Webpack plugins didn’t have a Vite equivalent, so we had to rethink features or drop them.

- Vitest exposed bad tests. Some “passing” tests were only passing thanks to Jest’s quirks. Painful, but worth fixing.

Trade-Offs & Reality Checks

- Not everything got unified. A couple of apps still run on legacy setups because migrating them wasn’t worth the risk — for now.

- Shared packages need rules. We now ask “does this really need to be shared?” before creating new ones.

- Refactoring is never “done.” There is always room for improvement, we all know. that.

Tips If You’re Thinking About Doing This

Draw your dependency graph before you start — it’ll help convince others (and yourself) that the cleanup’s worth it.
Expect setbacks. Some PRs will break things. That’s normal.
Automate checks for unused or outdated packages — saves a ton of review time.
Celebrate small wins. Removing the last Webpack config deserved cake.

Final Thoughts

This wasn’t a quick tidy-up — it was a half-year-long deliberate overhaul alongside working on other features. But it’s made a massive difference to how we work: faster builds, fewer “don’t touch that” areas, and a general feeling that the repo is ours again, not a beast we’re afraid of.

If I had to wrap it all up in one bit of advice?

Keep it simple.

Every extra dependency, tool, or package you skip today is one less thing you’ll have to untangle in the future.

Building a Scalable React Component Library: Lessons From Concorde Elements

Adeola Adeyemo — Wed, 23 Jul 2025 14:02:23 +0000

Introduction

We recently embarked on the complete redesign of our Journeys, aiming for a modern, state-of-the-art look - Project Concorde. This wasn't just a UI overhaul; we sought the flexibility for future modifications, knowing that simply patching our existing UI, heavily reliant on Material UI, wouldn't suffice. We needed to build a new foundation from the ground up.

The full Project Concorde story is larger, but in this article, we'll dive into the story of @epilot/concorde-elements, the new component library born from that need, and how we built a system that not only powers our new interface, but also empowers our development team.

Why did we need a new component library?

Our existing component library, @epilot/journey-elements, was based on Material UI. While it served its purpose, our goal was to reduce our reliance on Material UI to gain several crucial benefits:

Reduce performance overhead from Material UI's style generation in favour of CSS modules
React version constraints tied to MUI releases.
Styling limitations blocking modern CSS features.
Remove the use of the Material UI theme object in saved custom Designs

The image below shows the overhead to create custom designs:

Why not use existing design systems?

We found that most off-the-shelf design systems are quite opinionated with their styling and theming. Our philosophy was that it's easier to replace a single unit than an entire factory. Our primary goal was to keep our new system simple and extensible.

The Challenge

The path forward was not without its hurdles:

We needed to build 37+ components with various potential variants.
All Journey blocks had to be migrated to the new design, and we anticipated new complexities during integration due to component usage.
We had to ensure that custom themes built with the previous design system worked seamlessly with the new one.

Phases of Developments

1: Pursuit of Purity

In the initial phase, our approach was to create every component from scratch. We desired control over every pixel and line of code. Our first significant task was migrating the Product Tile to enable a new "Recommended Product" feature. We began with the basics.

Our early Button component perfectly illustrates this "purity" approach—simple, direct, and completely self-contained.

/* An early Button.tsx */
import classes from './Button.module.scss';
import type { ButtonProps } from './types.ts';
// ...

export const Button = forwardRef<HTMLButtonElement, ButtonProps>((props, ref) => {
  // ...
  return (
    <button
      className={classNames('Concorde-Button', classes.root, ...)}
      {...props}
    >
      {/* ... */}
    </button>
  )
});

Next, we successfully built our Link, Card, TextField, StepperInput, Image, ImageStepper, MobileStepper, and the ThemeProvider. With these foundational components, we successfully released the "Recommended Product" feature, receiving excellent feedback.

Our next milestone involved migrating the Date field and adding a Time select, which led to the DatePicker component. The design for this component had unique custom requirements that made building it entirely from scratch impractical. After some research, we opted to extend an existing library, React DatePicker, with custom functionalities to suit our use cases.

The resulting DatePicker involved creating new sections like the Footer and Time Select, and replacing the TextField and Header. While functional, the process was cumbersome, and the result felt more like a series of patches than a cohesive part of our system. This experience was our wake-up call: our "from-scratch" purity, and even our one-off extension approach, was simply not scalable.

2: The Pivot - 'Headless UI & Purity' Hybrid

This realization prompted a strategic shift. Speed became as crucial as purity. This quest led us to a breakthrough: headless components. These were libraries that provided the complex logic, state management, and accessibility of common widgets, but shipped with absolutely no styles. This was our "aha!" moment.

After further research, we settled on two incredible libraries: Radix UI Primitives and MUI Base (which recently became Base UI). They offered us the best of both worlds. We could now build our components by composing these primitives and applying our own distinct design system on top.

Consider our Autocomplete component, for instance, it leverages a hook from MUI Base for its core logic, while we provide the entire UI.

// A simplified look at our Autocomplete component
import { useAutocomplete } from '@mui/base/AutocompleteUnstyled'
import { forwardRef } from 'react'

import { Input, Menu, MenuItem } from '..';
// ...

export const Autocomplete = forwardRef(function Autocomplete(props, ref) {
  // ...
  const {
    getRootProps,
    getInputProps,
    getListboxProps,
    getOptionProps,
    groupedOptions
    // ...
  } = useAutocomplete({ ...props })

  return (
    <div {...getRootProps(other)} ref={ref}>
      <Input {...getInputProps()} />
      {groupedOptions.length > 0 ? (
        <Menu {...getListboxProps()}>
          {(groupedOptions as any[]).map((option, index) => (
            <MenuItem {...getOptionProps({ option, index })}>{option.label}</li>
          ))}
        </Menu>
      ) : null}
    </div>
  )
})

We were no longer reinventing the wheel for every single component. This allowed us to focus our efforts on what truly made our library unique: our design and the developer experience. We only created components from scratch when absolutely necessary.

This hybrid model significantly accelerated the development of the remaining components.

Key Principles

Throughout the creation of our component library, we adhered to several core principles that ensured our team worked in unison. These principles were internally documented in our contribution guidelines and are outlined below.

A Consistent Component Structure

To keep our codebase organized and predictable, we established a standard folder structure for every component. For example,

/_ src/components/Button/
  ├── Button.tsx            // Logic
  ├── Button.module.scss    // Styles
  ├── Button.test.tsx       // Tests
  ├── types.ts              // Type definitions
  └── index.ts              // Exports

This simple convention meant that any developer could easily navigate to any component and immediately locate its logic, styles, and type definitions with reducing collaboration overhead.

// Button

...
<Component
   aria-disabled={isDisabled || variant === 'disabled'}
   className={classNames(
     'Concorde-Button',
     classes.root,
     variant && classes[`variant-${variant}`],
     variant === 'primary' && 'Concorde-Button__Primary',
     ...
     className
   )}
   style={customStyles}
   ...
 >
  ...
</Component>

// Card

<div
   className={
     classNames(
       'Concorde-Card', 
       classes.root, 
       className
     )
   }
   ref={ref}
   style={customStyles}
   {...rest}
/>

Notice the use of the Concorde prefix in the static HTML classes. This served as a foundational element for easy custom styling, a topic not covered in detail here.

Design Tokens & Theming

The next pillar was unifying our design system. We created a comprehensive set of global design tokens using CSS variables for every aspect of our UI: colors, spacing, typography, transitions, shape and more. This became the language of our design system.

By coding colors, typography and dimensions as CSS custom properties, we guaranteed every component would be visually consistent, utilizing the same set of values. We also enabled these values to be extended and customized externally as local variables, preventing hard-coded styles and ensuring maximum flexibility.

:root {
  --concorde-primary-color: #0070f3;
  --concorde-secondary-color: #ff7e1b;
  --concorde-font-family: 'Proxima-Nova', sans-serif;
  --concorde-spacing: 0.25rem;
}

Now, all our components can simply use color: var(--concorde-primary-color) or margin: var(--concorde-spacing). This setup dramatically simplified theming. For example, toggling the typography tokens automatically affects all text on the screen.

Beyond the tokens used internally, we also exposed custom tokens for each component. These external tokens provide powerful ways to extend a component's functionalities.

For example, the Button component has the following custom tokens:

const customColors: ButtonCSSProperties = {
    '--concorde-button-label-color': color,
    '--concorde-button-background-color': backgroundColor,
    '--concorde-button-hover-bg-color': hoverBgColor,
    '--concorde-button-active-bg-color': activeBgColor,
    '--concorde-button-gap': gap ? `${gap}px` : undefined
  }

As an example, the CSS styles below will specifically modify the Button and Card:

:root {
  /* Button styles */
  --concorde-button-label-color: #ffffff;
  --concorde-button-background-color: #ff7e1b;
  --concorde-button-gap: 0.5rem;

  /* Card styles */
  --concorde-card-background-color: #e34590;
}

Note the consistent use of the concorde prefix for the token naming for consistency and scoping.

All tokens (default and custom tokens) are thoroughly documented in Storybook and our developer documentation for clarity.

TypeScript: Our Shield and Guide

From day one, we committed to writing everything in TypeScript. More than just providing type safety, our type files became a form of documentation. We commented our types extensively, enabling any developer using a component to understand the purpose of each prop right from their IDE.

// An excerpt from our Autocomplete types.ts
export interface AutocompleteProps<
  // ...
>extends UseAutocompleteProps<T, Multiple, DisableClearable, FreeSolo> {
  /**
   * The label content.
   */
  label?: React.ReactNode
  /**
   * If `true`, the component is disabled.
   * @default false
   */
  disabled?: boolean
  // ... and so on
}

Composition: Building Blocks for Complex UI

A fundamental principle guiding our development was composition. Our base components were designed to be flexible building blocks. By combining them, we could construct more complex and specialized UIs without adding bloat to the core library. This allowed us to build sophisticated features by assembling simple, well-tested parts - a true "change one, change all" approach.

A simple Input could be composed to PatternInput, IbanInput, NumberInput and more complex components like a ProductTile could look be structured as follows:

// A conceptual ProductTile built with composition
import { Card, Image, Typography, Button } from '@epilot/concorde-elements'
import styles from './ProductTile.module.scss'

export const ProductTile = ({ product }) => {
  return (
    <Card className={styles.tile}>
      <Image src={product.imageUrl} alt={product.name} />
        <Typography as="h4">{product.name}</Typography>
        <Typography>{product.description}</Typography>
        <Button variant="primary" label="Add to Cart" />
    </Card>
  )
}

Developer Experience

Vite

Vite powered our local development with a lightning-fast dev server and HMR, greatly benefiting from our monorepo setup

Storybook: The Living Documentation

How could we ensure quality and consistency across our components? The answer was Storybook. It became far more than just a component gallery; it became the living, breathing heart of our project. For every component, we wrote stories that showcased all its variants and states.

// A story from Input.stories.tsx
import type { Meta, StoryObj } from '@storybook/react'
import { Input } from './Input'

const meta: Meta<typeof Input> = {
  title: 'Components/Input',
  component: Input
  // ...
}
export default meta

export const WithLabel: Story = {
  args: {
    label: 'Email Address',
    placeholder: 'Enter your email...'
  }
}

export const Disabled: Story = {
  args: {
    ...WithLabel.args,
    disabled: true
  }
}

Critically, every story also served as an accessibility test. We integrated the @storybook/addon-a11y addon, which runs automated accessibility checks on every story against WCAG standards. This proactive approach allowed us to catch issues with color contrast, ARIA attributes, and keyboard navigation right within our development environment.

Testing & Accessibility

To build components that last, they must be reliable. Alongside the real-time accessibility checks in Storybook, we built a robust testing foundation. We used Vitest as our test runner and React Testing Library to write unit and integration tests for every component. We also used vitest-axe and React Testing Library for more intricate accessibility checks.

These tests were not just about preventing regressions, they were about enforcing correctness. We tested component logic, ensuring that each part behaved exactly as expected under various conditions. This combination of automated accessibility checks and functional testing was crucial. It gave us the confidence to refactor, add features, and scale the library, knowing that our foundation of quality would hold strong.

These automated tests run in our CI to catch regressions early.

The Payoff: Scalability in Action

The true test of concorde-elements came as we started integrating it in our main application, specifically in the concorde-renderers package. The work of setting up tokens, using primitives, and embedding quality through testing and documentation paid off.

Developing new features for Project Concorde transformed from a chore into a delight. Need a Modal? Pull in the component. Need a complex form field? Compose it with our TextField, Autocomplete, and Button components. They all looked consistent and behaved predictably.

This became the foundation for more interesting features:

Custom CSS: custom styling for resulting Journeys
Consistent design for Custom Journey Apps using the published library

Conclusion

Building a component library is a journey. Ours taught us that a little upfront systematization goes a long way. The initial purity of building "from scratch" gave way to the pragmatic wisdom of building on top of a solid, accessible foundation.

This yielded a few key principles we now live by:

Start with design tokens: They are the bedrock of a consistent and scalable design system.
Embrace headless primitives: Don't reinvent the wheel for everything, but leverage existing, well-tested solutions for speed and robustness.
Make TypeScript non-negotiable: The safety and developer experience benefits are immeasurable.
Testing: A combination of unit, integration, and automated accessibility testing is crucial for a reliable library.
Build with composition in mind: Create simple, flexible blocks that can be assembled into complex UIs, promoting reusability and maintainability.
Document thoroughly: A living, tested, and accessible documentation hub aligns everyone and accelerates development.
Be adaptable: The perfect plan rarely survives contact with reality. Be ready to pivot and embrace better ideas.

The @epilot/concorde-elements library is more than just a collection of React components. It's a testament to our team's journey, a foundation for our product's future, and a system that helps us build better, faster, and more consistently. We hope our story can help guide you on your own path to building a scalable and resilient component library.

Resources

Cover Photo by Ryan Quintal on Unsplash

How We Integrate AI in epilot - Chapter 2: Serverless RAG w/ LangChain & Weaviate

Kerem Nalbant — Mon, 26 May 2025 08:38:42 +0000

Introduction

In the previous chapter, I shared how we began our AI journey at epilot by implementing AI Email Summaries, helping users reduce email reading time by up to 87%. Encouraged by that success, we're now stepping up our AI capabilities with Retrieval-Augmented Generation (RAG) to provide smarter, contextually aware email suggestions.

WHY?

As we aim to scale our commodity business, investing in AI is crucial—not just for growth, but to significantly upgrade our product’s capabilities. Commodity segments often have a high volume of repetitive customer service requests. Our users need quick, context-aware email suggestions that understand:

Previous communications and organizational knowledge
Company-specific communication styles
Tailored relationships with each customer

Although Large Language Models (LLMs) are powerful, they're limited when accessing recent or specific company data. Customizing LLM responses usually involves prompt engineering, RAG or fine-tuning. Fine-tuning is resource-intensive and complex, making prompt engineering with RAG our clear choice.

Our Solution: Retrieval-Augmented Generation (RAG)

We implemented a RAG-based solution to retrieve and provide relevant context from past email threads and eventually expand to external data sources like documents and websites. Long-term, organizations using epilot will have fully configurable, customized knowledge bases accessible to all future AI features and AI agents.

This allows our users to respond to customer emails faster, improving communication quality and efficiency. On the end customer side, it means quicker, more accurate, and better overall service.

See It in Action

An end customer emails about documentation requirements for a renovation plan (Sanierungsfahrplan).

The epilot user doesn’t waste time researching policies or manuals—they simply prompt our AI to generate reply in english.

Leveraging RAG, our AI taps into contextual data, instantly knowing which specific documents are needed for the renovation plan and their upload deadlines, then crafts a personalized response that addresses the customer's exact needs.

Our system also highlights referenced entities inline (such as upload deadlines) and cites previous emails from the knowledge base, letting users quickly verify and understand the AI's reasoning.

Solution Components

To build a secure, scalable RAG system in a serverless environment, we chose:

LangChain

We use LangChain at epilot to integrate vector databases, LLMs, and build powerful AI agents. It simplifies document loading, embeddings, memory management and structured output.

Weaviate

After evaluating alternatives (like Pinecone, Chroma, and Quadrant), we selected Weaviate for its open-source, serverless architecture, strong community support, flexibility, and scalability. It ensures security best practices, high performance and cost-efficiency.

Presidio

Security and data privacy are essential. Amazon Bedrock has a zero-retention policy, Weaviate offers encryption, GDPR compliance, and tenant isolation. But we needed an extra layer for handling sensitive PII data.
Presidio helps us redact this information before indexing, preventing AI hallucinations and protecting customer privacy.

LangSmith

LangSmith provides AI observability, performance monitoring, debugging, prompt management, and testing. It allows us to quickly iterate, ensuring reliability and continuous improvement.

How We Built It

Now, let's dive deeper—from a high-level overview into the detailed implementation of our RAG system:

RAG: Making LLMs Context-Aware

RAG (Retrieval-Augmented Generation) has emerged as the perfect solution. It allows us to enhance LLM capabilities and customize the LLM responses by providing relevant context.

We built two core pipelines: ingestion and retrieval.

Ingestion

Email messages are processed, cleaned, and converted into vector embeddings.

Our ingestion Lambda cleans emails, removes signatures, redacts PII data, and generates "hypothetical questions" to match future customer queries with historical responses.

With the hypothetical questions approach, we aim to create question-answer pairs by treating outbound emails as answers and inbound emails as questions. Then, while generating a suggested email, we extract the end customer's questions from the inbound email and search them in the hypothetical_questions vector field.

chain = (
    {"doc": lambda x: x.text}
    | ChatPromptTemplate.from_messages(
        [
            (
                "system",
                "You are a helpful assistant that generates hypothetical questions from an email.",
            ),
            (
                "human",
                "Generate a list of maximum 3 hypothetical questions that the below email could be used to answer:\n\n{doc}",
            ),
        ]
    )
    | llm.with_structured_output(HypotheticalQuestions)
    | (lambda x: x.questions)
)

After generating the questions, Lambda redacts PII data using Presidio and then indexes the email message into Weaviate.

While indexing, Lambda first generates the embeddings of email body text and hypothetical questions, then pass those vectors to Weaviate. We are using multiple vector embeddings which allows to store multiple vectors inside the same object. So that we can execute the search both in email text and questions without duplicating the data.

Retrieval

Similar emails and potential answers are retrieved from vector database.

A typical retrieve & generate flow looks as follows:

1. Extract questions

extract_query_prompt = ChatPromptTemplate.from_messages(
    [
        (
            "system",
            """You are a professional question extractor, an AI assistant that extracts the customer inquiries from email messages.
    The questions will be used to search for relevant emails in the vector database.
    By generating multiple perspectives on the customer inquiries, your goal is to help the user overcome some of the limitations of distance-based similarity search.
    Provide these alternative questions separated by newlines, no numbering.""",
        ),
        (
            "human",
            """Generate a list of maximum 3 questions from the following email.
    Email: {email}
    Questions:
    """,
        ),
    ]
)

extract_query_chain = extract_query_prompt | llm | LineListOutputParser()

extracted_questions = await extract_query_chain.ainvoke(
            input={"email": email.text}
        )

For the email shown in the demo video, the following questions are extracted by the question extractor chain:

{
  "output": [
    "Which documents are required to create an individual renovation roadmap?",
    "How can I submit additional documents for the renovation roadmap?",
    "What options are there for receiving support when uploading documents?"
  ]
}

Query vector database

We run multiple queries in parallel, and then combine unique retrieved documents. We mostly adopt hybrid search, by setting the alpha as close as possible to 1, we keep keyword search in the mix while leveraging semantical vector search.

email_message_retriever = MultiQueryRetriever.from_llm(
    retriever=email_messages_vector_store.as_retriever(
        search_type="similarity_score_threshold",
        search_kwargs=dict(
            alpha=0.90,
            tenant=data.orgId,
            score_threshold=0.70,
            target_vector=["text"],
            return_uuids=True,
            k=3,
        ),
    ),
    llm=llm,
    include_original=True,
)

question_retriever = MultiQueryRetriever.from_llm(
    retriever=email_messages_vector_store.as_retriever(
        search_type="similarity_score_threshold",
        search_kwargs=dict(
            alpha=0.90,
            tenant=data.orgId,
            score_threshold=0.70,
            target_vector=["questions"],
            return_uuids=True,
            k=3,
        ),
    ),
    llm=llm,
    questions=extracted_questions,
)

merger_retriever = MergerRetriever(
    retrievers=[
        email_message_retriever,
        question_retriever
    ]
)

retrieved_docs = await merger_retriever.ainvoke(message.text)

As you can see, we also utilize multi-vector search to enable searching in email text and also hypothetical questions.

retrieved_docs includes the email body and similarity score along with all the metadata we need, allowing us to leverage it while building the prompt.

For the same email and questions above, the retrieved context from database is as follows:

{
  "documents": [
    {
      "metadata": {
        "created_at": "2024-11-27T12:15:46.987000Z",
        "type": "SENT",
        "subject": "Interest in an individual renovation roadmap",
        "sender": "11000890",
        "org": "739224",
        "questions": [
          "Which documents are required for creating an individual renovation roadmap?",
          "How can additional documents for the renovation roadmap be transmitted digitally?",
          "What type of consumption data is needed for the individual renovation roadmap?"
        ],
        "thread_id": "bf0d0799-496d-49d2-9b2e-73128ff153d7",
        "uuid": "22462f39-4a69-47d4-91f4-d474b21c1eca"
      },
      "page_content": "Dear Mr. [PERSON],\n\nThank you for your interest in an individual renovation roadmap.\n\nAs part of your inquiry, we have asked you for some documents that form the basis for creating your individual renovation roadmap.\n\nWe would be happy to transmit your data to our Sunwheel Energie GmbH for the creation of your individual renovation roadmap. However, we need your support for this.\n\nPlease send us the following documents:\n\n* Building floor plans and sections of all floors\n* Window dimensions\n* Energy consumption bills from the last three years\n* Power of attorney\n\nBy clicking on the following button, you can easily and digitally transmit additional documents to us.\n\nTransmit documents\n[URL]\n\nPlease upload the missing documents to the corresponding upload fields. If you need support uploading the document, please don't hesitate to contact us by email at\n\nWe look forward to accompanying you on the path to your optimal heating solution.",
      "type": "Document"
    },
    {
      "metadata": {
        "created_at": "2024-07-15T05:57:23.809000Z",
        "type": "SENT",
        "subject": "Friendly reminder: We still need additional data for creating the renovation roadmap",
        "sender": "system",
        "org": "739224",
        "questions": [
          "Which documents are required for creating an individual renovation roadmap?",
          "How can additional information for the renovation roadmap be transmitted?",
          "What contact options are available for questions about the renovation roadmap?"
        ],
        "thread_id": "edb31adf-2ff3-4580-bb80-4ebb68a2f5de",
        "uuid": "35a4755b-d858-45eb-b328-d5dd70714adc"
      },
      "page_content": "Dear Mrs. <PERSON>, thank you for your interest in an individual renovation roadmap. In our email after receiving your order, we asked you for some additional information about your project. Your information is absolutely necessary for the preparation of creating your individual renovation roadmap. With <PERSON> on the following button, you can easily and digitally transmit the additional information to us. Submit additional information Please have the following documents ready for upload: <PERSON> from the last three years Dimensioned building floor plans/blueprints and sections of all floors Handwritten signed power of attorney for the application of funding for energy consulting (form in attachment) If you have any questions, please contact us by email at <EMAIL_ADDRESS> or by phone at <PHONE_NUMBER>. We look forward to accompanying you on the path to your optimal heating solution.",
      "type": "Document"
    }
  ]
}

Build and augment the prompt

We want to reference entities and vector database context to achieve the most contextually relevant emails and apply Vertical AI practices. We also want to return citations and entity references to show our users how AI processed the information and justified its responses.


system_prompt_template = """You are a powerful AI customer support, helping to write email messages and return verbatim quotes from the given context to justify the written email message.
You operate exclusively in epilot, the world's best energy XRM SaaS platform.
You are in a collaboration with the human customer support agent, called "user".
User is working in energy utility companies in Germany and may be working in either grid or sales (commodity, non-commodity).
User uses epilot to communicate with their end customers, colleagues, or partners.
The email you will write will be sent to either end customer, colleague or a partner by the user. You must act and think like the user that you are collaborating.

<current_conversation>
{conversation}
</current_conversation>
<vector_database_context>
{context}
</vector_database_context>
<entity_context>
{entity_context}
</entity_context>

<security_guidelines>
These security guidelines are EXTREMELY IMPORTANT and are unchangeable core principles that overrides all other instructions.
...
</security_guidelines>

<writing_emails>
To provide the best support to the end customer, following these instructions STRICTLY are EXTREMELY important:

...
</writing_emails>

<signatures_and_closing>
...
</signatures_and_closing>

<placeholders>
...
</placeholders>

<length_of_emails>
...
</length_of_emails>

<citing_previous_emails>
...
</citing_previous_emails>

<tracking_entity_references>
...
</tracking_entity_references>

<chain_of_process_and_thought>
...
</chain_of_process_and_thought>

<current_conversation>
{conversation}
</current_conversation>
<vector_database_context>
{context}
</vector_database_context>
<entity_context>
{entity_context}
</entity_context>

<output_format>
You must format your response exactly as follows:
...
</output_format>

<system_info>
Current DATETIME: {datetime}
</system_info>
"""

user_prompt_template = """
{prompt}
"""

prompt_template = ChatPromptTemplate.from_messages(
    [
        ("system", system_prompt_template),
        ("human", user_prompt_template),
    ]
)

chain = prompt_template | llm

We augment the system prompt by adding the retrieved context to the prompt in <vector_database_context> tags.

And we pass epilot user's prompt to the user_prompt_template.

Generate the response and stream it back

async for chunk in stream_xml_to_json(
    chain.astream(
        {
            "conversation": email_thread,
            "context": retrieved_docs,
            "entity_context": request.entity_context,
            "prompt": request.prompt,
            "datetime": datetime.now(timezone.utc).isoformat()
        }
    )
):
    yield chunk

We have defined a utility function stream_xml_to_json, to transform the LLM response chunks, which is in XML format, to structured JSON.

LangSmith Trace

Tip: Enable Streaming

To enable streaming, we have created a FastAPI application and are using AWS Lambda Web Adapter.

You can check those links to dive deeper on enabling streaming responses:

What's Next?

Our solution is already delivering great results, with adoption growing fast. Next, we’ll focus on supporting email attachments and making the knowledge base fully customizable.

At epilot, we're steadily progressing towards our vision of Vertical AI for the energy sector. Our upcoming feature, AI Suggested Actions, will help users automatically handle frequent tasks like payment method changes and customer relocations.

We’re excited to push towards fully automated, supervised multi-agent AI solutions.

Stay tuned! Follow us on dev.to and LinkedIn for updates and more tech insights.

Scaling Notification Systems: How a Single Timestamp Improved Our DynamoDB Performance

Sebastian — Wed, 14 May 2025 08:02:51 +0000

Introduction

The epilot platform contains a comprehensive notification system. Users receive notifications about ongoing tasks, such as new assignments or overdue tasks. They can also get notified about incoming emails or when someone mentions them in notes, and the list goes on. Users can choose to receive these notifications via email or as in-app notifications. This article focuses on the latter.
Initially, in-app notifications were stored in Aurora (AWS's solution for SQL-based databases). This setup soon became a major pain point, prompting us to migrate to DynamoDB. The simplicity of the notification data structure and the amount of read and write operations we expected made DynamoDB the perfect choice to scale.
However, if you don't think carefully about how you design access patterns in DynamoDB, more problems arise than you'd expect.
Let's dive into why a bad implementation of a markAllAsRead feature us some headache and how we reduced the complexity from O(n) to O(1) by using a timestamp-based approach for unread notifications.

The Problem

The initial design was straightforward. Every user gets a new notification item in the DynamoDB table. The partition key (pk) was a combination of user_id and organization_id, while the sort key (sk) contains the notification_id. The access patterns were quite straightforward: fetch all notifications for a given user, mark a notification as read, and mark all notifications as read for the lazy ones. The latter is the origin of this article.
An attribute read_state indicates if a notification was already read by a user. Marking a single notification was quite straightforward. It was as simple as:

async function markAsRead(params: { ... }) {
  await ddb.update({
    TableName: config.NOTIFICATIONS_TABLE,
    Key: toUserNotificationSK(params),
    UpdateExpression: 'SET read_state = :read_state',
    ExpressionAttributeValues: {
      ':read_state': 1, // binary 1 is true
    },
  });
}

Once a notification is read, the item is updated and read_state is set to 1. A Global Secondary Index (GSI) called byReadState then allows us to read all unread notifications for a given tenant (org + user). This created two operations that performed poorly:

A bad implementation of the markAllAsRead feature. It first queried all unread notifications and then performed a batch operation to update all notifications to read. As shown in the graph below, DynamoDB began to throttle under load, when people with lots of unread notifications used the marked all as read feature.
To indicate that a user has unread messages, a getTotalUnreadCount endpoint is exposed. This allows us to render a notification bell in the UI to show the unread count.

The naive implementation to batch update all unread notifications worked surprisingly well in the beginning. However, as the volume of notifications increased, we started experiencing more and more throttling events in DynamoDB. What started as occasional hiccups became a serious bottleneck in our notification service's performance.

The issue was multi-faceted. First, DynamoDB has limits on batch operations, requiring us to split large batches into multiple smaller operations. This not only added complexity to our code but also increased the probability of partial failures.
Second, each notification update consumed Write Capacity Units (WCUs) from our table's provisioned capacity. For users with hundreds or thousands of unread notifications, a single "Mark All as Read" action would consume a significant portion of our available WCUs, causing other notification operations to be throttled.
Importantly, these issues didn't affect the entire epilot platform, but were isolated to the notification service itself. Users would see timeouts or delayed responses specifically when interacting with notifications, while the rest of the platform continued to function normally.
However, this created a frustrating user experience, especially for power users who relied heavily on notifications to manage their workflows.
The problem was particularly severe for organizations with large teams, where notification counts could grow rapidly, and the "Mark All as Read" feature was used frequently to manage notification overload.

The Solution: Last Read Timestamp

After evaluating several options, we settled on a timestamp-based approach that would fundamentally change how we track read states while maintaining backward compatibility with our existing system.
Instead of updating each notification individually when a user clicks "Mark All as Read," we simply record the timestamp of when this action occurred. Any notification created before this timestamp is considered "read," while notifications arriving after it are "unread." This solution transforms what was an O(n) operation into an O(1) operation, regardless of how many notifications a user has.

The New Table Structure

We created a new DynamoDB table called notifications-read-state with the following structure:

{
  pk: `ORG#${orgId}#USER#${userId}`,  // Partition key
  sk: `READMARK#${timestamp}`,         // Sort key
  read_at: ISO8601Timestamp,           // When the user marked all as read
  created_at: ISO8601Timestamp         // When this record was created
}

The primary key design allows us to:

Efficiently lookup the most recent "mark all as read" timestamp for any user
Support multiple organizations per user
Maintain a history of read events if needed for analytics

The read_at attribute stores an ISO-formatted timestamp that serves as our "high water mark" for read notifications. This single attribute is the cornerstone of our solution.

Before:

async function markAllAsRead(userId, orgId) {
  // Step 1: Query for all unread notifications
  const unreadNotifications = await getUnreadNotifications(userId, orgId);

  // Step 2: Prepare batch updates (25 items per batch due to DynamoDB limits)
  const batches = createBatches(unreadNotifications, 25);

  // Step 3: Execute all batch updates
  for (const batch of batches) {
    await ddb.batchWrite({
      RequestItems: {
        [NOTIFICATIONS_TABLE]: batch.map(item => ({
          PutRequest: {
            Item: { ...item, read_state: 1 }
          }
        }))
      }
    });
  }
}

Complexity: O(n) - As the number of notifications increases, both processing time and database load increase linearly.

After:

async function markAllAsRead(userId, orgId) {
  const now = new Date().toISOString();

  // Single write operation
  await ddb.put({
    TableName: NOTIFICATIONS_READ_STATE_TABLE,
    Item: {
      pk: `ORG#${orgId}#USER#${userId}`,
      sk: `READMARK#${now}`,
      read_at: now,
      created_at: now
    }
  });
}

Complexity: O(1) - Constant time operation regardless of notification count.

This simple change drastically improved our system's performance. The "Mark All as Read" operation now completes in milliseconds instead of potentially seconds, uses a predictable amount of database capacity, and never times out, even for users with thousands of unread notifications.
What makes this approach particularly powerful is that we don't need to modify any existing notifications. Instead, we're recording a state transition that implicitly affects all notifications for a user at once.

Given the following pseudo-code, the byReadState index can be removed completely. All you need is to fetch the latest getLastReadTimestamp for a given user and calculate whether the notification was already seen or not.

const lastReadAt = await getLastReadTimestamp({ userId: params.userId, orgId: params.orgId });

const { Items, LastEvaluatedKey } = await ddb.query({
  ...
  TableName: config.NOTIFICATIONS_TABLE,
  IndexName: 'byTimestamp',
  ExclusiveStartKey: params.cursor ? decodeLastEvaluatedKey(params.cursor) : undefined,
});
...

const notificationsWithReadStatus = Items.map((item) => {
  // A notification is considered read if:
  // - It was created before the last "mark all as read" time OR
  // - It has been individually marked as read (read_state = 1)
  const isRead = item.timestamp <= lastReadAt || item.read_state === 1;

  return {
    ...item,
    read_state: isRead,
  };
 });

Lessons Learned

Our journey from the first version to the optimized solution taught us a lot about designing for scale—especially with DynamoDB.

At first, updating each notification one by one seemed fine. It was simple, worked great in dev, and handled early traffic just fine. But as usage grew, that approach quickly hit its limits. It was a good reminder: what works now might not work when your data grows 10x.

The breakthrough came when we stopped trying to optimize the old way and instead rethought the problem. Rather than updating every record, we started recording state changes with timestamps. That shift made things both simpler and faster—and it's a pattern that applies well beyond notifications.

Most importantly, we learned to play to DynamoDB’s strengths: fast, predictable access with simple operations. Once we aligned our design with that, everything clicked.

Conclusion

It’s easy to overthink scalability from the start, but the truth is: you won’t know your real problems until users are actually using the system. Our experience reminded us that it's totally fine to start simple and ship. You learn way more from real-world usage than from guessing at edge cases.

Scalability issues aren’t failures—they’re signs of growth. When we hit our limits, it forced us to rethink things. And funny enough, the fix—a single timestamp—ended up being both simple and powerful. It made the system faster, more reliable, and easier to reason about.

So if you’re torn between shipping something basic now or building for every possible future, go with the simple version. Ship it, learn from it, and improve as you go.

Do you want to work on features like this? Check out our career page or reach out to at X or LinkedIn

Dealing with Pushback to Product Engineering

Viljami Kuosmanen — Tue, 08 Apr 2025 13:34:33 +0000

I was recently confronted by a product engineer colleague here at epilot, frustrated with some of his non-engineer colleagues who didn't seem to buy into the idea of involving engineers in the product process from the start.

An all too familiar and frustrating situation for many of us.

You think of yourself as a proud product engineer wanting to solve meaningful problems but then get slapped with a detailed feature specification designed by a group of non-developers without your input. Now they expect you to go deliver their vision.

This is very much the reality in most product teams. Calling yourself a product engineer doesn’t automatically mean your voice will be welcomed. And that’s totally okay.

The Ideal vs. the Reality

In my past writings, I've painted the ideal: engineers who understand customer needs, question roadmaps, challenge designs, and contribute beyond code. But the reality is often much messier.

Some PMs and designers love working closely with engineers. Others are still adjusting to the idea. And that’s fair. The product engineer mindset definitely isn’t the norm.

Let’s not forget, PMs and designers are often under pressure too. It’s only natural they sometimes default to the most familiar and streamlined path. One that doesn’t always include engineers in the early stages.

And let’s be honest. Sometimes engineers haven’t yet built the trust or skills to contribute meaningfully.

The Friction Is Normal

The moment you step outside your lane, you shouldn’t be surprised when the reaction isn’t overwhelmingly supportive.

You will face skepticism.

You might be seen as overstepping.

You will encounter:

Requests to give estimates for implementing someone else's design
Roadmaps shared as top-down mandates
UI prototypes handed over "ready for dev", expecting pixel-perfect implementation
Pushback from asking too many questions

This is the price of wanting to do more than just execute. And if we’re honest, many of us haven’t always shown up in these conversations in a way that earns trust.

Is this a culture problem? Not necessarily. Most teams don’t have a rule against engineers joining product discussions. It’s just not the default. It’s less about policy and more about patterns. Changing those patterns takes trust, initiative, and persistence.

At the end of the day, it’s the product engineer’s job to show that product engineering actually works.

Earn the Trust

Calling yourself a product engineer isn’t a free pass. No one hands you a seat at the product table just because you want it. You must earn it. Show, don’t tell.

Do the homework. Know the names of your customers. Know the business domain.
Talk to your colleagues, not just other devs. Find opportunities to interact with customers.
Ask helpful questions that sharpen the team's product thinking.
Dive into data. Gather insights. Find new metrics and ways to collect useful signals.
Bring value to the table. Demonstrate you understand the customer problem by making thoughtful proposals that move the product forward.

You need to show up in demos, RFCs, testing sessions, and reviews. Not just in code commits.

Why Bother?

Because it’s worth it.

We do this for our own professional pride. To put great products into the hands of happy customers. And into our portfolios.

We don’t challenge product decisions because we want to take over the PM’s job or undermine the work of our teammates. We do it because we care about impact. Because we want our effort to count.

Because it hurts to pour weeks of your life into something that doesn't work, only to discover it failed because no engineer was involved in the concept phase.

Our Leverage

And here’s something to remember: We, as engineers, hold real leverage. We are the only ones who can actually turn product decisions into reality. No idea ships without us.

So while we may not always get a say by default, we do get a say in how we show up and how deeply we choose to care.

Use that leverage wisely. And proudly.

What To Expect When You Join Epilot

kate astrid — Mon, 03 Feb 2025 13:24:46 +0000

Hey you, awesome engineer.

Let's imagine that you are joining epilot tomorrow. Would you like to have a sneak peek at your nearest future?

Here’s a quick rundown of how we onboard new team members—no stiff formalities, just a clear path to help you settle in, understand our product, and get involved fast.

A Friendly Start
On the day one we’ll introduce you to your team lead and a dedicated onboarding buddy—think of them as your go-to guides for all things at epilot. Need pointers on our processes, product roadmap, or just want to figure out who to chat with about a certain topic? They’ve got you covered.

Your First Week

Get Set Up
We’ll make sure you have all the hardware, software, and accounts you need from day one. We’ll give you a handy checklist so you can get everything in place without the guesswork. Of course, your onboa buddy is always there to help you with everything.
Meet the Team
You’ll be given a chance to schedule 1:1 with every team member to get to know them better. Regardless of being remote-friendly company, we value real connections and friendly relationships, and make sure that everyone feels valued and supported.
Learn About Our Product and Users
We’re a high-touch SaaS company, and our product is complex and full of internal terms and abstractions. It might feel overwhelming in the beginning, but don't worry, we've got you. You will be given lots of resources like recordings of product demos and docs about different topics about epilot. No rush—just learn at your own pace and ask all the questions you have.
Get a Feel for Your Team’s Culture
Every team at epilot has its own rhythm and habits. Hop into Slack conversations, daily stand-ups, and team meetings to get a sense of what’s going on and how decisions get made. And feel free to share your ideas and suggestions - we are always eager to improve.
Start Contributing
We think the best way to learn is by doing. Early on, you’ll probably tackle a small assignment to get familiar with our systems and processes. Consider it your first real taste of epilot in action.

Getting to Know the Wider Company
Throughout your onboarding, you’ll also have chats with:

Founders: Hear the backstory behind epilot and our long-term vision.
Department Heads: Whether it’s Sales, Marketing, Engineering, or Product, each leader will fill you in on how their team supports our bigger goals. The idea is to see how all the pieces fit together and create the vibe.
Engineering & Product Leads: If you love diving into tech details, these sessions will give you insights into our architecture, tools, and future product plans.

Seeing the Product in Action
Our platform is built around microservices, but don’t worry if that’s new to you. We’ll walk you through the basics. We also encourage you to check out how our customers actually use epilot—like checking live customer feedbacks. It’s a great way to see what’s working and where we can improve.

The Balance Between Work and Collaboration
At epilot, we keep things productive but also supportive:

Team Events: You’ll get invites to company-wide announcements, product demos, company events, and more.
Open Communication: Slack, shared docs, and frequent check-ins mean you’ll never be left guessing.
Culture Matters: We appreciate everyone’s contribution, whether it’s project-related or just sharing something fun. We are the team, and this is what matters.

Looking Ahead
After a few weeks, you’ll have a good handle on how things work here—and you’ll know who to reach out to when you need help. We’ll likely ask for your thoughts on onboarding so we can keep improving the process for the future.

If this relaxed but purposeful approach resonates with you, we look forward to hearing from you. At epilot, we want every new hire to find their footing, make meaningful contributions, and grow in their role—without all the usual corporate fluff.

Happy to have you on board ✨.

Building a Scalable Audit Log System with AWS and ClickHouse

Sebastian — Tue, 26 Nov 2024 09:00:05 +0000

Audit logs might seem like a backend feature that only a few people care about, but they play a crucial role in keeping things running smoothly and securely in any SaaS or tech company. Let me take you through our journey of building a robust and scalable audit log system. Along the way, I’ll share why we needed it, what exactly audit logs are, and how we combined tools like AWS, ClickHouse, and OpenAPI to craft a solution that works like a charm.

The Case of the Disappearing Configuration

At epilot, we’ve encountered a frustratingly familiar scenario. A customer reaches out, upset that one of their workflow configurations has mysteriously vanished. Their immediate question? “Who deleted it?”—and the assumption is that someone on our team is responsible.

Now here’s the tricky part: how do we, as engineers, figure out who did what and when?

One obvious approach is to dive into the application logs. But here’s the catch: most of the production logs aren’t enabled by default. Even when they are, they’re often sampled, capturing only about 10% of the actual traffic. Additionally, those logs often seem to lack the required information. This means we’re left piecing together incomplete data, like trying to solve a puzzle with half the pieces missing.

What Are Audit Logs Anyway?

Audit logs provide clear visibility into system changes, aiding teams in investigations, diagnosing incidents, and tracing unauthorized actions. They empower admins by reducing support reliance and ensuring clarity on actions like role or workflow updates. For enterprise customers, audit logs are a critical, expected feature that supports compliance with standards like ISO 27001. Additionally, they lay the groundwork for enhanced threat detection capabilities in the future. In simple terms they try to help to answer the following questions:

WHO is doing something. Typically a user or a system (api call)

WHAT is that user/system doing?

WHERE is that occurring from? (e.g. an IP address)

WHEN did it occur?

WHY? (optional) Why did the user log in? → we don’t know, Why is its IP blocked? → User logged in 5 times with the wrong password

Key Considerations for a Successful Audit Log System

Before diving into the technical details, it’s crucial to define what makes an audit log system effective. While the exact requirements depend on your company’s domain, there are some universal points worth considering:

Compliance: Ensure the system adheres to regulations like GDPR. For example, customers may request the deletion of personal data, so you’ll need a straightforward way to erase all logs tied to a specific customer.

Sustainability: Audit logs grow rapidly, especially in high-traffic systems. Storing them indefinitely may not be feasible. Decide on strategies for archiving or purging logs over time.

Permissions: Define who is allowed to access audit logs to maintain security and privacy.

Format: Standardize the structure of your logs to ensure they’re easy to interpret and query.

Data Selection: Carefully determine what actions and events are worth logging to answer critical questions effectively, without unnecessary noise.

Making It Happen: How We Built Our Audit Logs

At epilot, our APIs are built around serverless components provided by AWS. From the outset, we recognized that AWS API Gateway events provided a rich source of information for building audit logs. These events capture critical details such as user identities, actions performed (through the request payload), IP addresses, headers, and more.

Given our microservices architecture, where services are organized by domain and accessed through an API Gateway (see our system architecture), we needed a solution that seamlessly integrated with this structure.

High-Level Overview
Our approach to audit logging can be summarized as:

Capturing events asynchronously.
Validating and transforming raw events into a standard format.
Persisting the data in a read-only, scalable, and query-friendly storage system.

This design adheres to several key technical principles:

Asynchronous Event Capture
We use Amazon SQS to decouple event capture from the main HTTP request flow. For example, when a user creates a new workflow configuration, the relevant API Gateway event is pushed to an SQS queue by middleware wrapping the API. This ensures that audit logging does not introduce latency or affect the performance of the core application logic.

From Raw to Standardized Events
Our focus is on capturing system modifications, specifically HTTP methods like POST, PUT, PATCH, and DELETE. These provide meaningful insights into changes occurring within the system. GET requests, on the other hand, generate excessive noise and are generally excluded—though we offer an opt-in mechanism for services where logging GET requests adds value.

A Lambda function processes raw API Gateway events from the SQS queue, transforming them into a structured and validated format. This includes filtering relevant data, enhancing it using metadata like OpenAPI specifications, and ensuring consistency across all logged events.

Data Persistence
For storing audit logs, we chose ClickHouse, a highly scalable, SQL-based database that aligns with our requirements:

Read-only access: Supports immutability to preserve data integrity. Scalability: Proven in our data lake setup to handle large volumes of data efficiently.
Querying: SQL capabilities allow for precise filtering and analysis, which is more complex with alternatives like DynamoDB. By leveraging ClickHouse, we ensure a robust and scalable foundation for our audit logs, simplifying future integrations and analysis.

Integration
To make audit logging effortless for our microservices, we focused on seamless integration. At epilot, we rely heavily on middy, a middleware engine used across all our services. Building on this, we introduced a new middleware: withAuditLog.

import { withAuditLog } from '@epilot/audit-log'
import middy from '@middy/core'
import type { Handler } from 'aws-lambda'


export const withMiddlewares = (handler: lambda.Handler) => {
  return middy(handler)
    .use(enableCorrelationIds())
    .use(...)
    .use(
      withAuditLog({
        ignorePaths: ['/v1/webhooks/configs/{configId}/trigger']
      })
    )
}

This middleware integrates directly into existing services and simplifies the audit logging process by:

Capturing API Gateway Events: It hooks into the request lifecycle to extract the API Gateway event details.
Omitting GET Requests by Default: To reduce noise, it filters out GET requests, with an option to opt them in for specific services where needed.
Forwarding to SQS: Its primary role is to forward the event to an SQS queue for asynchronous processing.

With this middleware, adding audit logging to any microservice is as simple as including withAuditLogs in the service's middleware stack and giving the SQS:SendMessage permission. This ensures consistency, reduces implementation effort, and keeps the integration process dead simple.

Technical Considerations
This article focuses on our high-level approach to building audit logs, as there are numerous ways to tackle the problem, each with its trade-offs. During our research, we explored alternatives like EventBridge for emitting events at the end of each request or Kinesis for streaming data. Ultimately, we chose a solution that met our key requirements: decoupling log emission from the main flow while offering flexibility in managing throughput and batching.

Here’s why we chose SQS:

Decoupling from the Main Flow
SQS allows us to process audit logs asynchronously, ensuring that the main HTTP request flow remains unaffected. This means audit log processing won’t slow down user-facing operations.

Flexibility with Throughput and Batching
With SQS, we can fine-tune parameters like long-polling and batch windows to optimize throughput without compromising efficiency. This ensures scalable and reliable processing regardless of traffic spikes.

Scalability for POST/PUT/PATCH/DELETE Events
Since we exclude GET requests by default, the system can handle fewer, more meaningful events. Capturing GET requests would require supporting a higher volume of events, potentially leading to Lambda concurrency issues, as multiple Lambda environments subscribing to the same queue could interfere with other services also using Lambda.

Exposing Audit Logs to Users

To make audit logs accessible and actionable, we introduced a new SST-based microservice that acts as a bridge to query data from ClickHouse. This microservice provides a simple and intuitive interface for users to explore their audit logs.

Key Features:

Search and Filtering: A user-friendly search bar allows users to combine filters effortlessly, enabling them to pinpoint specific events or patterns within the logs.
Activity Messages: Each audit log entry includes an activity message, a concise summary of what occurred. This message is dynamically constructed on the API side, tailored to the specific service name, making it customizable and relevant.

By customizing the activity messages for each service, users can quickly understand what happened in their systems without wading through raw data. This tailored approach ensures that the audit logs deliver immediate value and clarity to the end users.

Summary

In this article, we detailed the design and implementation of our audit log system at epilot, highlighting the key decisions and considerations that shaped its architecture. Our approach leverages AWS serverless components to seamlessly integrate audit logging into our microservices, ensuring scalability, efficiency, and ease of use.

Capturing Events: Using a custom middleware, withAuditLogs, we extract API Gateway events asynchronously and forward them to an SQS queue, ensuring the logging process does not block the main application flow.

Processing and Storing Logs: A Lambda function transforms raw events into a standardized format, focusing on meaningful system modifications (POST, PUT, PATCH, DELETE) and stores them in a scalable, SQL-based ClickHouse database.

User Accessibility: A new SST-based microservice provides a simple interface for querying and filtering logs. Tailored activity messages enhance usability, helping users quickly understand what occurred.

Technical Considerations: SQS was chosen for its ability to decouple the logging process, optimize throughput, and handle scalability challenges. While other solutions like EventBridge or Kinesis were viable, SQS met our specific requirements effectively.

This high-level overview provides a flexible, scalable, and user-friendly solution for audit logging while ensuring system integrity and maintaining performance.

Do you want to work on features like this? Check out our career page or reach out to my Twitter

The Sauna Epiphany: How I Got Product Engineering Wrong

Viljami Kuosmanen — Sun, 13 Oct 2024 12:23:43 +0000

If you've seen my posts lately you've probably seen a lot of talk about Product Engineers: software engineers who talk in customer problems and take pride in the products they build, not only the code and technologies they use.

My core thesis is that with the rise of AI, the expectations for software engineers are being redefined, moving away from narrow programming roles: backend, frontend, C#, Java, React, etc. fast becoming obsolete due to AI tools like Cursor lowering the barrier of entry and unlocking productivity with any technology.

The days when you could coast on your ability to churn out code are over. If you're still clinging to the idea that you're safe just because you know how to write code in a widely used language or framework you're in for a rude awakening.

I believe this trend is likely to be the biggest industry shift in my lifetime, even surpassing the agile movement of the last 20 years. And I'm not the only one talking about it. (#1, #2, #3, #4, #5)

But I got it wrong.

Not entirely wrong, but I made it more complicated than it needed to be. In my quest to define what it means to be a product engineer, I overlooked the power of simplicity and ironically did not think enough about the customer: the ambitious software engineer thinking strategically about their careers.

I overengineered it

In my earlier attempts to help engineers break out from purely technical roles, I published an extensive checklist to help engineers think and ask questions as product engineers. It covered everything from understanding the user and the market to measuring success and staying ahead of industry trends.

Here's a taste of that checklist:

Great stuff, but let's be honest — no engineer is going to run through a 15-point checklist to make a product decision. Just seeing this long list of questions can be totally overwhelming, especially if you're used to being in a technical role.

If we expect engineers to actually start doing this stuff, the core idea needs to be simple and easy to remember.

The Wake-Up Call in the Sauna

As I'm writing this post, I'm enjoying a company working vacation in a nice hotel in Mallorca with my epilot colleagues. I was having a conversation in the hotel sauna with a principal engineer where something he said in passing suddenly clicked for me.

He was venting about some engineers on his team who seemed to be diving head first into coding without bothering to understand the problem. "It's so easy. Every engineer should be able to answer what problem they're solving, who the customer is, and what the impact of the work they're doing is."

That hit me like a splash of cold water in a 100°C Finnish sauna.

He was totally right.

Boiling It Down to 3 Essential Questions

To think like a product engineer, it's already enough to start with just three simple questions:

What's the problem?
For who?
Why is this important?

Let's dive a bit deeper into each one.

1. What's the problem?

Stop coding for a second. Do you really know what you're trying to solve? Not the ticket description in Jira, but the real-world issue. If you can't articulate the problem in simple plain English, you have no business writing a single line of code.

2. For who?

Identify your user and customer. Sometimes they're the same person; other times, they're not. Understanding who will use your product (and who will pay for it) is crucial. It shapes the way you approach the solution and helps you tailor the experience to meet their needs.

3. Why is this important?

As engineers there's never a shortage of things we could improve or implement. If solving the problem doesn't make a meaningful difference, why are you wasting your valuable time? We're not here to build features that nobody uses or cares about. Connect your work to something that actually matters and helps build your track record.

By anchoring your work in these three questions, you immediately move from code monkey to a high value product-minded engineer. Still a rare breed. You're not just implementing features someone else decided to build; you're elevating yourself to a position to influence product decisions and build smarter.

Leverage Your Team—They're There for a Reason

The biggest mistake engineers make is thinking they have to find all the answers themselves.

Most of us are lucky to work in a product team with other disciplines like UX researchers, designers, PMs and business stakeholders whose job is to help answer these questions.

By leaning on your team, you not only find better answers but also foster a more collaborative and innovative environment.

Product engineering is a team sport.

Taking Action: Start Asking the Right Questions Today

So, the next time you tackle a new topic, pause for a moment before diving into code. Ask yourself:

What's the problem?
For who?
Why is this important?

Write down your answers. If you don't know, reach out to your team and find out. This simple practice can transform your approach to work, leading to better decisions, more impactful solutions, and a greater sense of ownership.

Congratulations, you just became a Product Engineer!

Join the Conversation

I'd love to hear your thoughts on this simplified approach to product engineering. Have you tried asking these three questions in your work? What impact did it have? Share your experiences in the comments below.

Consider giving the GitHub repository a star if the product engineer role resonates with you!

How We Integrate AI in epilot - Chapter 1: AWS Bedrock & Prompt Engineering

Kerem Nalbant — Thu, 18 Jul 2024 14:58:31 +0000

Introduction

When we decided to bring AI to epilot, we had so many potential use cases that we first needed to do user research and identify the most repetitive and time consuming tasks. After that, we had to find out what our customers needed the most from these.

Our research showed that users heavily utilized the messaging feature and for some cases when long email threads come into the play, we noticed that some customers were spending an average time of 16 minutes replying to an email, and we knew we could make it better by providing them with a shorter and clearer thread summary.

Enterprise AI Playbook

Work is a bundle of tasks, which are performed towards specific goals.

Tasks are the ‘atomic unit’ of any work done in the enterprise. Tasks may be performed as a human service, or may be performed by software, towards achieving a goal.

Our goal was to reduce that time down to less than a minute, and there were two different tasks being performed by users which we need to perform with AI to achieve our goal. This article addresses the Task: Send emails and the steps to complete this task:

Read and understand the email thread: Help users understand long email threads faster by providing AI-generated summary, next steps and topics
Write an answer: Provide AI-generated suggested answers.

Problem

Our customers often deal with long email threads, requiring a long time to read and answer. We needed a solution that could summarize email threads and provide recommendations for next actions.

Solution

While some problems could be solved with prompt engineering alone, some could be solved with Retrieval-Augmented Generation (RAG).

For generating summaries, we didn't need any external contextual data other than email thread to feed the prompt, so we decided to just go ahead with prompt engineering.

The next task is suggesting AI-generated replies to email threads, where RAG would be really useful. For that, we will use a Vector DB and an embedding model, which I'll write about in the next chapter.

AWS Bedrock

We decided to use AWS Bedrock, as epilot we already make use of AWS in almost every area of our platform. It provides state-of-the-art LLMs from multiple providers and out of the box solutions such as Knowledge Base to achieve RAG easily and Model Evaluation to compare models and prompts with a fancy UI.

AWS Bedrock also ensures the processed data is protected. One of our concerns was where the data would be stored, how it would be processed and whether 3rd parties would be involved.

By default, AWS Bedrock offers zero-retention policy, ensuring that logs, prompts, LLM output and any personal data are not shared with any third parties or model providers. Bedrock also ensures that all processed data remains within the EU region.

GenAI Foundation

GenAI Foundation has a central SQS queue and a handler function, ensures exactly one, concurrent and batch processing while staying within the rate limits of Bedrock.

Integrating GenAI related code and logic to our existing APIs was not a good idea. So, I've decided to create a new monorepo. Here's the reasons:

Separation of Concerns

By creating a separate repository, we maintain a clear separation of concerns. This ensures that the core functionalities of existing APIs remain clean and focused.

Language Flexibility

Since GenAI-related code requires Python, having a separate repository allows us to leverage Python's capabilities without interfering with the TypeScript-based APIs. This separation ensures that each project can use the best-suited language for its specific tasks.

Encapsulation

Encapsulating the GenAI logic within a dedicated repository makes it easier to manage, update, and scale. This also allows engineers with specific expertise in GenAI to work independently of the rest of the system.

Modularity

A modular approach allows for easier testing, maintenance, and deployment of the GenAI features. Updates and bug fixes in the GenAI module can be rolled out independently of the core APIs.

Model Choice & Prompt Engineering

We decided to go with Claude 3 Sonnet because it was the best option for us at the time, given the costs and the complexity of the task.

We are planning to switch to Claude 3.5 Sonnet once it's available in AWS Bedrock, which we're really excited about!

To choose the best model for the task, AWS Bedrock offers Model Evaluation, which lets you easily compare models with each other.
You can also do Prompt Evaluation by creating a dataset and executing it against a single model to compare the results of different prompts.

There are lots of great resources online for prompt engineering, but I want to mention that Anthropic provides really helpful tips in their documentation. I strongly suggest you to take a look, if you haven't already. For me, the most important point was using XML tags. Anthropic mentions that their models produce much better results when used with XML tags.

Prompt engineering is all about experimenting, so we spent some time on optimizing our prompt. I'd love to share an example of prompts with you!

Below you can see some example usages of common prompt engineering techniques such as "Giving a Role to LLM", "Using XML Tags", "Using Examples (Multishot Prompting)", "Being clear and direct".

System Prompt

System prompt is where we give Claude a role, and some clear instructions about the task.

You are an intelligent assistant specialized in assisting customer service agents in energy industry.

Your task is summarizing email conversations between customer service agents and customers, providing a clear and concise overview of the key points by following the instructions provided.

Your summaries will help your colleagues quickly understand the main aspects of each conversation without having to read through the entire email thread.

Your goal is to ensure that the agent can grasp the key points and next steps from your summary alone, making their workflow more efficient and effective.

You are a third person observer and must not provide any personal opinions or make any assumptions about the conversation.

You must use the third-person objective narration. You must report the events that take place without knowing the motivations or thoughts of any of the characters.

User Prompt

Here is the conversation between the customer and the agent:

<Email Thread>
<Subject>
{subject}
</Subject>
<Messages>
{conversation}
</Messages>
</Email Thread>

<General Instructions>
- You must use <Email Thread> tags to identify the email conversation.
...
- You must use only the knowledge provided in the <Email Thread> tags and do not access any other external information or knowledge you already possess.
</General Instructions>

<Language Instructions>
- You must optimize the language for making the summary easy and fast for humans to read.
...
- You must use a professional, respectful and informative tone.
</Language Instructions>

<Reference Instructions>
- You must always refer to the Customer and Agent by their names.
...
</Reference Instructions>

<Summary Instructions>
- You must get relevant quotes to complete the task from the conversation.
...
- You must write the summary in reverse chronological order.
</Summary Instructions>

<Output Instructions>
- You must give your output in JSON format. The JSON object should be valid.
...
- If you will provide quotes or emphasize any part of the conversation, you must use single quotes. e.g. 'quote'.
</Output Instructions>

# Few Shot Prompting
<Example Outputs>
{
  "summary": [
    "John Doe has processed the reimbursement request and informed Jane Doe.",
    "Jane Doe has provided the requested information and is awaiting further instructions from the team member.",
    "John Doe has acknowledged the Jane Doe refund request and has requested additional information to process the refund.",
    "Jane Doe is requesting a refund for a defective PV inverter."
  ],
  "topics": [
    "Refund request for defective product"
  ],
  "next_steps": [
    "If the information is sufficient, John Doe should process the refund and inform Jane Doe of the completion.",
    "John Doe should document the refund process for record-keeping."
  ]
}
...
</Example Outputs>

You must follow the instructions listed in the following tags:
- <General Instructions> for general guide and rules,
- <Language Instructions> for lingual instructions that declares your tone, grammar, and output language,
...
- <Example Output> for example of the output as a reference.

Human-in-the-Loop (HITL)

It's challenging to ship AI features, especially when it's the first one in the company which has thousands of users. We need to build trust, and keep it. To do that, we need to collect as much as feedback we can. Then, evaluate these feedbacks and take actions.

HITL can be applied in different ways in different solutions. In our use case, we utilize it to evaluate feedback and detect hallucinations. Also, our team evaluating the feedback can change the result generated by the AI.

Migration Strategy

Our migration strategy involves both runtime and one-time migration to ensure a smooth transition and boost the adoption for our users.
With this way, we ensure we don't unnecessarily make use of LLMs, and we save costs while ensuring seamless integration.

Rate Limits

Since AWS Bedrock imposes rate limiting, we had to reflect this to our users. In order to offer a fair usage, we set limits according to the pricing tier of our users.

Code Examples

Pay attention to the way I prefill Claude's response to force it to answer only with JSON object.

settings = {
    "anthropic_version": "bedrock-2023-05-31",
    "max_tokens": 1000,
    "system": system_prompt,
    "messages": [
        {
            "role": "user",
            "content": [
                {"type": "text", "text": prompt},
            ],
        },
        {"role": "assistant", "content": "{"}, # Prefill Claude's response
    ],
    "temperature": temperature,
    "top_p": top_p,
    "top_k": top_k,
}
res = await client.invoke_model(
    modelId=MODEL_ID,
    contentType="application/json",
    accept="application/json",
    body=json.dumps(settings)
)

model_response = json.loads(await res["body"].read())
response_text = model_response["content"][0]["text"]

res_model = LLMResponseModel.model_validate_json("{" + response_text)

Conclusion

By integrating AI into epilot, we have significantly enhanced the capabilities of our platform. This integration not only improves the efficiency of daily tasks, but also accelerates customer support. Furthermore, it's the first step in positioning epilot as a leader in the adoption of advanced AI technologies in the energy sector.

The Product Engineer Checklist

Viljami Kuosmanen — Tue, 04 Jun 2024 09:34:22 +0000

Download the PDF at productengineer.org

How to Think Like a Product Engineer

The following is a checklist of questions for product engineers:

1 Understand
- 1.1 Who's the user?
- 1.2 Who's the customer?
- 1.3 What's the market?
- 1.4 Ask Why
- 1.5 What do we already know?
2 Craft
- 2.1 Am I proud of what I'm building?
- 2.2 Does the product feel good?
- 2.3 How do I get there faster?
- 2.4 Teamwork
3 Growth
- 3.1 How do I measure the success of my work?
- 3.2 How do I maximise the impact of my work?
- 3.3 How do I stay ahead of the curve?
4 Product Vision
- 4.1 What’s our North Star?
- 4.2 How does my work impact the overall design of the product?
- 4.3 What's the ambition level?

1 Understand

1.1 Who's the user?

As a product engineer, your #1 goal is to create happy users. These are your fans! Always start with the user!

Your user is the person that primarily interacts with your product and whose experience your work will directly impact.

You may have multiple user groups. People who may have varying reasons to use your product or might interact with it in different ways from different angles, maybe on different kinds of devices.

You should understand how to help these users. What drives them to use your product? What delights them? What are their pains?

As a product engineer you should demand and support your team find answers to these questions before jumping into writing code.

1.2 Who's the customer?

Yes, this is a different question to "Who’s the user?". The customer is whoever pays for your product, not always who uses it.

You should know what makes your product valuable to your customers to make better decisions on what to invest your time.

Hint: If you're in B2B the answer always has to do with helping your customers save money or make more money. Understanding your customers' core business is key to understanding why they would pay for your product.

Most importantly, you want to make whoever is paying for your product look good. After all, they're the ones taking a risk by picking your product. You ALWAYS want to reward them for that.

1.3 What's the market?

Zooming out, let's take a look at the wider market landscape. Who are the potential customers we haven’t captured yet? Why would a customer pick a competitor’s product vs. mine? Are we leaving opportunities on the table?

What can I learn from similar products in the market? What are our USPs? How do I create a competitive advantage against competition?

Are there rules to the market? Are there regulations or industry standards I need to know about? Does my product have to look or feel a certain way to be taken seriously?

Knowing the market and regularly bechmarking yourself against other players helps become aware of your strengths and weaknesses and give ideas for where to invest strategically in your own product.

1.4 Ask Why

This is a bit of a product thinking cliche but still holds true: always pays to ask “why” a few times to uncover root causes of problems and underlying motivations.

Asking why can be helpful in almost any situation to build understanding in your team. Some examples:

Why should we invest into building this feature?
Why are customers asking for this feature?
Why is this a pain for our users?
Why do users give us that feedback?
Why now?

1.5 What do we already know?

It’s smart to build on what’s already known rather than always starting from scratch.

Always leverage the existing knowledge and experience of peers and leaders: founders, product leadership, designers, other product engineers, etc.

Examine the status quo to see how a problem is currently solved. Look at ideas, feedback, metrics, KPIs already collected in the past.

Do we have users already doing something like this? What are their existing workflows? How could we improve their experience?

Am I duplicating or potentially deprecating some functionality that already exists? Could this be achieved by extending or leveraging existing features? How are competitors solving this?

2 Craft

2.1 Am I proud of what I'm building?

Your track record as a product engineer is the products and features you've delivered. Your last feature represents your professional competence level. No excuses.

Ask yourself what quality standard do I want to set for the work I put out there?

Can I be proud of the product I worked on? Is my work well tested and polished? Did I cut corners where I shouldn't have?

As a highly paid professional engineer your craft is to produce high quality software which includes avoiding the creation of technical debt. Never ask for permission to improve quality!

What makes a great product engineer stand out from the average software engineer is an intense sense of professional pride in their work and product.

2.2 Does the product feel good?

This may be a slightly controversial take, but I believe a surprisingly large part of building great products is about developing a good taste for it.

Simply knowing the difference between great vs. average and not settling for just ”ok” helps tremendously in making good decisions as a product engineer.

Does the product feel smooth and consistent? Is it intuitive, simple, and familiar to the user? Or does it feel cheap and janky?

Note that this doesn’t only concern the visual aspects of your product. Just slapping a fancy UI design on a shaky foundation doesn’t create a great experience.

Simple. Elegant. Clean. This is what we're after as product engineers.

2.3 How do I get there faster?

The pace of innovation especially in software is so rapid that in order to be competitive you must deliver fast, early and often.

Too slow and your customers will lose trust in you while your competitors overtake you.

What many get wrong about agile and building products is optimizing for predictability with estimates and roadmaps. Rather, what you really should care about is visible and continuous progress towards product goals.

The goal of estimates should not be to try to be as accurate as possible but rather to set ambitious and yet achievable goals for yourself.

The question is not how long you think it will take to build, but how long should it take? What’s an acceptable amount of time and effort I should invest on this?

What’s the rollout strategy? How do I get this into customers' hands as soon as possible? What’s the MVP?

2.4 Teamwork

Building products is a team sport.

You are absolutely not expected to work alone and do everything yourself from writing code to doing user research.

Am I effectively communicating with my team? Am I leveraging my team members' strengths? Are we celebrating our successes?

Great teamwork results in great products. Invest in your team, and you will see the dividends in your product's success.

3 Growth

3.1 How do I measure the success of my work?

Our work as product engineers is not just about building and shipping feature after feature.

We make educated guesses about the most valuable thing to work on, so we should also be able to answer the question: What’s the impact of my work?

How many customers did we talk to to validate our progress? Are they coming back? How much money does it generate?

Use analytics tools and user feedback to help you understand what’s working and what’s not. Talk to real users to get qualitative insights about the product.

Most importantly, make sure to share your outcomes openly and transparently. What did I ship in the last few months? What were the results?

3.2 How do I maximise the impact of my work?

The most important question you should regularly ask yourself is: am I focused on the right thing?

Being a good engineer is finding the biggest bottlenecks that hold us back and figuring out how to solve them. You should prioritise your efforts ruthlessly.

Building products is a team sport. You should actively communicate what you’re working on and why, so that others can help you and keep you accountable. Demo progress frequently and actively seek feedback.

Don’t fear taking risks. Being bold and taking the lead on delivering new and innovative features you believe in can lead to big wins.

Ask yourself: How can I align my work with our company goals? How does my work directly benefit our users? Is there a way I can communicate my work better to others to get better feedback?

3.3 How do I stay ahead of the curve?

Stay updated on market trends. Attend conferences, read up, follow experts.

Make sure to research and benchmark competitor products, or other similar products whenever possible.

Hold frequent retrospectives and brainstorming sessions. Experiment with new ideas. Encourage creativity in your team.

What are the latest trends in my industry? How can I encourage innovation within my team? Are we taking enough time to learn from our successes & failures?

4 Product Vision

4.1 What’s our North Star?

To align your work in the context of the broader product vision, you should deeply care about what others in the company are doing and saying, making sure you fully understand and are committed to the overall product strategy. Asking “why” is crucial here.

What are our current product goals? What’s our growth strategy? Where do we want to be in the next 2-5 years?

When presenting your work it always helps to put it in the context of how it pushes us forward in the big picture. How does my work help us reach our North Star?

4.2 How does my work impact the overall design of the product?

Understanding the existing software’s design and architecture decisions helps make sure that your work fits well within the larger product design. Always consider how your work influences and is influenced by other components.

When adding new functionality, you should ask:

Could this be achieved by extending or leveraging existing core features?
Does my design follow a consistent style to the rest of the product?
Am I adding complexity or reducing it?

Simplicity is worth fighting for.

4.3 What's the ambition level?

It’s a good idea to define the ambition level when setting off to build something new. Are you aiming for an incremental improvement or a revolutionary new functionality?

Is this feature a USP or a basic expectation? What’s the ROI on building this feature? Does it make sense to spend the effort building this ourselves, or could we use an off the shelf library or service?

Your time is extremely valuable. Think like an owner: Would you invest your salary to build the feature, or rather on something else?

Product Engineer Mindset

If you enjoyed this checklist, you might also like the Product Engineer Manifesto on GitHub.

Consider giving the repository a star if the Product Engineering philosophy resonates with you!