DEV Community: Steve Sewell

How I use Cursor (+ my best tips)

Steve Sewell — Wed, 12 Mar 2025 08:41:11 +0000

I was just chatting with one of our senior engineers, and he mentioned that he barely types code anymore—he just types into the agent box in Cursor.

I told him that is basically what I do these days as well (~80% of the time).

This only works well for complex tasks if you know what you are doing, so let's get into how exactly I use Cursor and my best tips.

Using YOLO mode

One of the coolest features that people often overlook in Cursor is letting the agent write code until it verifies the code is correct beyond just lint. To do this, you need to turn on YOLO mode.

Go to settings, scroll down, and make sure you turn YOLO mode on. Don't be scared by the name. When I first heard it, I figured I shouldn't use it, but you absolutely should, in my opinion.

When you check the box, you get a bunch more options. You can give a prompt, an allow list, and/or a deny list. Here's my prompt for YOLO mode:

any kind of tests are always allowed like vitest, npm test, nr test, etc. also basic build commands like build, tsc, etc. creating files and making directories (like touch, mkdir, etc) is always ok too

This way, if it wants to run mkdir, tsc, or check for lints directly, it can. A really cool workflow is to run tsc, and if there are any build errors at all, it'll just go ahead and fix them. It'll find any build errors in any file, fix them up, and iterate until the build passes. That's awesome.

Handling complex tasks

Let's explore how I use Cursor for more complicated things. I'm going to give it this prompt:

Create a function that converts a markdown string to an HTML string.

I like this prompt because pretty much no AI can consistently one-shot this. It can take a few iterations to figure it out.

Traditionally, you'd type this in, see some code, test it by hand, come back and say, "Oh, that's not right," and go back and forth. You start to feel like your job is a bit like a QA tester. That's fine, but it's not always fun.

That's why people invented automated tests and test-driven development. I'm not a fan of test-driven development in general, but when it comes to AI, I actually really am. So let's make this prompt a thousand times better, especially for non-trivial things, by adding one more line:

Write tests first, then the code, then run the tests and update the code until tests pass.

I didn't have to tell Cursor anything. I didn't have to add example context files or approve it creating a test directory. It's going to find that from the package.json—it's quite smart.

Watching the AI work

Here's where it gets interesting. Cursor is creating a test file, and unlike most AI-generated code, we get some level of guarantee that the behavior of the code the AI writes is correct. If it passes these tests, it's at least doing something right.

Now it's writing its implementation code, and it's going to run the tests. We've got six tests failed. Because I have YOLO mode on and told it it can run tests automatically, it's iterating on the code until the tests pass. And I'm doing nothing.

You do need to babysit these things. There are plenty of times when it's definitely just going off track. It's super common that I have to hit stop and say, "Wait, wait, wait, you're way off track here. Reset, recalibrate, get back on the right track."

In my case, eventually, all tests passed!

Building on existing test suites

Another cool thing you can do is build off an existing test suite. You can say, "Let's add a few more test cases, then make sure the code passes."

In another project I work on, we have certain compilers and converters for Builder. I have this one converter that sometimes errors. What I do is I grab the code that couldn't convert from our logs every few days. I paste it into Cursor and say, "Run this code and see what doesn't compile, then write a test for that issue and then update my code until all the tests pass."

It's amazing. I just do that every few days, pulling in new cases. It's a great way to improve your code's robustness.

UI design with Cursor

Someone in the comments asked if you can make a design look good with Cursor. Unfortunately, Cursor and other LLMs are very bad at checking if a design looks right in your code.

If you're not already familiar, you could use Builder.io (our Figma plugin) to convert designs to code. It gives you a certain level of guarantee, especially if you set up your design file a bit.

Here is my preferred design to code workflow:

Fixing TypeScript errors

Let me show you another technique I like a lot.

Let's say I've been working on some code for a while, just coding away, and then I realized - ah, TypeScript issues. You run the build and hit all these errors. Here's what you do: Just go over to Cursor and say

I've got some build errors. Run nr build to see the errors, then fix them, and then run build until build passes.

I end a lot of my work this way. I do a bunch of stuff, there's lint issues or whatever, who cares. I literally just tell Cursor to fix everything.

In all projects, I usually keep a command I call "pre-PR". It's usually the build steps that run the fastest. It might not run the end-to-end tests that run in the CI, but I rarely break those. It might just run tsc, Prettier, and ESLint - all the fast stuff that can quickly tell me if this is going to break the CI build or not.

Then I just have Cursor run that and fix up the PR until everything passes, and it works pretty great for me, even in large and complex projects.

Debugging with logs and iterative fixing

Sometimes, when you're dealing with a particularly tricky issue, it can be helpful to have Cursor output logs and then use those logs to fix the problem. Here's how I do it:

Let's say we're still struggling with our original issue. We've tried a few things, but we're not getting the results we want. Here's what I do next:

I tell Cursor: "Please add logs to the code to get better visibility into what is going on so we can find the fix. I'll run the code and feed you the logs results."
Cursor will add logging statements to the code at key points.
I run the code and collect the log output.
Then I go back to Cursor and say something like, "Here's the log output. What do you now think is causing the issue? And how do we fix it?"
I paste the raw log output for Cursor to analyze.

This approach gives Cursor more concrete information to work with. It's like giving it a diagnostic report of what's happening inside the code.

From there, Cursor can propose a more targeted fix based on the actual behavior of the code, not just its static analysis.

This method is particularly useful for those hard-to-crack bugs where the issue isn't immediately obvious from just looking at the code. It's like having a junior developer who can quickly add logging, run the code, and then help you interpret the results.

Remember, though, that this process might take a few iterations. You might need to add more logs, or focus on different areas of the code. But it's a powerful technique for solving complex issues with the help of AI.

Using Command K and Command I

Let me show you a couple of other cool things you may or may not already know. These are really handy shortcuts in Cursor.

Command K is great for making quick changes. For example, let's say I want to make all the fonts smaller except for the main copied text. I can just select the relevant code, hit Command K, and tell Cursor what I want to change.

Command I will open up the agent. If you select some code and hit Command I, it'll put that code in the context for the agent chat. This is useful when you want to chat about or modify specific parts of your code.

The main reason to use Command K instead of Command I is that it's faster. It knows we're only working with the selected code, so it's really quick. The inline UI is kind of nice too - it's simple and feels good to use.

Terminal AI with Command K

Command K in the terminal is another favorite of mine. I love using it to save keystrokes. For example, I can say "list my five most recent git branches" and it'll do that for me.

I don't bother memorizing complex git commands. I have more time and mental capacity to solve the things that matter most. Cursor just does what I want, and I love it.

Autocomplete and tab navigation

We all know about autocomplete, but let me show you something I find sometimes unintuitive in Cursor. After using this thing for over a year, I think I've figured out how to do it right most of the time.

See that gray text? If I hit tab again, it's going to auto-complete me over there. The confusing part about Cursor is this "tab again" feature is a little different. Sometimes it completes code, sometimes it just jumps to the next suggestion. You've got to get a feel for it.

Generating commit messages

You may not know that Cursor can generate commit messages for you. I always forget about this feature, but I really should use it more often. Just go over to the source control tab and hit the little magic wand icon.

It's not always perfect (see the clip below 😅), but it can give you a good starting point for your commit messages.

The bug finder

Let's talk about the bug finder feature. It's pretty cool. You can access it by hitting Command Shift P and typing "bug finder".

The bug finder compares your changes to the main branch and tries to identify potential bugs you've introduced. It's not perfect, but it can catch some issues you might have missed.

For example, it might catch things like forgetting to handle zero values properly, which can bite you especially when dealing with positions in UI work.

Balancing AI assistance and coding skills

A common question people ask is whether using AI tools like Cursor might lead to losing coding skills. It's a valid concern, but here's my take on it:

If we define coding skills as the ability to build good products in a good way with code (which is probably what most companies are hiring for), then I don't think a person needs to hand-code software all day, every day, to maintain or increase these skills.

AI can automate a lot of the basic work, which saves time and mental energy and allows you to focus on the harder problems. I can let AI, using Cursor's Composer, build out these UIs, and then when things get harder, I can jump in.

In my opinion, there's no world where you can just vibe code an entire product. I've never seen anyone do it, and I don't think anyone will ever be able to do it. These AI tools are getting better, but they're also plateauing in some areas.

We will always need to be fixing and manually implementing things in the code. If you don't know how to code at all, if you don't know how to debug, those are good skills to build. You will get stuck by coding and you'll need to unblock yourself.

I think AI helps a senior developer the most. The best thing anyone can do is learn as many development skills as they can, and let the AI help you, but you need to actually know how to work with this code.

As long as you're still working with code, you're still building coding skills. And most importantly, you're building productivity and efficiency skills. I would rather hire developers who use Cursor well than developers who refuse and hand-code everything.

I want developers who work as leanly and as efficiently as possible. I can't solve hard problems for eight hours a day, every day. But I can vibe code a UI for four hours and then work two hours on really hard problems by hand, and that gets a lot more done ultimately.

Conclusion

Overall, Cursor is an incredible tool. If you're just using autocomplete, you're probably missing out. Try the agent features and some of the other exciting features. Build familiarity with the types of workflows that work well, what works well with the agent, and what works well for testing.

Remember, it's about delivering a better product to customers that'll ultimately drive more revenue for the business. Use these AI tools to help you do that more efficiently, but don't forget the importance of your own coding skills and problem-solving abilities.

That's it for my best Cursor tips. Give them a try and see how they can improve your workflow.

Turn Figma Designs into Full Stack Apps Using Lovable and Builder.io

Steve Sewell — Wed, 22 Jan 2025 13:05:11 +0000

If you’ve ever turned Figma designs into a functional application, you know just how complex that process can be.

Every aspect of the design needs to become part of a working application—from mapping out component structures and handling responsive layouts to setting up data flows and interactive states.

The good news is that the manual back-and-forth between design and implementation is about to change dramatically. The Lovable and Builder.io integration lets you turn designs into full applications, so you can focus on what matters: building features and delivering great products.

What is Lovable?

Lovable is a full stack AI software engineer and editing environment. It's designed to let you create and iterate on your projects quickly, so you can move from an idea to a real application, deployed live, without wrangling complex tools or environments.

What is Visual Copilot?

Visual Copilot is Builder.io’s AI-powered Figma to code toolchain that leverages AI models to convert Figma designs into React, Vue, Svelte, Angular, HTML, etc code in real-time. It can even reuse your existing components in the output and use any styling library or system you need.

Check out our Product Hunt launch and share your thoughts on bridging the gap between design and code.

Convert designs to full stack apps

Lovable builds your front-end using React, Tailwind, and Vite, and lets you build full stack apps (to store data, and build with third party backends) using Supabase:

Create a Lovable account

Head over to lovable.dev and create an account.

Launch the Visual Copilot Figma plugin

In Figma, load the project you want to turn into code. Open the Actions menu from the toolbar, then click the Plugins & widgets tab. Search for “Builder.io AI-powered Figma to Code” and run it.

You can also try the plugin from the Figma community page directly.

Export to code

Select the design you want to convert. You’ll see a preview in the plugin. If you’re satisfied with your selection, click the “Export to Code” button. This starts the code generation process behind the scenes.

Connect to Builder.io

If you’re not logged in to Builder.io, you’ll be prompted to connect your account first. Otherwise, the plugin will jump straight into its AI-powered code generation. It might take a minute or two to process your design. That’s normal—it’s building out the code, setting up file structures, and making sure everything is wired correctly.

View the app in Lovable

When the process is complete, you’ll see a button to open the converted design in Lovable. Click it, and your app will open right in Lovable. You’ll see all the code generated from your Figma layers—tables, forms, even placeholder data in some cases.

Modify with Lovable

Now you’re in Lovable’s territory. If you want a table to pull in real data from Supabase, or you need a registration flow, just type what you want (“Hook this form up to my user database,” for instance), and Lovable will handle the rest.

Sync to GitHub

For seamless collaboration and hand-offs, connect Lovable to your GitHub account. This automatically syncs the code into your repository, supporting more advanced workflows where you need full version control and team collaboration.

Sync design changes to Lovable (coming soon)

Soon you'll be able to update your app in Lovable whenever designers make changes to their Figma designs. AI will handle merging the design changes while keeping all your custom code intact. A few clicks, and changes flow straight to production, no manual reconciliation needed.

Want to be among the first to try this feature? Join our waitlist and we'll notify you as soon as it's ready.

Conclusion

By pairing Lovable with Visual Copilot, you’re removing the most repetitive parts of front-end development and giving yourself (or your team) a major head start. No more hunting down font sizes and margins in a Figma file, or troubleshooting CSS that doesn’t match the original design.

Instead, you can generate working code in a few clicks, integrate it into Lovable, and immediately start adding real logic and data connections.

It's a workflow that frees you to focus on building features that delight your users—and shipping complete, functional applications faster than ever.

If you enjoyed this post, you might also like:

Windsurf vs Cursor: which is the better AI code editor?

Steve Sewell — Wed, 18 Dec 2024 12:15:01 +0000

Let's take a look at two popular AI-powered IDEs: Windsurf and Cursor. Both claim to accelerate your coding experience, but which one is better for you? I've deeply evaluated both, and here's what I've found.

If you're a fan of the Cursor Composer's new agent mode like I am, where you can chat without first specifying some files for context and it can automatically generate and run shell commands for you, then you've got Windsurf’s Cascade to thank (they did it first).

Cursor's new agent mode that can generate code across multiple files, run commands, and automatically figure out what context it needs (no need to manually add files explicitly).

Windsurf generally has a cleaner UI compared to Cursor's. It feels like comparing an Apple product to a Microsoft one — those little details really make Windsurf feel more refined.

Windsurf's Cascade feature — the original AI IDE agent that can automatically fill context, run commands, etc.

And, Windsurf is cheaper. While it starts at $15/seat, compared to Cursor's $20/seat, I'll say the pricing isn’t as clear cut as Cursor’s.

Things like "model flow action credits" — whatever the heck that means — are part of the pricing model. So, hopefully you won't run out of those. Fortunately, I didn't.

Overall, it seems worth diving in to compare these products in more depth.

Standard features

Both IDEs support the standard features you'd expect, like:

AI-driven auto-completions
Chatting with your codebase
Generating and updating multiple files at a time
Inline editing of code with AI

These have become the staple features of an AI-driven IDE, and both do a solid job at the fundamentals.

They’re both Claude

Both IDEs effectively have the same brain, as they're using Claude 3.5 Sonnet under the hood for the hard stuff.

I've noticed that functionally, I don't see a major difference in the quality of code generations or updates by either of these IDEs.

And I don't expect to — these are really just UIs on top of standard models.

When people try to compare and say "this code generated better than that one," it's my belief that they're mostly comparing the randomness of Claude outputs.

Both use smaller models for smaller edits and inline completions, but I similarly didn't find a major difference in the quality of one small model versus another.

Windsurf: simple and beginner-friendly

Windsurf really seems to push to be a very simple, easy-to-use product that's beginner-friendly and pushes for high-level, simple interactions with your code:

The default chat mode is the Agentic mode.
It indexes and pulls relevant code as needed.
It will run commands for you.
It won't clutter the UIs with buttons and code diffs everywhere.
You need to click the Open Diff button to see it in the full code pane.

This creates for a much better it-just-works experience. There aren't 100 features to learn with lots of different modes and tabs and options and sub options.

Just hop in, start writing what you want changed, and it'll change. It's easy, the UI is clean, and just overall feels really nice.

Cursor: More manual, more control

Cursor leans towards more manual control:

The Composer mode always defaults to normal (not agentic).
You need to choose what files you want to put in the context for it to generate.
It always shows you inline code diffs.
It seems to push the point that you should always be reviewing this code closely every time.

In general, Cursor feels like more of a power tool with more precise control and handling.

There is a steeper learning curve, and I find many people don't even know about Cursor's best features (unlike Windsurf), but once you learn it, you have every tool you need and more.

Windsurf keeps you better in the flow

One thing that Windsurf does by default that I really like is the AI generations are written to disk before you approve them.

That means you'll see the results right inside your dev server in real time. You can use that to also see if the generated UI is what you want, if the new code creates build errors, and so on —before accepting them.

If the updates still aren't quite right, you can chat again to continue to modify before you accept the changes entirely. And if you don't love the changes, even after some iteration, you can revert to a previous step or even discard everything in one go.

Compare that to Cursor — you need to accept changes before you can actually get the results. Sometimes that means you make changes that end up looking horrible.

Reverting them isn’t as easy, either. I need to figure out where a chat started (which sometimes could be midway through a long thread and hard to find), then manually revert to that prior state and time.

Cursor is deep with power features

Where Cursor really shines is in a lot of its power features. For instance, Cursor supports multi-tabbing.

If it detects that you made a change that would benefit from another change later in the code, you can just hit tab and tab again to keep applying those new changes.

I find this very cool when it works, but it can be kind of clunky and confusing when the changes don’t apply in the right spots.

Cursor generally takes a “kitchen sink” approach to AI in the IDE. Everything has an AI button:

See an error? There's a Fix with AI button.
Dropdown options? There's a Fix with AI button.
Error in the terminal? There's a Debug with AI button.
And so on...

Clicking these generally opens the chat so you can go back and forth with the AI to fix the issue.

I find these “kitchen sink” features useful, but they add clutter to the UI. For instance, I've had cases where these overlays are annoying and in the way, and I can't dismiss them when I need to.

Having an AI in the terminal that's a keyboard shortcut away at any point is really helpful if you're like me and like to focus more on what your code needs, rather than remembering exact command names or arguments.

One thing that annoys me, though, is how Cursor has specifically hijacked Command+K for this, which means I can't use it to clear the terminal. It doesn’t seem like there’s a way to override that or turn it off.

Both IDEs let you specify custom rules to inform their completions, but Cursor's support is a bit more robust and flexible. It includes the ability to have Notepads you can search for and include in your context as needed.

Generally speaking, Cursor is way more robust at context management. Both IDEs let you tag certain files you want to include in the context, but Cursor lets you add:

Whole doc sets
Specific pages from the web
Specific git branches and commits
Tagging @web to do a web search

One other really cool feature of Cursor is automatically generated commit messages with one click.

Even better, it respects your .cursorrules file. I found the default output to be more verbose than I like, so I added a note in my .cursorrules to keep commit messages shorter and voilà, perfect.

Another cool experimental feature of Cursor is the new bug finder.

After looking past an aggressive error message and clicking a button that has its own usage-based pricing (so may cost a dollar or a few just for the click) it'll scan through all the code changes on your feature branch versus main and look for potential bugs in your updates.

It's found some really useful bugs and saved me time having to find and fix them. For each bug it finds, it gives a confidence rating as well as an overview.

When you choose to fix it in Composer, it will fill the details into the prompt and generate updates like you're used to.

What's missing from both

The main thing I wish both products had was a more robust debugging loop. Something like what Devin promises (and sometimes can achieve), but right here in the IDE.

Both Windsurf and Cursor claim they have agents, but I am not sure if what they offer are truly agents.

I'm used to the idea of an agent being something that can try something, evaluate it, and repeat until it verifies that the result is correct. Which neither of them actually do. They only generate code, and if you've got bugs, it's your responsibility to go in and fix them.

Fortunately though, unlike the Zed editor, Windsurf and Cursor are forks of VS Code. This means you have access to the massive VS Code extension marketplace.

One extension available is Cline, which works great with either of these IDEs. Cline is more of an agent than Cursor or Windsurf have as it can take a series of steps, evaluate the result, fix its own issues, and continue.

Cline is surprisingly smart in some cases, like being able to run commands to validate its own correctness, open a browser to inspect what your dev server rendered, and so on. It's not perfect, but it's a solid step in the direction that I hope both Cursor and Windsurf ultimately go with their agents.

My favorite workflow

Speaking of compatibility with other tools, let's check out how you can turn a Figma design to code through Cursor or Windsurf and use AI to bring the design to life.

All you have to do is launch the Builder.io Figma plugin, select the element or frame you want to convert, click Generate, and copy the resulting command and paste it into the Cursor or Windsurf terminal.

Builder.io will generate pixel-perfect code from the design, even reusing your existing components just like you would.

Now that you have the markup I need, you can hop into Cascade or Cursor composer and prompt them to add whatever functionality you need like data, interactivity, and business logic, to name a few.

This is the end-to-end workflow I typically use: grab the design, have the AI add the logic, auto-generate a commit message, and fire up a PR. It truly feels like magic.

So, which should you use?

Honestly, I don't think you can go wrong with either Windsurf or Cursor.

Both are fantastic IDEs that support AI-driven auto-completion, inline editing, multi-file editing, chatting with your codebase, and agentic workflows where they can read and write to multiple files based on what you ask.

While I find Windsurf to be a little simpler, a little more intuitive, and with a slightly nicer and more polished UI, Cursor is just loaded with power features.

All those buttons everywhere can add clutter, but I find it hard to let go of some of these features I just love. I love typing Command+K in my terminal to generate commands when I forget what I need, I love the auto generated commit messages, and I even sometimes love using the multi-tabbing (when it works).

And while I'll suffer through some glitches in the UI, for a professional development tool, it's my personal preference.

For a more beginner developer, or someone who really prefers UI polish, Windsurf is probably a better go-to. It also has a lower starting point, and if you're coming off a tool like Bolt.new, Windsurf will feel a lot more like what you're used to.

But that's just what I think. What do you think?

We've launched a State of AI Tools survey for you to share what tools you find work best for you. Fill out the survey to receive the results once ready.

PS: I made that site with Cursor + Windsurf :)

Devin review: is it a better AI coding agent than Cursor?

Steve Sewell — Wed, 11 Dec 2024 20:42:26 +0000

I paid the $500 a month to use Devin, the AI coding agent, so you don't have to. Let's compare it to Cursor agents and see if it's worth the $2 billion valuation of their company.

What is Devin?

The main thing to know about Devin is it's primarily a Slack-based workflow — it's not an IDE.

You tag Devin in Slack and ask Devin to update something, fix something, et cetera. Devin includes:

A remote server
Browser interface
VS Code editing interface
A planner
A chat interface

You can follow along step-by-step to see what Devin did and what it's doing.

Testing Devin with an image generation model

I heard about this new image generation model that's supposed to be small enough to run on consumer-grade hardware.

I was hoping for a basic web UI, but then I found all of this and realized I don't code in Python. I didn't know what to do with this, so I asked Devin.

Devin went to work and in the course of about 12 minutes it:

Cloned the repo
Got it spun up
Generated an image of a cat for me
Attached the image back to me

I then asked for four more images of a dog riding in a hot air balloon. And I got my images in full, terrifying quality.

Now that's not Devin's fault, of course; that's the model we're using.

Asking Devin to create a web UI

I saw one of the todos on this repo is to create a local, real-time interactive app.

So I asked Devin if he could clone this repo and add a web-based UI to type prompts and see images. Devin began spinning things up and sending me updates.

One really interesting thing Devin does is it takes notes and stores them in a notes.txt file to refer back to and use in subsequent prompts.

This seems like an interesting technique to summarize information that's important and carry it across subsequent steps.

Devin will also sometimes create knowledge entries, which are like bits of information that could be useful to refer back to in totally subsequent runs.

It'll store these and look them up when needed, which is supposed to emulate the tribal knowledge that exists within a team.

Devin was in fact able to add the web UI I asked for, but we hit other issues. More on that in a moment.

Devin's capabilities and limitations

Overall, Devin's pretty impressive. It:

Creates plans
Writes code
Finds bugs in the code
Corrects the code
Runs its own end-to-end tests to verify it works
Responds to your feedback if you find issues and attempts to address them

Anything you reply in Slack, Devin will start working on a reply to. In this case, it was able to verify we're hitting deployment issues.

I kept working on debugging it, but unfortunately, after a lot of back and forth, it still never was able to solve it. Eventually, I gave up because I was sick of trying.

Personally, slack thread hell is not my favorite method of developing/debugging:

I prefer not to be demoted to the “any updates?” guy.

Finally, I asked if I could just pull this code down locally and run it locally. It gave me instructions, but they weren't valid because it didn't actually send this code in a pull request.

Devin's pull request capabilities

That's not to say that Devin can't do a pull request. One of my very first runs of Devin was to add a feature to a weather app.

It was able to add the feature I wanted as well as respond to my feedback that I wanted it to look more like iOS styling.

The final pull request was not bad. It added two packages, and the code was pretty good, but there was a console log in the code.

It also forgot to uninstall a package that it no longer needed after my feedback. But we can go in and just leave comments, like a normal person, to remove this log and that this package is no longer needed.

One cool thing Devin did when we were going back and forth on what the UI of this weather app update should be is it actually generated a deployment with a preview URL without me asking.

So when I type in a city, I can see that the feature I wanted has an iOS style like I asked. Even though I actually don't have a deploy preview set up on this repo, it deployed a version for me to see anyway.

When it learned I want an iOS style for this app, it proposed this to save in the knowledge. I can review and approve it, and it'll remember that during subsequent runs.

For some reason though, I couldn't get Devin to reply to my feedback this time, even though I've seen it do it before. I don't know what went wrong.

In general, I hit a few bugs along the way while using Devin, but nothing super crazy that I couldn't usually work around.

Fixing bugs with Devin

A separate task I asked of Devin was to fix a bug in our existing website. After about 12 minutes, it spun up a PR with a fix, finding the necessary boolean and updating it from true to false.

But then it updated some other stuff I didn't expect:

Added a fallback: true in getStaticPaths, even though getBuilderStaticPaths already sets fallback to 'blocking'
Removed a check, even though we already turned that value to false
Added a type declaration that I know firsthand isn't needed

The cool part is I asked in the PR why it did this, and Devin added the eyes emoji to tell me it sees this. Then it explained itself.

I'll be honest, I was kind of hoping it would fix those things. But it did provide a thorough explanation. It just wasn't a good one. Most of this information is not actually true.

Fallback true does not enable client-side navigation or enable Builder.io's preview system. Fallback blocking, which was already used, is our preference. Also, the tabler icons react type definition is just not needed. It's included in the package.

It made some weird comment that these components are part of the client-side navigation system, whatever that means. But the nice part is I can talk to Devin like a human, leave a comment, and it can make updates accordingly.

Adding a backend feature

The last thing I asked Devin to do is implement a backend feature. I said to add to our GraphQL admin API the ability to read and write from the comments collection.

Devin created a PR that was decent. It added this reflect metadata package that I don't think is needed (we haven't needed it to date).

But most importantly, it did recognize we use this resolver structure. It created a comments resolver and added it.

This code actually looks pretty typical of how we've written this on the backend. Now it did make up a couple of fields that would have been nice to ask me what the schema is. But otherwise, I'd say this is decent code.

Workflow issues with Devin

Overall, I'd say the biggest problem I have with Devin is this is just not my preferred workflow. I don't want to make an ask and wait 15 minutes for a pull request, and then have this back and forth on the pull request and/or Slack.

I much prefer Cursor's workflow where I have all of this right in my local environment and IDE. I can see the updates in real-time and can commit and debug locally, without jumping to some remote server and other set of tools I don't know, and having all these long waits and delays that are just unfamiliar and unproductive.

I get that the idea of Devin is to set some asynchronous agent coworkers off at a task and let them do lots of things in parallel and just come to you with results.

But that really isn't a great workflow until Devins are a lot better. I don't want the AI to just go off and do its thing and come back only when it's done, unless I have high confidence it's going to be really, really reliable at that.

Otherwise, I'd prefer my IDE just do it.

Comparing with Cursor agents

Let's compare some of the same tasks with Cursor's new agent features.

Context handling

The big difference between Cursor agents and the standard Cursor composer view is you don't have to manually add files to the context. Cursor will scan your codebase and find the relevant files and add them for you.

Cursor was able to find this no client-side routing variable and flip it to false. If I accept the updates, we can see it did exactly what we wanted. One basic minimal diff.

User control and feedback

Cursor's not always perfect, but the part I like most is I'm in control and in the driver's seat. If I want something different, I could also say, just delete that variable and all references altogether. And I could see the update immediately. There's less waiting and more action.

While I'm more closely in the loop, I have more trust with this process. Because I know what I want, and if it can scan my code, update multiple files and not make me have to worry about the details, I can provide real-time feedback and hand modifications and send the pull request my way.

That's a much easier to adopt workflow for me and my team.

Ownership clarity

With Cursor it's also more clear who owns the pull request: it's me. I find this process faster, easier, and nicer. We don't have weird bots creating pull requests where it's unclear who actually owns that and is responsible for making sure the code is good.

Nobody has to clone down that bot's PR and push updates to it. And every update happens pretty quickly.

GraphQL example

I also tried the GraphQL prompt with our very large internal repo in Cursor agent mode as well. And I got very similar results. It:

Added the comments resolver
Integrated it into the API
Added the types as well

So pretty similar results to what you'd expect with Cursor's composer view. But again, because of the agent mode, I didn't have to specify files. I just typed my prompt and it happened. That was nice.

Image generator repo clone

Now let's try a more agentic workflow where we have it clone this image generator model repo. You'll see the main difference between Cursor agents and Devin is it asked me before it runs commands.

Cursor is generally more cautious than Devin, which is nice because it's running on my local machine. But also sometimes I wish it would just run this stuff for me.

I've noticed if it catches an error, it'll automatically try to fix it, which I've seen it be successful at, which is great. Now it's written the code, which I'll accept. It found an error and it's rewriting the command accordingly.

Unfortunately, my computer froze before I could show you if Cursor was able to finish that task. It looked to me like it was generating the image fine, but it turns out that model is meant for having a real GPU and not burning through my laptop CPU like I was trying to do.

The results

Overall, I don't think Devin will take off like Cursor. And it's not just because of the $500 a month starting point. Cursor is so much easier to adopt and I like their incremental approach.

Devin, I fear, is trying to jump too far. They've raised all this money saying there's this all-new way to build software with agents, but it just wasn't my preferred workflow.

Maybe one day when LLMs are even better and agents are extremely reliable. But I'm not sure the rate of progress will get us there really soon. And I personally believe more in Cursor's incremental approach than Devin's "let's change everything" approach.

My preferred AI stack

My preferred workflow looks more like this:

A developer works iteratively with Cursor
Other teammates like designers iterate with their tools
Products like builder.io can convert designs to code and also patch in design updates as they're needed

Ultimately your workflow doesn't change much. You're still coding and debugging locally. You're pushing changes as needed.

But I will say that I'm excited to have a new player in the agent coding space to push Cursor even further. And I can't wait to see what comes out from the result of this.

But that's my quick take. From everything you saw, what do you think? Let me know in the comments. And if you made it to the end and you want to see more videos like this, be sure to like and subscribe.

Is OpenAI's o1 model a breakthrough or a bust?

Steve Sewell — Tue, 10 Dec 2024 13:20:47 +0000

OpenAI's latest model, o1, isn't getting nearly as much buzz as its predecessors GPT-3 and GPT-4. Let's dive into why that is and what it means for the future of AI.

AI progress is slowing down

Let's start with a real-world example. Here's a comparison between o1 and Claude converting a Figma design into code via Builder.io:

See full clip and details below

As you can see, o1 is significantly slower. It's also more expensive, and only sometimes better.

So why is OpenAI investing so heavily in o1? It all comes down to this:

Source: Alex Albert

Each new large language model (LLM) that comes out seems to be only incrementally better than the last. In fact, it's expected that OpenA/I's latest model, Orion, isn't even always better than GPT-4.

Why? We're simply running out of data to train on.

Source: Will we run out of data? Limits of LLM scaling based on human-generated data

The need for speed

If we're hitting limits on how smart we can make AI models, what's the next move? Making them faster.

AI players are investing in specialized hardware to speed up inference and lower costs. They're building their own data centers and even looking into nuclear power for more sustainable, cheaper energy.

Companies like Groq and Cerebras have seen up to 10x performance increases with LLM-optimized hardware. This isn't just theoretical - Amazon's already released their new chips, and Apple is planning to use them.

Faster inference could open up new workflows that weren't feasible before due to long wait times and poor user experience.

The Smartness vs Speed trade-off

But here's the million-dollar question: does being faster matter if AI can't get any smarter? Or to put it another way:

If the way we train models today isn't getting better, if AI is plateauing on intelligence, can we use increased speed and decreased cost to find another path to smarter AI outputs?

The answer might surprise you.

Let's borrow a concept from Daniel Kahneman. He talks about two systems of thinking:

System 1: Fast, automatic thinking. Like knowing 3 + 4 = 7 without having to calculate it.
System 2: Slower, more deliberate thinking. Like solving a complex math problem step by step.

System 1 thinking is automatic.

System 2 thinking requires taking things step-by-step.

Using System 2 thinking is used to break down complicated problems into simpler steps.

For example, you likely don’t have the answer to 39 + 48 memorized the same way as you know the answer to 3 + 4.

Instead, you’d need to break the more complicated question down into steps like this:

Current LLMs work a lot like System 1 thinking. They give you an answer fast, but as complexity increases, accuracy can suffer.

O1, on the other hand, is more like System 2 thinking. It breaks down complex problems into smaller, manageable steps. This approach can help with one of LLMs' biggest weaknesses: hallucinations.

For example, if you ask o1 how many Rs are in "strawberry", it might first guess incorrectly. But then it'll go through the word letter by letter, count the Rs, and give you the right answer.

This step-by-step approach isn't entirely novel. Chain-of-thought techniques have been around for a while.

In fact, open-source models, like the QwQ model from Alibaba, are already beginning to use this approach with similar performance. What's new is training a model to use this approach specifically.

Source: Alibaba

As speeds increase and costs decrease, we might be able to afford this extra time to get better answers without hurting the user experience.

The downsides of o1

The problem is, o1 doesn't always give a better answer to every type of problem. But it is always more expensive and slower.

Source: Aider LLM Leaderhboards

Currently, o1 Preview costs four times more per token than Claude Sonnet. It also outputs 2-10 times more tokens because of all the "thinking" it does.

This means an o1 output could cost up to 40 times more than Sonnet - and it's not always better.

The slowness is a big issue too. You often don't see any results from o1 for 10, 20, 30 seconds or more. That's a much worse user experience.

Conversion above done with Builder.io using claude-3-5-sonnet-20241022 and o1-preview-2024-09-12 (latest API-accessible model versions). I hand checked the results on ChatGPT with the latest o1 and didn't see any major differences in speed or accuracy.

The product in the video above costs $20 per month now. If the o1 model costs up to 40 times more, it doesn’t seem worth it to pay $800 a month for an offering that is slower and only marginally better, if at all.

The potential of AI agents

So why is this still interesting? AI agents.

We want to use AI to complete a series of tasks without constant human supervision.

For that to work, AI needs to be better at breaking things down and completing tasks step by step. It also needs to have a lower failure rate and catch its own mistakes sooner.

Traditional LLMs are great at completing the next word in a sentence, but they weren't trained to break down and execute tasks.

For instance, Claude’s computer use today only has a 15% success rate at accomplishing real-world tasks. O1 is showing us what happens when we do train for that.

Source: Anthropic

The big questions

How much better can this get with novel training methods? Will this lead to new breakthroughs, or are we heading for an AI bubble burst?

The average person's life hasn't changed much despite all the AI hype. If AI models aren't getting that much better, and these new techniques don't lead to major breakthroughs, some of the hype might start to cool off.

And you know what? That might not be the worst thing.

Lessons from the dot-com bubble

As the dot-com bubble peaked in 2000, people invested in the web as if its major challenges had already been solved. There was widespread excitement about the web's potential for explosive growth and profitability.

However, that excitement deflated when companies burned through hundreds of millions of venture capital and failed to become profitable. Only a few companies survived, and they now serve as models for how online businesses can succeed.

Today, we're seeing a similar pattern with AI—growing excitement and venture capital flowing freely. The enthusiasm mirrors what we saw during the dot-com bubble's growth.

But there will be hard problems. Progress will come in S-curves. While AI could solve an immense number of problems, it might not happen today. Some AI applications might not be effective enough for mass adoption for another decade.

Remember Webvan and Pets.com? They were viable internet businesses - just 10 years too early.

Before you hit me with the "but this time it's different, man" – don't forget that that's what people say every time.

What's working now

While the future is uncertain, some AI use cases are already working well:

AI chat assistants for brainstorming, researching, writing, and editing
AI-assisted coding, whether through specialized IDEs, Copilot, full application builders, or tools that convert designs into high-quality code

These categories are seeing major adoption with generally happy users.

The bottom line

Maybe AI agents will change everything as soon as next year. Or maybe it'll take longer. We just don't know yet.

What we do know is that some AI applications are already proving their worth. Those are the ones I'm watching closely, and I'm excited to see what comes next.

Solved: Why ChatGPT Won't Say "Brian Hood" (Blame Regexes)

Steve Sewell — Tue, 10 Dec 2024 13:03:27 +0000

We've figured out why ChatGPT refuses to say certain people's names, no matter what. And it's for the best engineering reason ever: regexes.

Yep, you heard that right. ChatGPT's internal code has regexes looking for specific names and throwing an error anytime it sees them coming out of ChatGPT directly.

The oldest security measure in the book

What's the oldest and best security measure? Regexes, of course. But why are they even doing this?

It turns out these are people who tried to sue OpenAI. So how can you get in any more trouble with ChatGPT making up lies about people? Simple: just never allow ChatGPT to mention that person's name. Ever. No matter what.

But isn't that terrible engineering?

Sure, you might say, "Wow, what terrible engineering. How disgusting." But really? What are they going to do instead?

Are they going to retrain a new model for millions of dollars just to not mention these few random people's names? And do that every time they're worried about lawsuits?

Or if someone's mounting a suit against you, are you going to let yourself continue to be liable to the same thing over and over?

Sometimes simple is best

It's easy to make fun of this approach, but it's such a simple, easy-to-leverage solution that they did in a pinch. Because it was a good idea.

Fun fact: the actual Builder.io codebase for our Figma to code product does literally the same thing. Yes, we have advanced machine learning, malicious content detection, yada yada.

But sometimes when we see a very clear pattern that we need to just stop immediately, we just add it to an array of regexes and disable any content that matches. It's fast. It's cheap. It's effective.

Y'all, you don't have to over-engineer every single problem.

How strict are these regexes?

The regexes are pretty strict. If you had a dot in the name, if you had the person's middle initial or middle name in there, it's fine. It'll output it. But just first name, space, last name? No chance.

And the same goes for other names like this.

The power of regex

While AI hallucination is no new thing, and it's probably the number one problem that LLMs have had forever and probably will have for some time, at least we know when you can't fix it with the world's most advanced AI, you can always fix it with a regex.

Don't let anyone tell you in your pull requests that a regex is no good or a bad practice. The biggest, best, most innovative companies do it. And you should do it too.

Why AI Is Making Dev Skills More Valuable, Not Less

Steve Sewell — Wed, 04 Dec 2024 13:27:22 +0000

This chart is the reason that AI is not making developers obsolete anytime soon. In fact, it shows how development skills are more valuable than ever.

This isn't a phenomenon of just one product; it affects every product that gives non-developers some degree of ability to code.

The way AI model progress has been plateauing lately suggests this may not change any time soon, if ever.

The pit of death

Anyone who's not a developer but has used tools like Bolt, Cursor, or v0 to build basic applications from prompts knows all too well the pit of death that you eventually always fall into.

Things start fast and awesome: the basic scaffold of your page or site, or even some of the basic functionality, comes in seconds. But then you start prompting more and more, and you struggle to get what you want out of it.

As the application gets more complex, the behaviors you're looking for don't come out quite as easily.

Eventually, some prompt you use breaks everything. Nothing works, and no matter what you try, the AI cannot get out of this corner.

That's the pit of death. I've seen it happen over and over: some esoteric bug is just too challenging for the LLM to solve on its own, and then that's it. Your app is done, if you're lucky enough to not land at a pit of death.

The plateau of death

You'll probably hit a plateau of death instead, where after a certain number of prompts, you just don't seem to be able to make progress anymore.

There's truly only one fix for these situations: you've got to roll up your sleeves, get in the code, and fix whatever the AI couldn't.

Whether that means solving some very esoteric bug or implementing features so that the AI can continue off of your implementation, there's always a point where the human is needed.

The limits of AI in development

Even if the models got 10x better, I don't think you'll actually offset the development need by that much.

The reality is LLMs are good at code generation, but that's only one part of many in software development. In my experience, when an engineer comes in to unblock the AI, the next pit of death is just right around the corner.

As the application gets more complex, you need developer intervention more and more, in shorter and shorter intervals.

Eventually, what ends up happening is you go from products that don't require any development to ultimately exporting that code and needing developers to fully take it over. The developers then use different tools that are similar but more focused on code editing.

The value of no-coding-skills-needed AI coding tools

That's not to say that the tools that didn't require development in the beginning aren't useful. They can be incredibly useful for things like prototyping, making simple internal tools, and micro applications like a pricing calculator.

Making your life better with AI tools

The big question is how you use these things to make your lives better.

As an engineer, designer, product manager, marketer, or whatever your role is, I think there are a few heavily overlooked use cases that you could adopt in your workflows now to make your life a lot better by using these tools and leaning into where they're going.

The pain points we can improve

As an engineer myself, the two things that drive me the most insane are poorly written specifications (where we didn't fully understand the problem and what's really needed to solve it) and ever-changing requirements.

It turns out our original guesses were wrong, customers and feedback told us different things, and now halfway through building the thing, we're changing how we're doing it, and everything's becoming a mess.

The value of hands-on testing

In my experience, you have to get hands-on with an application to really know how it needs to be used. Even better, get it in real users' hands, even if it's not fully functional.

Just having users go through the actual flows and testing the things that looked good in wireframes and designs, but maybe actually don't feel right in practice, can really help improve your product UX.

But if you have to build the whole thing before you learn that, it's wildly inefficient and can lead to very messy codebases.

Rapid prototyping with AI

Alternatively, with these rapid prototyping AI tools, we can iterate before we actually commit to building the full thing.

We can try out the flows, even if all functionality isn't there.

If we're lucky, we can even get direct user feedback in small ways and learn as much as possible early before we commit to a finalized spec and building the real full application.

Less handwritten CSS

There are other implications coming soon too. While I don't think programming is going away at all, I think it will change. I think every year we're going to see a major reduction in actually handwritten CSS code.

The same way you don't handwrite a PDF, you export it from a tool. Design and design-to-code tools are becoming so good that they can generate CSS for you, shockingly close to how you would have written it yourself.

In doing that, we can stay exactly true to the original design spec and avoid a lot of red lining and back and forth to make sure it looks correct.

A more collaborative future

Looking to the future, the big question is: How do we get more tedious development off of our plates? So that developers don't need to update colors, add a simple section, modify text, or tweak some logic.

If marketing or product or design need small changes, how can we give them tooling like they'd have for a fresh new app with something like Bolt, but compatible with a real application with real users and real developers that work on it daily?

AI in team workflows

Luckily, I work on a team that builds one of these products, so I can show you what's coming soon, specifically around how AI can better fold into full team workflows and automate a lot more than just an initial prototype creation.

Code sync

The first interesting feature is something called code sync. This is where you can not just convert a design to code with AI, but you can then modify the code, and when the design updates again, there's a link back to the code.

So the code gets granularly updated with just the design change, without developers having to do that by hand at all.

Let's say AI converted the design to code, and you added all kinds of logic and modifications. Now the design has new colors, new buttons, modified text, and tweaked styles.

With code sync, you can just push the changes right back to your codebase with the most granular update possible and review and merge it. This is currently in private beta, and we're hoping to roll it out soon.

AI-integrated CMS

Another interesting and under-leveraged approach that already exists today is using a deeply AI-integrated CMS. In the case of Builder.io, you can section off parts of your site or app and decide what regions can be controlled by the CMS.

These regions can be assigned to specific teams, like the homepage body goes to the marketing team, hero section goes to a merchandising team, etc.

Then you can use generative AI, just like you would with Bolt, Cursor, v0, or whatever, but the updates go directly to the site or app published over an API.

Think of it as Bolt embedded in your site or app, with whatever rules and restrictions you want around it.

This unlocks not just crazy generative AI use cases within your live sites and apps, but opens up other really interesting use cases.

AI agents in CMS

One use case we're exploring is integrating agents into this AI CMS structure. Imagine having a Slack bot where you might have a conversation about what you'd like to change in your homepage or ideas on things you could do differently.

You just tag the Builder.io agent in Slack, and it reads the conversation, applies the suggestions as a five or 10% test, and then comes back a week later to let you know how the test performed.

In this world, for the self-contained regions that are probably more marketing-focused and things you don't really need or want to have in code anyway, the agent could just make those updates end-to-end autonomously, do the tedious work, and just let you know what the results are each time.

Auto-optimization with AI

We can even take that one step further and let the AI auto-optimize things. In a CMS like Builder.io, where all content is also tied to analytics, when you produce content through the CMS, the AI agent can automatically start creating A/B test variations.

It leverages context that it already knows about your business, finds what variant converts best, and scales that up to a hundred percent of traffic completely autonomously.

It can also take your audiences, create personalized versions of content for each audience, and play with the knobs to make sure every piece of content is as highly converting as possible, in a 100% automatic way.

Real-time personalization

If you take that one step further, you can see we're not that far away from a world where we can do one-to-one personalization where the AI is generating parts of the user journey in real-time as they're browsing.

As you browse the site or app, or I do, we can see completely different flows, optimized for exactly us, generated by LLMs in real-time.

This one's probably the furthest because we have to worry about cost and latency. But as more LLM-optimized hardware comes out, and we're seeing up to five times gains in speed and efficiency of inference, this might not actually be that far away.

Conclusion

While I don't think everyone will be a developer overnight because of AI, I do think the more development skills you have, the more you can use AI to generate code, unblock yourself, and do incredible things.

I think the biggest unlock that's coming is better connecting the AI tools with your existing teams and workflows. This will enable us to build and iterate on real applications faster than traditional methods, maximizing the skills of everyone.

We'll all build better products faster together, focus on our unique skills and creativity, and spend less time on redundant menial work.

Oh and if you found any of what we are working on around using AI to accelerate design to code and/or publishing to your live site, talk to us to get a demo from our team.

The Big Lie AI Vendors Keep Telling You

Steve Sewell — Tue, 03 Dec 2024 11:39:19 +0000

OpenAI and other big AI vendors keep pumping one huge lie, even though all the data suggests they're definitely wrong. Sam Altman loves to preach that Artificial General Intelligence (AGI) is right around the corner.

"We're about to give you AGI," he claims, even despite key leadership roles leaving the company. In my opinion, if you really were right around the corner from AGI, I wouldn't be leaving.

At the same time, their latest model just looks like GPT-4 with a chain of thought prompt, which is already a common technique, and just a really long output window (not to mention how incredibly slow it is).

The truth about AI model progress

It seems like OpenAI is noticing the truth here, which is that new models are plateauing in terms of how effective they are compared to older ones.

So instead of shipping a GPT-5 that's probably only marginally better and probably larger and more costly, they're trying to make GPT-4 work better for different use cases.

At the same time, Anthropic consistently produces better models. But even those are only incrementally better each time they launch one. This is a big gap from the GPT-2 to 3 to 3.5 days, where we saw huge leaps each time. We're just not seeing that anymore.

Source: Business Insider

Three interesting implications for the AI ecosystem

This leads to three really interesting implications of where the AI ecosystem is going, and one really big unknown that I also think people are really, really lying about.

1. Vendors are specializing

OpenAI's Sam Altman used to tout "good luck" if you try and compete with their models, but they simply don't have the best models anymore, and haven't for a while. It seems they're acknowledging this and keep focusing time on specializing in consumer products built around the models.

They make three-quarters of their revenue from ChatGPT, which they're investing in better native apps, better experience, better search — all the consumer experience around the model. They're conceding that they just don't build the best models themselves anymore.

In fact, if you're a vendor using OpenAI's APIs, in almost all cases you probably should use Anthropic, which is reflected in their revenue — 85% of their revenue comes from the API.

They invest far less in their consumer products, even though the model is better. The user experience of the products just isn't as good. They are clearly not investing there. They're putting everything into just being a few percent better of a model.

Source: Anthropic

2. Specialized hardware is coming

More and more vendors are building specialized hardware for LLMs which can drastically increase the speed of inference.

New chip companies like Groq and Cerebras are able to do LLM inference three to five times faster than typical GPUs.

Source: Cerebras (evaluation of Llama 3.1 - 8B)

The interesting key here is they're not just five times faster. The can also evaluate three to five times cheaper because they're simply much more efficient.

So when we can make AI products that are equally as smart but massively faster and cheaper, that could unlock a lot of new use cases that were previously just prohibitively slow and expensive.

3. Better utilization of LLMs is coming

The third implication is that vendors are going to get a lot better at figuring out how to use LLMs, even if they don't get that much smarter.

Below is an example of a launch title for the Xbox 360 - Perfect Dark Zero, which came out when the 360 was brand new and the graphics are…fine. Like, it's representative of that era of the console.

But what's fascinating is when you remember launch titles versus late in a console series titles. On the same exact foundation, there was no change to the console or hardware itself, but years later we see games like this.

This is Halo 4, and this looks like a whole different generation of console. Yet, this is on the same hardware as the prior game.

What happened over those years? Developers got way better at optimizing how they use this technology. Game engines found more and more clever approaches to get the most out of what they had.

This is what I think will happen with LLMs. Those who make products on top of LLMs will get better and better at getting more of the smart parts out of them and avoiding the suboptimal, problematic parts.

I don't think we're anywhere close to AGI, but I do think we'll see major leaps in product quality, just by using the LLM technology better and better.

The reality of AI agents

What about agents? What about Devin, the AI software engineer that made all these crazy claims, raised a ton of VC money, and just kind of disappeared off the radar? Or Anthropic's Claude with computer use that will navigate your computer and use it like you?

Realistically, I think the models are just not smart enough yet for this to make sense. Even in Anthropic's own announcement post, they cited that Claude with computer use only has a 14.9% success rate. I don't see any world where we give AI agents the ability to act on our behalf without them having something more like a 99— if not 100% — success rate.

Source: Anthropic

In all of my experience and testing, the models are not smart enough to be able to accomplish autonomous tasks. They need a human in the loop. Otherwise, they derail quickly and those small errors compound very rapidly.

The future of AI products

But that's not to say that some form of agents won't exist in the short term. I think mini or micro agents do show a lot of practical value. For example, the open-source Micro Agent project that can generate tests, take a set of tests, and write and iterate on code until the tests pass.

Or in the case of Builder.io and our product that uses AI to turn Figma designs into code, we've actually found that rather than using one big traditional LLM, we can use a series of smaller agents that communicate with each other to have a way higher quality output when you convert that Figma design into code.

By using a series of smaller agents, in combination with self-trained models, we found that you can both increase reliability, speed, and accuracy, compared to just plugging in something off the shelf.

These are the types of techniques that I think will bring us to the next generation of our era. Even if the LLMs don't get smarter and we don't have personal coding assistants doing all our work tomorrow, we sure as heck will keep getting better and better products that are more reliable.

We’ll automate tedious work — like converting designs to code, using generative AI to produce designs and functionality, adding functionality to your designs, making them real-world applications — or develop other kinds of interesting use cases around content generation, code generation, co-pilots, and other interesting products Builder.io keeps producing.

Source: Visual Copilot 2.0

Just be sure to pay a keen attention to which ones people are actually using, versus what are gathering hype but actually don't work. Both will keep happening. I'm certain of that. I don't think we're anywhere near being past the hype curve of BS products intermixed with real effective products.So while I think there'll be a lot of noise in the market, I think we have a lot to look forward to as well.

Generate Figma Designs with AI

Steve Sewell — Mon, 25 Nov 2024 18:04:13 +0000

"Design me a landing page for a fintech app.”

A few years ago, this would have kicked off hours of creative exploration in Figma. Today, it's a prompt that can generate complete UI designs in seconds.

This is a big deal for designers, but it got me thinking — what happens after you've got that slick AI-generated mockup?

Figma AI

Figma AI is a suite of artificial intelligence features integrated into Figma and FigJam. The idea? Make design workflow easier for UI and UX designers by helping out with both creative and routine design tasks.

Figma AI plugins

Before Figma built their own AI features, designers were already experimenting with AI through figma plugins. Now that it's built right in, let's check out what Figma AI can actually do.

What can Figma AI do?

As a designer, you spend a lot of time in your design process — organizing design elements, finding the right design assets, and putting it all together. Figma's new AI toolkit is still in beta, but here's what it can do:

First draft: Type in a description and get a full design to work with. It's great for rapid iteration and exploring ideas.
Smart duplication: Duplicate elements with AI-generated content variations. Perfect for creating lists and grids quickly.
Text improvements: Generate relevant text that fits your product's tone instead of using Lorem Ipsum. Rewrite and translate text directly in your designs.
Visual search: Upload an image to find matches or similar designs in your team's work. Makes finding assets a lot easier.
Smart layer renaming: It can rename and organize layers automatically. Your teammates might appreciate this one.
Background removal: Take out image backgrounds with a single click. No need for manual selection.
Instant prototyping: Turn static designs into interactive prototypes automatically. Saves time on manual connections.

How to use Figma AI

The AI features are easily accessible through the action button at the bottom of your Figma interface or by pressing ⌘ + k / Ctrl + k. Here's how to use each key feature:

First draft

Select First Draft from the templates (Basic App, App Wireframe, Basic Site, or Site Wireframe). Enter your prompt, like "e-commerce store for iPhone accessories with a search bar." Once generated, use quick actions to switch themes, adjust spacing, modify border radius, or make additional prompt-based changes.

Smart duplication

Create an auto-layout list or grid, then simply drag the handle at the bottom. The AI will automatically generate new content variations while maintaining design consistency.

Text improvements

Select any text element to access a range of AI-powered enhancements, including translation to different languages, tone adjustment (professional to casual), and length modification.

Visual search

Type what you're looking for or upload an image through the action menu. AI will find matching elements visually, regardless of naming, and help navigate to the found components.

Smart layer renaming

Select the layers you want to rename, choose "Rename Layers" from the action menu, and AI will automatically rename them based on their content and context.

Background removal

Select any image and choose "Remove Background" from the action menu. Within 30 seconds, you'll get a crisp, clean cut-out with impressive detail preservation.

Instant prototyping

Select up to 10 screens and choose "Make Prototype" for AI to automatically create basic navigation connections. You can then review and refine these connections as needed.

Current limitations of Figma AI

While Figma's new AI powered features are impressive, there are two important considerations:

Limited availability: The beta is currently restricted to a small subset of users, making it challenging for teams to fully integrate these features into their workflows.
The design-to-code gap: Figma AI is great for cranking out designs, but there's still a big gap between a great looking mockup and responsive code. This is where things usually slow down - designers hand off their work, and developers have to figure out how to turn it into a real website or app.

This is exactly why we built Visual Copilot at Builder.io — to bridge this gap between design and implementation. Instead of just generating static mockups that need to be rebuilt in code, what if we could generate functional, production-ready components from the start?

Improved design generation with Visual Copilot

Our approach is pretty straightforward: since designs need to become code anyway, why not do both at the same time?

When you bring your Figma designs into our Visual Development Platform, you can:

Keep generating designs with AI, just like in Figma
Generate code in real-time as you make changes
Publish your work to production with one click when you're happy with it

Whether you're working on simple layouts or complex interfaces, Visual Copilot maintains your team's focus and style — from brand colors to fonts. The boost in development speed means you can iterate faster and get to your final product sooner.

Generate designs using your design system

Generate designs using natural language prompts, just like in Figma
Iterate on designs with AI-powered suggestions and variations
See both the visual preview and corresponding code in real-time

This approach means you're not just creating static mockups — you're generating production-ready components that can go live immediately. No more lengthy handoff process between design and development teams.

Complete context of your product

Visual Copilot isn't just another AI demo generator. Its power lies in its ability to actually understand your product:

It knows your Figma components, design tokens and documentation
It knows your code components, coding standards, and development patterns
It can connect to your APIs and data models, and business logic

This means describing complex features in just a few words gets you exactly what your team would build — using your design system, following your standards, connected to your services.

Static designs to interactive code

In Builder.io, your designs come to life:

Make stuff interactive
Use your actual data
Edit things live and see changes instantly
Deploy updates automatically through our headless CMS

Generate production-ready code

Visual Copilot turns your designs into code you can actually use, giving you dev ready websites that just work:

Pick your favorite framework and styling setup
Skip the prototyping phase and go straight to live previews
It'll even make things responsive for mobile

Getting started with Visual Copilot

Using Builder.io's Visual Copilot plugin is straightforward and immediately accessible:

Open the Visual Copilot Figma plugin.
Select a layer or design in your Figma file.
Click the Export to Code button.
Copy the generated code into your project.
Add animations, state, visual effects, and other required functionality with natural language prompts.

Figma AI is a neat step forward, but we need more than just static designs to build modern websites. Visual Copilot bridges that gap, letting you create designs and functional code at the same time.

Turn a Figma Design Into a Working Dashboard App in 5 Minutes

Steve Sewell — Wed, 13 Nov 2024 13:39:28 +0000

I just turned a Figma design into a working dashboard app with real APIs, data, and pagination in less than five minutes.

And the best part is: I can export my creation as clean code across multiple files, including reusing your existing components in your codebase and/or design system.

React, Vue, Tailwind, SwiftUI, and many more outputs are supported, and everything is customizable.

Let me show you how to do it.

The process

Step 1: exporting from Figma

First, in Figma, launch the Builder.io Figma plugin in a design of your choice, for instance with the dashboard design I used.

Note: If you use the dashboard design file I did, you will likely have to duplicate it to your drafts first so you have full access:

Note: This plugin works the same in Dev mode too.

Once you have the plugin launched, select the Dashboard frame and click the Export to Code button, and you'll be launched into Builder.

If you don’t have a Builder.io account yet, you will be asked to sign up, which only takes 2 quick steps.

Note: If you use the same Dashboard design as I did, you will see a “Low auto layout usage” warning. Using auto layout for exported designs is a best practice, and without it your results might vary. In this case, this design imports pretty well without auto layout, so we can ignore it.

Step 2: adding context for the AI

Now that we’re in Builder, your design has been converted into code, and what shows in the Builder canvas is a preview of the resulting code.

This is cool and all, but it's a lot cooler to make this code actually do something!

In order to be able to do something useful, we’ll want to give the AI some context about us first.

For example, if you want to use data, where should the data come from? Is it a public resource or your own private API? What inputs and outputs can this API take?

Note: The Builder Visual Editor AI chat released with Visual Copilot 2.0 is currently in a rolling beta — meaning some users will have access now, and some will come soon. We add new users each week — so if you don’t have access just yet, request to be notified when your accent gets access

For my dashboard, I used these instructions. Feel free to copy and use them, including the API used:

We have an API that can list our users that works like:

https://user-api.builder-io.workers.dev/
https://user-api.builder-io.workers.dev/?page=2
https://user-api.builder-io.workers.dev/?sortBy=name.first
https://user-api.builder-io.workers.dev/?sortBy=location.city

the output looks like:

<output>
{
  "page": 1,
  "perPage": 10,
  "total": 100,
  "data": [
    {
      "gender": "female",
      "name": { "title": "Mrs", "first": "Aada", "last": "Ojala" },
      "location": {
        "street": { "number": 9896, "name": "Pyynikintie" },
        "city": "Loviisa",
        "state": "Tavastia Proper",
        "country": "Finland",
      },
      "picture": {
        "large": "https://randomuser.me/api/portraits/women/51.jpg",
        "medium": "https://randomuser.me/api/portraits/med/women/51.jpg",
        "thumbnail": "https://randomuser.me/api/portraits/thumb/women/51.jpg"
      },
    },
    ...
  ]
}
<output>

when adding tables of users include their images too. also make tables horizontal scroll on mobile

for loading states, give things a nice spinner

Notice that at the bottom I included a couple extra tips. For instance, I want the AI to generate tables with images in them, to make them horizontally scrollable on mobile, and be sure to use a nice loading spinner.

Instructions are infinitely customizable, so experiment with them! Use trial and error to find out what works best for you. With the right approach, they are incredibly powerful.

Note that you can also instruct the AI to use well known APIs without much guidance (like specific input/output examples). Some other fun ones to use and try:

catfact.ninja: returns random cat facts
dog.ceo: random dog images by breed
pokeapi.co: Pokemon data (stats, moves, types)
jservice.io: trivia questions from the Jeopardy! game show
Open Notify: space data like the current location of the ISS and people in space.
RandomUser.me: random user data (names, photos, addresses)
Advice Slip API: random pieces of advice (https://api.adviceslip.com)
Universities API: information on universities worldwide (http://universities.hipolabs.com/)
Open Meteo: weather forecasts, historical weather, and current weather data

Can't find a public API that interests you there? Just try asking Visual Copilot, and it can recommend an API for your use case too.

Step 3: making it interactive

Creating the user table

In the case of our design, it wasn’t even complete!

But Builder can handle this as well — we can use AI to generate or update any part of the design, completely matching the existing aesthetic of the design, including components, design tokens, and so on.

In our case, you can click the Users table goes here text and enter the prompt:

replace this with a table of our users

Visually tweaking the table

We have a nice table, but let's start making some visual modifications. Builder.io is a full Visual Development Platform, so we can continue to select parts of the code and edit it visually.

For instance, we can go to the S*tyle* tab and change the width to Fill, and even drag to change the top margin size as well.

Play around all you like to get things pixel perfect.

Adding sorting

Now, we have a table connected to real data. But, how do we even know? For all we know, this was hard generated into the design.

So let's spice things up and add some real interactivity.

In our design, we have a couple buttons to choose different table sorts. Let’s click on that and use my favorite prompt:

make these buttons work

Yes, just from the context of the design and the API info we gave the system, Visual Copilot in this case can infer what these buttons should do, and hook up the functionality for us.

And voilà — we have working sort buttons!

You can try it out via the quick preview mode, for instance by hitting ⌘ + \ on mac, or Ctrl + \ on PC/Linux (yes, I mentioned Linux for all one of you).

In preview mode, you can click around your app just like a typical user. Try out your awesome new API-driven dashboard table!

Implementing pagination

Let’s now implement pagination. We’ll follow similar steps as before. Let's click the pagination buttons and use my favorite prompt:

make these buttons work

Disclaimer: A super vague prompt like that is not the general best practice, I just think it's fun that it works in this case.

Generally, be descriptive about what you want, and if the AI doesn’t get it right on the first time, give it feedback and have it try again. Working with AI is a trial and error process to get good results.

Step 4: generating production code

When you're happy with the functionality, let’s grab the code.

First, switch to Develop mode to show all of your code generation options.

You can pick between a variety of code frameworks, such as React, Vue, Angular, Svelte, React Native, SwiftUI, HTML, Qwik, Solidjs, Jetpack Compose, Flutter, and more.

For styling frameworks, built in support includes Tailwind, CSS, CSS modules, Styled Components, Emotion, and you can even add anything else not listed here using custom instructions.

Add custom instructions

You can also add completely custom instructions in natural language for any additional preferences you have — including frameworks, styling frameworks, testing frameworks not mentioned previously, as well as anything completely custom — like stylistic preferences, libraries you use, or even samples of your existing code so your code can be reused or the AI can match your coding style.

In my case, I love React Query, so I have this custom instruction saved for all code generations so that React Query is always used for data fetching:

use @tanstack/react-query for all data fetching

Generate the code

When we’ve got things setup the way we like, we can just click Generate Code to get our code.

Keep in mind you can always change code after the fact too — by chatting to make modifications, as well as by updating your custom instructions and generating again.

This process can take trial and error to get exactly right—- but is worth it to make sure the system is outputting code just like you and your team would write.

And remember, the AI can respond to nearly any kind of feedback. If there is anything that makes you say “no I would’ve written this differently”, just tell the system.

Step 5: getting the code into your project

Now what good is code if it's not in your codebase. I got u fam don’t doubt me so much.

Click Sync Code.
Copy the provided command.
Run it at the root of your project.

Yes, yes I know I need to update Cursor.

And there you have it — the dashboard code, with functionality, tests, and reusable components is yours!

Bonus: Map Your Components

This is great and all — but what if you have a design system of components already?

Ideally, we’d want to reuse those components when importing and generating, including any other components local to our codebase that we typically build with.

For instance, here is an example of code I generated using the Cloudscape design system:

Fear not, Builder supports this as well. But that is a topic for another post, but you can visit the docs in the meantime and try it out yourself.

What's Next?

Try this out on more designs, code, and APIs!

Remember: AI is a tool. It doesn’t replace engineering work, but when used right it sure as heck an accelerate it. AI will be good at some things, and not good at others. The trick is to experiment — use trial and error and find what works great for you and it can be your new secret superpower.

If you’d like more use cases and even more ideas on what you can accomplish with Visual Copilot, read Introducing Visual Copilot 2.0: Design to Interactive.

Give it a try — I can’t wait to see what you build!

Resources

Useful docs

Turn a Figma Dropdown Design Into Working React Code in 1 Minute

Steve Sewell — Tue, 12 Nov 2024 13:10:33 +0000

I just transformed a Figma design into working React code, complete with TypeScript and tests, in less than 1 minute.

The best part? The code is clean, customizable, and ready to integrate into your project.

Let me show you how to do it.

The Process

Step 1: Exporting from Figma

First, grab your design (if using mine, be sure to duplicate it to your drafts first, so you have full edit/plugins access).

Launch the **Builder.io Figma plugin** and hit the "Export to Code" button, and you'll be launched into Builder.io.

Step 2: Adding Interactivity with AI

Once in Builder, it's time to make your design interactive. Use this prompt:

when I click the button, toggle the menu and rotate the chevron

Hit enter and let the AI work its magic. It will add the interactivity for you while you sit back and relax.

Then, toggle quick preview (cmd+\ or ctrl+\) to verify that everything works as expected. You should see the menu toggle and the chevron rotate when you click the button.

Note: The Builder Visual Editor AI chat released with Visual Copilot 2.0 is currently in a rolling beta — meaning some users will have access now, and some will come soon. We add new users each week — so if you don’t have access just yet, request to be notified when your accent gets access

Step 4: Generating Production Code

When you're happy with the functionality, it's time to get the code:

Hit the "Develop" button
Click "Generate Code"
Tweak any options as needed (frameworks, styling, languages, etc.)

Builder supports many frameworks, like Vue, Svelte, Solid.js, as well as native outputs like SwiftUI, Flutter, and Jetpack Compose.

You can type to chat with the code to modify it in any way you like. You can also add custom instructions to customize even futher.

Step 5: Syncing the Code to Your Project

To get the code into your project:

Click "Sync Code"
Copy the provided command
Run it at the root of your project
And there you have it - your new React component with TypeScript and tests, ready to be committed to your repository.

And there you have it - your new React component with TypeScript and tests, ready to be committed to your repository.

What's Next?

Try this out with your own designs! Remember, AI is a powerful tool that can significantly speed up your development process, but it doesn't replace the need for engineering expertise.

Use trial and error to get a good feel for what this AI is good for. Use it to accelerate your workflow and focus on the more complex aspects of your projects.

Resources

Give it a try - I can't wait to see what you build!

Introducing Visual Copilot 2.0: Design to Interactive

Steve Sewell — Thu, 31 Oct 2024 17:46:45 +0000

We've all been there. The design looks perfect in Figma. Everyone's approved it. But then comes the hard part: turning that static design into a living, breathing feature.

Developers spend hours implementing each interaction. Designers watch their vision get lost in translation. Product managers juggle timelines as simple UI changes turn into week-long projects.

Today, we're changing that forever by bringing our AI Assistant, Visual Copilot, to the Builder Visual Editor — enabling you to make Figma designs interactive with natural language, using your actual code, data, and APIs.

Real data, real APIs, real features

But it goes way beyond basic interactions. Unlike other AI tools, Visual Copilot isn't generating static code or isolated components.

It understands your entire tech stack. From creating complete data tables to adding advanced features like sorting and pagination – all while working with your real data.

Interactive designs from natural language

Select any element and say what you want. "Turn this FAQ section design into an interactive accordion" and watch as static design content transforms into a smooth, collapsible interface with production-ready code.

No coding required, just clear instructions.

Sync design changes to code

When designers update their Figma designs, merging those changes into your customized code base can be challenging.

Visual Copilot makes it seamless – intelligently syncing design updates while preserving your code edits. One click, and changes flow straight to production, no manual reconciliation needed.

Your design system automated

Colors, typography, spacing, components – Visual Copilot understands and enforces your design system automatically.

Whether your components are from your design files or generated by Visual Copilot, they're implemented exactly as intended, maintaining your brand's look and feel.

No more generic AI designs.

Publish without bottlenecks

Update landing pages, change layouts, or modify pricing tables in minutes. Your marketing team gets pixel-perfect, on-brand results while developers focus on complex features.

Even better - you can use the AI to personalize your site or app to different audiences, and publish updates (with approval workflows of course!) in a click.

Everyone moves faster, with complete control over when changes go live.

Filling the space between design and coding tools

After seeing all these capabilities, you might wonder – what exactly is happening here? It's not exactly designing, though we're using your design tokens and Figma components. It's not exactly coding either, even though we're using your code components and APIs.

It’s a new type of activity emerging in the space between design and development.

This is a Visual Development Environment – a tool that sits in the space between design and development.

Because you can completely harness it with natural language, it’s not just for designers or developers; it’s for everyone in your organization.

The power of design, code, and business context

Unlike other AI tools that generate isolated components or demos, Visual Copilot is deeply integrated with your entire product:

Your design context: It understands your Figma components, design tokens, and documentation
Your code context: It knows your code components, coding standards, and development patterns
Your business context: It connects to your APIs, data models, and business logic

This means describing complex features in just a few words gets you exactly what your team would build – using your design system, following your standards, connected to your services.

More speed, more fun, better products

Our goal is to automate the grunt work, so you and your team can focus on the impactful work that helps you build better products, faster.

By reducing tedious tasks, we increase the time spent on enjoyable, creative work.

Faster iteration cycles mean more opportunities to test, learn, and refine, ultimately leading to superior products.

Get Started with Visual Copilot 2.0

The new Visual Editor UI is available now for all Builder users
Visual Copilot features in the Visual Editor are rolling out gradually to all customers starting today
Code Sync capabilities will be released soon