DEV Community

Cover image for We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.
Sunil Kumar Dash Subscriber for Composio

Posted on • Edited on

39 37 38 39 39

We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.

We at Composio are building the tool infrastructure for AI agents. One of our users' biggest requests was toolkits for building custom coding agents that work. So, we created SWE-Kit, a starter template with all the toolkits for building AI coding agents.

These agents can run locally end-to-end to automate your coding workflows.

To test the efficiency of our tools, we built a comprehensive AI agent complete open-source using LangGraph and tested it on SWE-bench verified, and it got 48.60% and 41% on SWE-bench lite.

SWE bench ranking

SWE-bench is a benchmark with over 2,200 real-world Python issues from repositories such as Django, Flask, Sklearn, SciPy, etc.

SWe-kit on SWE bench GIF

Check out the technical report for more: Tool design is all you need for SOTA AI agents
Complete code for the SWE agent: SWE-kit Agent


The tools you need for running the agent locally

  • Code Analysis Tool: Intelligently retrieves relevant code snippets from the repository.
  • File Tool: Facilitates navigation and updates to files.
  • Shell Tool: Performs shell operations.
  • Git Tool: Handles version control tasks.
  • Composio Docker workspace for isolated code execution.

We optimized the tools for improved function calling accuracy.


What can you build with SWE-kit and Composio?

The code is open-source, and you can even modify it to add external integrations like GitHub, Jira, Linear, Slack, etc., using Composio to build a full-fledged AI software engineer.

You can automate many aspects of your Software development workflows with custom agents such as,

  • Writing codes
  • refactoring code bases
  • testing
  • documentation
  • Project management with Linear or Jira, etc.
  • Communication using Slack and Gmail.

For an architectural explanation of the SWE-Kit agent, check out the SWE-Kit agent blog published on LangChains’ blog.

I am not even kidding. Many companies have raised millions just from this.

Start building your custom local coding agent with SWE-kit now.

Get Started with SWE-kit

Do your career a big favor. Join DEV. (The website you're on right now)

It takes one minute, it's free, and is worth it for your career.

Get started

Community matters

Top comments (0)

Heroku

This site is powered by Heroku

Heroku was created by developers, for developers. Get started today and find out why Heroku has been the platform of choice for brands like DEV for over a decade.

Sign Up

👋 Kindness is contagious

Immerse yourself in a wealth of knowledge with this piece, supported by the inclusive DEV Community—every developer, no matter where they are in their journey, is invited to contribute to our collective wisdom.

A simple “thank you” goes a long way—express your gratitude below in the comments!

Gathering insights enriches our journey on DEV and fortifies our community ties. Did you find this article valuable? Taking a moment to thank the author can have a significant impact.

Okay