DEV Community

Cover image for We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.
Sunil Kumar Dash Subscriber for Composio

Posted on • Edited on

39 37 38 39 39

We made an AI SWE that solved 48.60% of issues on the SWE bench, 100% open-source.

We at Composio are building the tool infrastructure for AI agents. One of our users' biggest requests was toolkits for building custom coding agents that work. So, we created SWE-Kit, a starter template with all the toolkits for building AI coding agents.

These agents can run locally end-to-end to automate your coding workflows.

To test the efficiency of our tools, we built a comprehensive AI agent complete open-source using LangGraph and tested it on SWE-bench verified, and it got 48.60% and 41% on SWE-bench lite.

SWE bench ranking

SWE-bench is a benchmark with over 2,200 real-world Python issues from repositories such as Django, Flask, Sklearn, SciPy, etc.

SWe-kit on SWE bench GIF

Check out the technical report for more: Tool design is all you need for SOTA AI agents
Complete code for the SWE agent: SWE-kit Agent


The tools you need for running the agent locally

  • Code Analysis Tool: Intelligently retrieves relevant code snippets from the repository.
  • File Tool: Facilitates navigation and updates to files.
  • Shell Tool: Performs shell operations.
  • Git Tool: Handles version control tasks.
  • Composio Docker workspace for isolated code execution.

We optimized the tools for improved function calling accuracy.


What can you build with SWE-kit and Composio?

The code is open-source, and you can even modify it to add external integrations like GitHub, Jira, Linear, Slack, etc., using Composio to build a full-fledged AI software engineer.

You can automate many aspects of your Software development workflows with custom agents such as,

  • Writing codes
  • refactoring code bases
  • testing
  • documentation
  • Project management with Linear or Jira, etc.
  • Communication using Slack and Gmail.

For an architectural explanation of the SWE-Kit agent, check out the SWE-Kit agent blog published on LangChains’ blog.

I am not even kidding. Many companies have raised millions just from this.

Start building your custom local coding agent with SWE-kit now.

Get Started with SWE-kit

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

The Most Contextual AI Development Assistant

Pieces.app image

Our centralized storage agent works on-device, unifying various developer tools to proactively capture and enrich useful materials, streamline collaboration, and solve complex problems through a contextual understanding of your unique workflow.

👥 Ideal for solo developers, teams, and cross-company projects

Learn more

👋 Kindness is contagious

Please leave a ❤️ or a friendly comment on this post if you found it helpful!

Okay