About Agentless
Agentless is a groundbreaking solution designed to tackle software development challenges without the need for complex agents. This innovative system works through a straightforward three-phase process: localization, repair, and patch validation, making it a powerful tool in the realm of automatic software issue resolution.
The Three Phases Explained:
-
Localization: Agentless uses a hierarchical approach to pinpoint faults. It begins by locating the issue within specific files, narrows it down to relevant classes or functions, and finally identifies precise edit locations. This meticulous process ensures that problems are effectively isolated for targeted solutions.
-
Repair: Once the problem areas are defined, Agentless creates multiple candidate patches for each bug using a simple diff format. This method allows for a variety of potential solutions to be explored, increasing the chances of finding the best fix.
-
Patch Validation: In this final phase, Agentless determines which regression tests are necessary and generates additional reproduction tests to mimic the original error. The results from these tests enable Agentless to re-rank all patches and select the optimal one for submission.
Recent Updates
Agentless has seen significant developments recently. It achieved a new milestone on October 28th, 2024, with the release of OpenAutoCoder-Agentless 1.5. Earlier, on July 1st, 2024, version 1.0 was released, marking Agentless as the leading open-source solution on the SWE-bench lite platform, solving 82 issues (27.3%) at an average cost of $0.34 per issue.
Setup
To utilize Agentless, users must follow a few setup instructions:
-
Environment Setup: Clone the repository, create a conda environment with Python 3.11, activate it, and install the necessary requirements.
git clone https://github.com/OpenAutoCoder/Agentless.git cd Agentless conda create -n agentless python=3.11 conda activate agentless pip install -r requirements.txt export PYTHONPATH=$PYTHONPATH:$(pwd) -
OpenAI API Key: Ensure to export your OpenAI API key accordingly.
export OPENAI_API_KEY={key_here}
Post setup, Agentless is ready to be deployed on the SWE-bench problems.
Comparison
Agentless stands out among open-source agent-based solutions for SWE-bench lite. It surpasses competitors in terms of efficiency and problem-solving capabilities, as evidenced by comparison graphs that highlight its superior performance.
Artifacts
For those interested in deeper insights, the complete artifacts of Agentless are available for download from the v1.5.0 release. These include:
- agentless_swebench_lite: Complete run on SWE-bench Lite
- agentless_swebench_verified: Comprehensive execution on SWE-bench Verified
- swebench_repo_structure: Structured information for each SWE-Bench problem
Additionally, manual classifications of SWE-bench-lite can be accessed in the classification/ folder, which also contains filtered SWE-bench-lite-S problems.
Citation
To reference the work related to Agentless, use the following citation:
@article{agentless,
author = {Xia, Chunqiu Steven and Deng, Yinlin and Dunn, Soren and Zhang, Lingming},
title = {Agentless: Demystifying LLM-based Software Engineering Agents},
year = {2024},
journal = {arXiv preprint},
}
Acknowledgement
Agentless has been developed with contributions from notable projects like SWE-bench, Aider, and SWE-bench-docker. These collaborations highlight the integrated approach and collective expertise that have shaped Agentless into a formidable tool in software engineering.