# Our most productive software engineer

From a record-setting benchmark to your next AI teammate - how Lumen became our most productive engineer.

Lumen is able to solve bugs, build features, refactor code, and everything in between either fully autonomously or paired with the user, like working with a colleague, not just a copilot.

[Experience Lumen](/content/login/index.html) [Read the full report](/content/blog/genie-autonomous-software-engineer/index.html)

## What is AutoPM? - YouTube

### Bringing a supervisor model from training time to product

Introducing AutoPM.

A monumental leap in complex feature and iterative coding performance.

With AutoPM enabled Genie is closer to human-level performance than ever before. Completely outshining every other agent currently on the market.

**Human Benchmark**  
237 real world tasks worth a total of $236k  
$155,550  
72% pass rate

### Genie 2: State-of-the-art on SWE-Lancer

Breaking through the 50% barrier.

Our latest Genie 2.1 iteration manages to solve just over half the published SWE-Lancer benchmark tasks.

This represents an equivalent value of $107k of human work out of a maximum of $250k.

$ 107,750 Genie 2.1  
$ 88,250 Genie 2  
$ 65,250 o3-high  
$ 58,000 3.5 Sonnet  
$ 56,375 o4-mini-high  
$ 28,500 o1-high  
$ 25,500 o3-mini-high  
$ 14,000 GPT-4o  
$ 8,000 Genie 1

Genie 2 scores published early May 2025.

Genie 2.1 is an [improved iteration](/content/blog/genie-2-1-performance-upgrade-slack-integration/index.html) which represents the current model currently serving users.

[technical report](/content/blog/genie-autonomous-software-engineer/index.html)

### Genie 2 early internal MVP

Building Genie 2.

We took everything we learned from the dogfooding process and applied it to building Genie 2. We focused on making the model more reliable, and the product more intuitive. We incorporated user feedback to enhance the overall experience.

**Built for the post-IDE era**  
In this post-IDE era, Genie lets you assign any ticket — or even your entire backlog — and works fully asynchronously. You can return later to review, make small tweaks, and merge, all without opening your IDE. It writes unit tests, runs your CI, and when you want to collaborate in real time, Genie takes the lead while you copilot.

**Collaborate live or offload your work**  
Genie is completely headless, but even the best code needs quick, on-the-spot adjustments. That's why, in the past, when a Genie-generated pull request was 99% right, you could still feel stuck. This is why you have access to the same editor that Genie does so you can lean over its shoulder and make a change, instead of having to prompt it. It's the closest experience we've had to working with a truly asynchronous teammate. We're not claiming Genie can do everything for everyone. But it's accelerated our own team's velocity, and we know it can do the same for yours.

### Select customers onboarded to Genie 1

Vibe coding doesn't scale.

When we started using Genie 1 in real workflows, it highlighted that several of our assumptions about how the model and product were built were off. [Our benchmark scores were strong](https://openai.com/index/gpt-4o-fine-tuning/), but they didn't reflect many of the challenges that show up in real-world use.

We learned the hard way that vibe coding doesn't scale, especially when working autonomously across large, existing codebases. So our approach had to become more nuanced.

By this time Genie 1 had already become our most prolific contributor internally.

### New state-of-the-art SWE-bench score with Genie 1

The start of something big.

Last summer, we made real progress.

We achieved the [biggest score jump in SWE-Bench history](/content/blog/genie-technical-report/index.html) and figured out how to generate billions of tokens of synthetic data that mimicked human reasoning, long before reasoning models were making headlines.

We were onto something big, but we celebrated too early.

**Our SWE-Bench Full score Summer 2024**  
Results outdated, used for illustrative purposes  
30.08% Genie 1  
19.27% Factory Code Droid  
18.83% AutoCodeRover  
14.6% AppMap Navie  
13.82% Amazon Q Developer  
13.8% Devin (claimed)  
12.47% SWE-agent + GPT 4  
10.51% SWE-agent + Claude 3 Opus  
3.79% RAG + Claude 3 Opus  
3.44% Devin (actual)  
1.96% RAG + Claude 2  
1.31% RAG + GPT 4  
0.7% RAG + SWE-Llama 13B  
0.17% RAG + ChatGPT 3.5

Results since outdated, used for illustrative purposes.

Read more about Genie 1 in our original [technical report](/content/blog/genie-technical-report/index.html)

### Cosine moves from VS Code to be cloud-first

**Get started**  
Download for Mac  
macOS](https://software.cosine.sh/desktop/release/latest/cosine-desktop-darwin-arm64.dmg) [Windows](https://software.cosine.sh/desktop/release/latest/cosine-desktop-amd64-setup.exe) Linux](https://software.cosine.sh/desktop/release/latest/cosine-desktop-linux-amd64.zip)  
[Login to Cloud](/content/login/index.html)

Download the CLI  
curl -fsSL https://cosine.sh/install \| bash

**macOS**  
**Linux**  
**Windows**  
[Docs](/content/docs/cli/index.html)  
[VS Code Extension](https://marketplace.visualstudio.com/items?itemName=Cosine.cosine-vscode)  
Contact us