AI Pair Programming: What Studies Show About Claude and Copilot

AI pair programming means working on code with an AI model the way you would with a human partner: you describe the goal, the AI proposes code, and you review, redirect and decide. Tools such as GitHub Copilot, Cursor and Claude Code all work this way, and Anthropic's Artifacts feature adds a visible side panel where the result appears while you talk.

What Is AI Pair Programming?

Pair programming is an old practice: two developers share one task, one writing ("driving") while the other reviews and thinks ahead. AI pair programming swaps the second person for a language model, which is software trained on large amounts of text and code to predict useful next words.

In practice the model plays several roles. It can write a first draft from a plain-language request, explain code you did not write, or suggest the next few lines as you type. The human still owns the decisions: what to build, whether the result is right, and when to throw it away.

Anthropic describes Claude Code as "an agentic coding tool that reads your codebase, edits files, runs commands, and integrates with your development tools." "Agentic" means the AI takes several steps on its own, such as editing files and running tests, instead of answering one question at a time. According to Anthropic's Claude Code documentation, it runs in a terminal, in code editors such as VS Code and JetBrains, in a desktop app, and in a web browser. In practice that means you can ask it to change a file, run the tests and report back, then review what it did.

Pairing With AI vs Vibe Coding: What Is the Difference?

"Vibe coding" is a casual label for describing an app in plain words and accepting whatever the AI produces, often without reading the code.

The difference is review. In a pair, the human reads the proposed changes, runs the program and pushes back. In vibe coding, the human judges the result only by whether the app seems to work. Both use the same tools, so the line is about habits. A developer who reads each change, runs the tests and rejects weak suggestions is pairing, even with the same tool that someone else uses to generate a whole app unread.

For a throwaway prototype, skipping review may be fine. For anything that handles money, personal data or other people's work, someone has to read the code, because the model can write plausible code that is wrong.

What Are Claude Artifacts, and Where Do They Fit?

Artifacts are the visible part of the pairing in Claude's chat app. According to Anthropic's help center, an artifact is "anything Claude makes for you that you'd put in front of someone," and it opens beside your conversation, where you can edit it, return to it and share it.

The help center lists documents, code snippets, single-page websites, images, diagrams, dashboards and small interactive tools as artifact types. You can ask Claude in chat to change one, and in template artifacts such as documents you can edit directly. Artifacts start private to you.

This matters for non-programmers because the output is something you can look at and click. You can say "make the button green" and see the change. Because the artifact sits next to the chat, each request and its result stay visible together, so it is easy to see what changed after every instruction.

Availability depends on the plan. The help center says creating artifacts in a chat works on Free, Pro, Max, Team and Enterprise plans, and that the feature requires "Cloud code execution and file creation" to be switched on in settings. Artifacts also work inside Claude Code on every plan that includes it.

Does Pairing With AI Actually Make Developers Faster?

Two studies measure speed. One is a randomized experiment by researchers at Microsoft Research, GitHub and MIT. According to the paper on GitHub Copilot, developers asked to write an HTTP server in JavaScript finished 55.8% faster with Copilot than a control group, with a 95% confidence interval of 21% to 89%. That is a wide range, and it covers one task.

The other is from METR, a nonprofit that evaluates AI systems. In its 2025 study, 16 experienced open-source developers worked on 246 real issues in large codebases they knew well. With AI allowed, they took 19% longer. They had expected a 24% speedup, and afterward still believed AI had sped them up by 20%.

The Copilot test used a fresh, self-contained task. The METR test used mature projects where the developers already knew the code. AI help seems to matter more when you are new to the task or the code, and less when you are the expert.

Some of the gap between the studies may come from what counts as work. In the METR update, developers using newer agentic tools chose different kinds of tasks than before, so differences in time do not map cleanly onto differences in value. The Copilot paper measured one clearly defined task with a clear finish line, which is easier to time than open-ended project work.

Has the Picture Changed Since Early 2025?

METR followed up. In its February 2026 update, it studied 57 developers, 143 repositories and more than 800 tasks with late-2025 tools. For the original developers it estimated an 18% speedup (confidence interval from 38% faster to 9% slower). For newly recruited developers it estimated a 4% speedup (confidence interval from 15% faster to 9% slower). Both estimated ranges include zero, so the data cannot rule out no change at all.

METR itself called these unreliable signals. Many developers declined to take part because they did not want to work without AI, and some withheld tasks they did not want to do without it. METR says this likely biases the estimate downward, and calls the data "very weak evidence" for the size of any increase.

The early-2025 slowdown probably no longer describes current tools, but nobody has yet measured the new speedup well.

How Do You Pair With AI Without Fooling Yourself?

The METR result carries a warning: developers felt faster while measuring slower. A few habits help, and they come from the studies' own caveats.

Read every change before accepting it. METR noted that developers reported differences in the quality of final work between conditions.

Keep tasks small. A request you can verify in a minute is easier to trust than a request that rewrites a dozen files.

Write down the project rules. Claude Code reads a file called CLAUDE.md at the start of every session, where you can set coding standards and preferred libraries, according to Anthropic's documentation. Other tools have similar settings.

Measure your own work. Time yourself on a few tasks with and without the AI. That is the same method the studies used, at a small scale. Each of these habits puts a check between the model's output and your project, so a wrong answer is caught while it is still cheap to fix.

For related background on how AI tools connect to other software, see our explainers on computer-use AI agents and the Model Context Protocol, the open standard Anthropic's documentation says lets Claude Code read documents and update tickets in other tools.

What to Watch Next

METR said its late-2025 data was weak, so cleaner studies are the first thing to follow. Look for results on tasks of different sizes, since the two studies above disagree partly because they tested different work.

On the Artifacts side, Anthropic's help center lists templates still marked beta, including Dashboards and Motion, and sharing and data-storage features are limited to paid plans.