The research challenges current beliefs about AI productivity because advanced developers took longer to complete tasks when they used AI coding tools in their familiar work environments. The research conducted by METR nonprofit organization demonstrated that professional programmers using Cursor AI assistant finished their work at a 19% slower rate than when working without the tool.
The participants who were experienced open-source developers predicted AI would enhance their task duration by 24%. The participants maintained their belief that AI tools reduced their work duration after the trial. The actual data revealed that tasks required more time than before the introduction of AI.
The study results surprised Joel Becker and Nate Rush who conducted the research. Rush revealed his expectation of a 2x speed-up before starting the trial.
The study results contradict the common assumption that AI technology always enhances engineering productivity which serves as a primary reason for AI tool investments in software development. The existing research on productivity benefits from AI tools shows mixed results because previous studies measured either entry-level tasks or unfamiliar codebases.
The METR research indicates that AI tools create obstacles instead of speed improvements when developers work in familiar codebases with high contextual understanding. The findings about AI implementation and task selection for real value delivery have significant implications for engineering teams.
